2020-07-18T03:15:40.463 INFO:root:teuthology version: 1.0.0-10de4c5 2020-07-18T03:15:40.466 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi/5235893 --name zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi --owner scheduled_zyan@teuthology --description fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_8} tasks/client-recovery} --verbose --block --lock -- /tmp/teuthology-worker.l91vhob0.tmp 2020-07-18T03:15:40.507 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-07-18T03:15:40.588 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi/5235893 branch: wip-mds-inode-smartptr description: fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_8} tasks/client-recovery} email: null first_in_suite: false job_id: '5235893' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: centos os_version: '8.1' overrides: admin_socket: branch: wip-mds-inode-smartptr ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - but it is still running - slow request - MDS_CLIENT_LATE_RELEASE - t responding to mclientcaps sha1: 0baf19aeafe8fa5b354a28b0fb65984f752d8b9d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 0baf19aeafe8fa5b354a28b0fb65984f752d8b9d selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mds-inode-smartptr sha1: 0baf19aeafe8fa5b354a28b0fb65984f752d8b9d owner: scheduled_zyan@teuthology priority: 10 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.1 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - client.2 - client.3 sha1: 0baf19aeafe8fa5b354a28b0fb65984f752d8b9d sleep_before_teardown: 0 suite: fs suite_branch: wip-mds-inode-smartptr suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-mds-inode-smartptr/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 0baf19aeafe8fa5b354a28b0fb65984f752d8b9d tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_recovery teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.8461 2020-07-18T03:15:40.588 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-mds-inode-smartptr/qa; will attempt to use it 2020-07-18T03:15:40.589 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-mds-inode-smartptr/qa/tasks 2020-07-18T03:15:40.589 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-07-18T03:15:40.592 INFO:teuthology.task.internal:Checking packages... 2020-07-18T03:15:40.628 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d' 2020-07-18T03:15:40.629 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-07-18T03:15:40.629 INFO:teuthology.packaging:ref: None 2020-07-18T03:15:40.630 INFO:teuthology.packaging:tag: None 2020-07-18T03:15:40.630 INFO:teuthology.packaging:branch: wip-mds-inode-smartptr 2020-07-18T03:15:40.630 INFO:teuthology.packaging:sha1: 0baf19aeafe8fa5b354a28b0fb65984f752d8b9d 2020-07-18T03:15:40.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-mds-inode-smartptr 2020-07-18T03:15:40.818 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-3461.g0baf19a 2020-07-18T03:15:40.819 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-07-18T03:15:40.834 INFO:teuthology.task.internal:no buildpackages task found 2020-07-18T03:15:40.835 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-07-18T03:15:40.871 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-07-18T03:15:40.872 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-07-18T03:15:40.935 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_zyan@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi/5235893'} 2020-07-18T03:15:40.963 DEBUG:teuthology.lock.ops:locked ubuntu@smithi047.front.sepia.ceph.com, ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:15:41.059 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f12a4807d08>, signals=[15]) 2020-07-18T03:15:41.301 INFO:teuthology.provision.fog.smithi047:Scheduling deploy of centos 8.1 2020-07-18T03:15:41.305 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of centos 8.1 2020-07-18T03:15:41.704 INFO:teuthology.orchestra.console:Power off smithi018 2020-07-18T03:15:41.706 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-07-18T03:15:41.729 INFO:teuthology.orchestra.console:Power off smithi047 2020-07-18T03:15:41.730 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-07-18T03:15:41.744 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:41.757 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:45.783 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:45.798 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:49.902 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:49.922 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:50.129 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2020-07-18T03:15:50.142 INFO:teuthology.orchestra.console:Power off for smithi047 completed 2020-07-18T03:15:50.231 INFO:teuthology.orchestra.console:Power on smithi018 2020-07-18T03:15:50.232 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-07-18T03:15:50.247 INFO:teuthology.orchestra.console:Power on smithi047 2020-07-18T03:15:50.248 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-07-18T03:15:50.262 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:50.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:54.298 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:54.318 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:58.426 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:58.442 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-07-18T03:15:58.657 INFO:teuthology.orchestra.console:Power on for smithi047 completed 2020-07-18T03:15:58.749 INFO:teuthology.orchestra.console:Power on for smithi018 completed 2020-07-18T03:15:58.760 INFO:teuthology.provision.fog.smithi047:Waiting for deploy to finish 2020-07-18T03:15:58.850 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish 2020-07-18T03:18:15.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:15.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:24.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:24.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:33.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:33.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:42.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:42.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:51.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:18:51.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:19:00.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:19:00.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:06.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:06.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:06.790 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-07-18T03:20:06.794 INFO:teuthology.orchestra.run.smithi047:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-07-18T03:20:06.918 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-07-18T03:20:06.919 INFO:teuthology.orchestra.run.smithi018:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-07-18T03:20:07.150 INFO:teuthology.provision.fog.smithi047:Node is ready 2020-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi047:> hostname 2020-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047.front.sepia.ceph.com 2020-07-18T03:20:07.167 INFO:teuthology.orchestra.run.smithi047:> grep smithi047.front.sepia.ceph.com /etc/hosts 2020-07-18T03:20:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47 smithi047.front.sepia.ceph.com smithi047 2020-07-18T03:20:07.224 INFO:teuthology.orchestra.run.smithi047:> sudo hostname smithi047 2020-07-18T03:20:07.255 INFO:teuthology.provision.fog.smithi018:Node is ready 2020-07-18T03:20:07.255 INFO:teuthology.orchestra.run.smithi018:> hostname 2020-07-18T03:20:07.272 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com 2020-07-18T03:20:07.272 INFO:teuthology.orchestra.run.smithi018:> grep smithi018.front.sepia.ceph.com /etc/hosts 2020-07-18T03:20:07.327 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018 2020-07-18T03:20:07.328 INFO:teuthology.orchestra.run.smithi018:> sudo hostname smithi018 2020-07-18T03:20:07.355 INFO:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hosts 2020-07-18T03:20:07.388 INFO:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/172.21.15.47/172.21.15.47/g' /etc/hosts 2020-07-18T03:20:07.442 INFO:teuthology.orchestra.run.smithi018:> sudo sed -i -e 's/smithi018.front.sepia.ceph.com/smithi018/g' /etc/hosts 2020-07-18T03:20:07.461 INFO:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hostname 2020-07-18T03:20:07.476 INFO:teuthology.orchestra.run.smithi018:> sudo sed -i -e 's/172.21.15.18/172.21.15.18/g' /etc/hosts 2020-07-18T03:20:07.533 INFO:teuthology.orchestra.run.smithi047:> sudo hostname smithi047 2020-07-18T03:20:07.547 INFO:teuthology.orchestra.run.smithi018:> sudo sed -i -e 's/smithi018.front.sepia.ceph.com/smithi018/g' /etc/hostname 2020-07-18T03:20:07.603 INFO:teuthology.provision.fog.smithi047:Deploy complete! 2020-07-18T03:20:07.605 DEBUG:teuthology.parallel:result is None 2020-07-18T03:20:07.619 INFO:teuthology.orchestra.run.smithi018:> sudo hostname smithi018 2020-07-18T03:20:07.690 INFO:teuthology.provision.fog.smithi018:Deploy complete! 2020-07-18T03:20:07.691 DEBUG:teuthology.parallel:result is None 2020-07-18T03:20:07.851 INFO:teuthology.lock.ops:Checking smithi047.front.sepia.ceph.com 2020-07-18T03:20:07.853 INFO:teuthology.lock.ops:New key found. Updating... 2020-07-18T03:20:07.879 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com 2020-07-18T03:20:07.880 INFO:teuthology.lock.ops:New key found. Updating... 2020-07-18T03:20:07.909 INFO:teuthology.lock.ops:Updating [smithi047.front.sepia.ceph.com]: set os type and version on server 2020-07-18T03:20:07.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-07-18T03:20:07.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:08.056 INFO:teuthology.orchestra.run.smithi047:> true 2020-07-18T03:20:08.235 INFO:teuthology.orchestra.run.smithi047:> uname -m 2020-07-18T03:20:08.308 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2020-07-18T03:20:08.309 INFO:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2020-07-18T03:20:08.380 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Linux" 2020-07-18T03:20:08.380 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8 (Core)" 2020-07-18T03:20:08.381 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2020-07-18T03:20:08.381 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2020-07-18T03:20:08.381 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8" 2020-07-18T03:20:08.381 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2020-07-18T03:20:08.381 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-07-18T03:20:08.382 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2020-07-18T03:20:08.382 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-07-18T03:20:08.383 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.centos.org/" 2020-07-18T03:20:08.383 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-07-18T03:20:08.383 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:20:08.384 INFO:teuthology.orchestra.run.smithi047.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-07-18T03:20:08.384 INFO:teuthology.orchestra.run.smithi047.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-07-18T03:20:08.384 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-07-18T03:20:08.385 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-07-18T03:20:08.385 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:20:08.387 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2020-07-18T03:20:08.415 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server 2020-07-18T03:20:08.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-07-18T03:20:08.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:08.498 INFO:teuthology.orchestra.run.smithi018:> true 2020-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi018:> uname -m 2020-07-18T03:20:08.728 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2020-07-18T03:20:08.729 INFO:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2020-07-18T03:20:08.782 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Linux" 2020-07-18T03:20:08.783 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8 (Core)" 2020-07-18T03:20:08.783 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2020-07-18T03:20:08.783 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2020-07-18T03:20:08.784 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8" 2020-07-18T03:20:08.784 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8" 2020-07-18T03:20:08.784 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-07-18T03:20:08.784 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2020-07-18T03:20:08.784 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-07-18T03:20:08.785 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.centos.org/" 2020-07-18T03:20:08.785 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-07-18T03:20:08.785 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:20:08.785 INFO:teuthology.orchestra.run.smithi018.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-07-18T03:20:08.785 INFO:teuthology.orchestra.run.smithi018.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-07-18T03:20:08.786 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-07-18T03:20:08.786 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-07-18T03:20:08.786 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:20:08.787 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2020-07-18T03:20:08.809 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-07-18T03:20:08.850 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDERrVDQBxNSbMkwE3jucLqhwEvV0gUUUnYNWwPnW2daUJ0tuGhQy2hSrPYqs8OjmFSpwTe1ZUNBWdoh8yj6aRtC01hTIDJ0d6HSG7L5pwd43Wo0/7U0SL6bDHhzAp+yslLtn+fCqK73tZbotM/x39cwRIKbCoNgbnmsr+57E7KL13H9/AJ5Anxbrn1TaikhAaOoTyiQ4wty9AyjSEfg5gtCfshn3XthIkgIpr5NCzkiUIi69vw1FJ9jNV8uPs93JQM0sx7SAiLo6qF1DRPauigmOm7my4N2tNb6B0MeeiwsXm6W6euqsYA/Z4dqPFR55+TzmQ4R2pYdWfl6n1TAWdch1tlNPFl74hqkiqebo7LlhvWH3JmQVFqAebtzWXlFrpNUQtuHfN1YCOpOXCqaNOTwjGOEdLWt7s2jn7pnfgppuQ9BVDG/CDZ1Z6bs/3We1ASZaEjM1tK6s4ZEgjzZTGqapOwDoiUOx8RLi+BhhEMKjbavChfpbDXVgHwsbunaKk= smithi047.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlfLY62OC2NvIEUH5HHA/+xaCu1+D/xCdbiIctxbMCcyVzawjI+g16O57Opk7YYwfM9lWp6oS3eAybmdD3auMlq7UjUOlid4b0ROHRN6pquEyTfqy76YEGLgy2L31o8TS78w78B2DS1tbIL4QvmmGnD4V0985+UUTZRIG7254oMoEqwEqmeBFS0FhjR34+xwj9OENwkVQT/zZKjg4/h3fH38EF59pLwYWWs7ne5aLxs1dzMbr1KtfAtvNxKv75dfBdznh1P5B5HB8fvAeV0xeIUDrbJYCAc/AF9ikGO3znvLeQVhDkhgsTGV5wp+lkFi081c4hmuy4jD2pUkjB7ouU4C/XFIRw72HoUVTn+d+jOUEhcsJsibU0jr7kTxgGbm/vG/k2JQUM93gc0Fmpx3IqttyK/dYwkq0eDQa6Q7ZMPqbBG0YAObZsXTRVwobDL75fIEJEJcQGEKyn9RjrvI9NXSXLGx/6c+iq+2++O0TFJlnWKuDXhH2U1PP5/TbhBlc= 2020-07-18T03:20:08.851 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-07-18T03:20:08.912 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-07-18T03:20:10.272 INFO:teuthology.task.internal:Saving configuration 2020-07-18T03:20:10.424 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-07-18T03:20:10.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-07-18T03:20:10.497 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-07-18 03:15:40.943783', 'description': '/home/teuthworker/archive/zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi/5235893', 'locked_by': 'scheduled_zyan@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCEGjRWYNDZmgR4FOBWhnS8DsqK4Y2iD3y1sTWDTVNR3Q90uDtS2wWNLzy8xkYvy9sO8StxFDeT7IfbXWHHljeA=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi047.front.sepia.ceph.com'} 2020-07-18T03:20:10.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-07-18 03:15:40.943821', 'description': '/home/teuthworker/archive/zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi/5235893', 'locked_by': 'scheduled_zyan@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPZxUEobWbgnLgafsd6UU+m4yYdc5GaFcqR684w6nNU8cHhcHGE6w4h/mhNpkTDUv5lS100TVTco7pv8uumPhv8=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi018.front.sepia.ceph.com'} 2020-07-18T03:20:10.516 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-07-18T03:20:10.539 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'] 2020-07-18T03:20:10.540 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3'] 2020-07-18T03:20:10.540 INFO:teuthology.run_tasks:Running task console_log... 2020-07-18T03:20:10.642 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f12a47c9bf8>, signals=[15]) 2020-07-18T03:20:10.643 INFO:teuthology.run_tasks:Running task internal.connect... 2020-07-18T03:20:10.653 INFO:teuthology.task.internal:Opening connections... 2020-07-18T03:20:10.653 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:20:10.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:10.746 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:20:10.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:20:10.832 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-07-18T03:20:10.849 INFO:teuthology.orchestra.run.smithi047:> uname -m 2020-07-18T03:20:10.880 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2020-07-18T03:20:10.881 INFO:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2020-07-18T03:20:10.935 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Linux" 2020-07-18T03:20:10.935 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8 (Core)" 2020-07-18T03:20:10.935 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2020-07-18T03:20:10.935 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2020-07-18T03:20:10.936 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8" 2020-07-18T03:20:10.936 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2020-07-18T03:20:10.936 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-07-18T03:20:10.936 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2020-07-18T03:20:10.936 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-07-18T03:20:10.937 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.centos.org/" 2020-07-18T03:20:10.937 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-07-18T03:20:10.937 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:20:10.937 INFO:teuthology.orchestra.run.smithi047.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-07-18T03:20:10.937 INFO:teuthology.orchestra.run.smithi047.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-07-18T03:20:10.938 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-07-18T03:20:10.938 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-07-18T03:20:10.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:20:10.949 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2020-07-18T03:20:10.983 INFO:teuthology.orchestra.run.smithi018:> uname -m 2020-07-18T03:20:11.000 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2020-07-18T03:20:11.001 INFO:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2020-07-18T03:20:11.054 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Linux" 2020-07-18T03:20:11.055 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8 (Core)" 2020-07-18T03:20:11.055 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2020-07-18T03:20:11.055 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2020-07-18T03:20:11.055 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8" 2020-07-18T03:20:11.055 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8" 2020-07-18T03:20:11.056 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-07-18T03:20:11.056 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2020-07-18T03:20:11.056 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-07-18T03:20:11.056 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.centos.org/" 2020-07-18T03:20:11.056 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-07-18T03:20:11.057 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:20:11.057 INFO:teuthology.orchestra.run.smithi018.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-07-18T03:20:11.057 INFO:teuthology.orchestra.run.smithi018.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-07-18T03:20:11.057 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-07-18T03:20:11.057 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-07-18T03:20:11.058 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:20:11.058 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2020-07-18T03:20:11.080 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-07-18T03:20:11.121 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-07-18T03:20:11.218 INFO:teuthology.task.internal:Checking for old test directory... 2020-07-18T03:20:11.219 INFO:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2020-07-18T03:20:11.227 INFO:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2020-07-18T03:20:11.241 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-07-18T03:20:11.261 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-07-18T03:20:11.262 INFO:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2020-07-18T03:20:11.282 INFO:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2020-07-18T03:20:11.309 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-07-18T03:20:11.364 INFO:teuthology.run_tasks:Running task kernel... 2020-07-18T03:20:11.388 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}, 'client.3': {'kdb': True, 'sha1': 'distro'}} 2020-07-18T03:20:11.389 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-07-18T03:20:11.389 INFO:teuthology.orchestra.run.smithi047:> uname -r 2020-07-18T03:20:11.404 INFO:teuthology.orchestra.run.smithi047.stdout:4.18.0-193.6.3.el8_2.x86_64 2020-07-18T03:20:11.405 INFO:teuthology.task.kernel:Running kernel on smithi047: 4.18.0-193.6.3.el8_2.x86_64 2020-07-18T03:20:11.405 INFO:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2020-07-18T03:20:13.889 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-8 - AppStream 40 MB/s | 7.0 MB 00:00 2020-07-18T03:20:15.208 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-8 - Base 16 MB/s | 2.2 MB 00:00 2020-07-18T03:20:15.662 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-8 - Extras 76 kB/s | 5.5 kB 00:00 2020-07-18T03:20:16.106 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 19 MB/s | 7.4 MB 00:00 2020-07-18T03:20:17.713 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 4.1 MB/s | 20 kB 00:00 2020-07-18T03:20:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-07-18T03:20:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-193.6.3.el8_2.x86_64 is already installed. 2020-07-18T03:20:18.870 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2020-07-18T03:20:18.871 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2020-07-18T03:20:18.872 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2020-07-18T03:20:18.989 INFO:teuthology.orchestra.run.smithi047:> echo no | sudo yum reinstall kernel || true 2020-07-18T03:20:19.418 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:02 ago on Sat 18 Jul 2020 03:20:17 AM UTC. 2020-07-18T03:20:19.675 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-07-18T03:20:19.676 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-193.6.3.el8_2.x86_64 (from BaseOS) not available. 2020-07-18T03:20:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:Error: No packages marked for reinstall. 2020-07-18T03:20:19.763 INFO:teuthology.orchestra.run.smithi047:> sudo yum reinstall -y kernel || true 2020-07-18T03:20:20.191 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:03 ago on Sat 18 Jul 2020 03:20:17 AM UTC. 2020-07-18T03:20:20.450 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-07-18T03:20:20.451 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-193.6.3.el8_2.x86_64 (from BaseOS) not available. 2020-07-18T03:20:20.481 INFO:teuthology.orchestra.run.smithi047.stderr:Error: No packages marked for reinstall. 2020-07-18T03:20:20.522 INFO:teuthology.orchestra.run.smithi047:> rpm -q kernel --last | head -n 1 2020-07-18T03:20:20.590 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-4.18.0-193.6.3.el8_2.x86_64 Thu 16 Jul 2020 06:30:18 PM UTC 2020-07-18T03:20:20.591 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.6.3.el8_2.x86_64 2020-07-18T03:20:20.592 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-07-18T03:20:20.593 INFO:teuthology.orchestra.run.smithi018:> uname -r 2020-07-18T03:20:20.615 INFO:teuthology.orchestra.run.smithi018.stdout:4.18.0-193.6.3.el8_2.x86_64 2020-07-18T03:20:20.616 INFO:teuthology.task.kernel:Running kernel on smithi018: 4.18.0-193.6.3.el8_2.x86_64 2020-07-18T03:20:20.616 INFO:teuthology.orchestra.run.smithi018:> sudo yum install -y kernel 2020-07-18T03:20:23.324 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - AppStream 36 MB/s | 7.0 MB 00:00 2020-07-18T03:20:24.585 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-07-18T03:20:25.037 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Extras 85 kB/s | 5.5 kB 00:00 2020-07-18T03:20:25.409 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 23 MB/s | 7.4 MB 00:00 2020-07-18T03:20:27.004 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 4.5 MB/s | 20 kB 00:00 2020-07-18T03:20:28.015 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-07-18T03:20:28.016 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-193.6.3.el8_2.x86_64 is already installed. 2020-07-18T03:20:28.175 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2020-07-18T03:20:28.176 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2020-07-18T03:20:28.177 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2020-07-18T03:20:28.310 INFO:teuthology.orchestra.run.smithi018:> echo no | sudo yum reinstall kernel || true 2020-07-18T03:20:28.741 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:01 ago on Sat 18 Jul 2020 03:20:27 AM UTC. 2020-07-18T03:20:28.989 INFO:teuthology.orchestra.run.smithi018.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-07-18T03:20:28.990 INFO:teuthology.orchestra.run.smithi018.stdout:Installed package kernel-4.18.0-193.6.3.el8_2.x86_64 (from BaseOS) not available. 2020-07-18T03:20:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:Error: No packages marked for reinstall. 2020-07-18T03:20:29.083 INFO:teuthology.orchestra.run.smithi018:> sudo yum reinstall -y kernel || true 2020-07-18T03:20:29.512 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:02 ago on Sat 18 Jul 2020 03:20:27 AM UTC. 2020-07-18T03:20:29.755 INFO:teuthology.orchestra.run.smithi018.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-07-18T03:20:29.755 INFO:teuthology.orchestra.run.smithi018.stdout:Installed package kernel-4.18.0-193.6.3.el8_2.x86_64 (from BaseOS) not available. 2020-07-18T03:20:29.811 INFO:teuthology.orchestra.run.smithi018.stderr:Error: No packages marked for reinstall. 2020-07-18T03:20:29.857 INFO:teuthology.orchestra.run.smithi018:> rpm -q kernel --last | head -n 1 2020-07-18T03:20:29.923 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-4.18.0-193.6.3.el8_2.x86_64 Thu 16 Jul 2020 06:30:18 PM UTC 2020-07-18T03:20:29.924 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.6.3.el8_2.x86_64 2020-07-18T03:20:29.925 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-07-18T03:20:29.925 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-07-18T03:20:29.925 INFO:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-07-18T03:20:29.968 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2020-07-18T03:20:29.987 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-07-18T03:20:29.988 INFO:teuthology.orchestra.run.smithi018:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-07-18T03:20:30.023 INFO:teuthology.orchestra.run.smithi018.stdout:ttyS1 2020-07-18T03:20:30.040 INFO:teuthology.run_tasks:Running task internal.base... 2020-07-18T03:20:30.054 INFO:teuthology.task.internal:Creating test directory... 2020-07-18T03:20:30.055 INFO:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-07-18T03:20:30.083 INFO:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-07-18T03:20:30.101 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-07-18T03:20:30.114 INFO:teuthology.run_tasks:Running task internal.archive... 2020-07-18T03:20:30.129 INFO:teuthology.task.internal:Creating archive directory... 2020-07-18T03:20:30.130 INFO:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-07-18T03:20:30.145 INFO:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-07-18T03:20:30.176 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-07-18T03:20:30.199 INFO:teuthology.task.internal:Enabling coredump saving... 2020-07-18T03:20:30.199 INFO:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-07-18T03:20:30.202 INFO:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-07-18T03:20:30.274 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-07-18T03:20:30.278 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-07-18T03:20:30.291 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-07-18T03:20:30.296 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-07-18T03:20:30.298 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-07-18T03:20:30.312 INFO:teuthology.task.internal:Configuring sudo... 2020-07-18T03:20:30.313 INFO:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-07-18T03:20:30.315 INFO:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-07-18T03:20:30.349 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-07-18T03:20:30.375 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-07-18T03:20:30.376 INFO:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-07-18T03:20:30.386 INFO:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-07-18T03:20:30.405 INFO:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-07-18T03:20:30.492 INFO:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-07-18T03:20:30.548 INFO:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-07-18T03:20:30.622 INFO:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-07-18T03:20:30.678 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-07-18T03:20:30.765 INFO:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-07-18T03:20:30.802 INFO:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-07-18T03:20:30.870 INFO:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-07-18T03:20:30.905 INFO:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-07-18T03:20:30.958 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-07-18T03:20:31.043 INFO:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2020-07-18T03:20:31.088 INFO:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2020-07-18T03:20:31.149 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-07-18T03:20:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-07-18T03:20:31.488 INFO:teuthology.run_tasks:Running task internal.timer... 2020-07-18T03:20:31.500 INFO:teuthology.task.internal:Starting timer... 2020-07-18T03:20:31.501 INFO:teuthology.run_tasks:Running task pcp... 2020-07-18T03:20:31.528 INFO:teuthology.run_tasks:Running task selinux... 2020-07-18T03:20:31.543 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-07-18T03:20:31.544 INFO:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2020-07-18T03:20:31.611 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: [ OK ] 2020-07-18T03:20:31.613 INFO:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2020-07-18T03:20:31.683 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: [ OK ] 2020-07-18T03:20:31.685 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-07-18T03:20:31.685 INFO:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2020-07-18T03:20:31.704 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2020-07-18T03:20:31.705 INFO:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2020-07-18T03:20:31.722 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2020-07-18T03:20:31.723 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi018.front.sepia.ceph.com': 'permissive'} 2020-07-18T03:20:31.723 INFO:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-07-18T03:20:31.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:20:31.782 INFO:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-07-18T03:20:31.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:20:31.817 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-07-18T03:20:31.818 INFO:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2020-07-18T03:20:31.855 INFO:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2020-07-18T03:20:31.898 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-07-18T03:20:31.912 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-07-18T03:20:31.913 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-07-18T03:20:31.959 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-07-18T03:20:31.963 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi018.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-07-18T03:20:32.433 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-07-18T03:20:33.642 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-07-18T03:20:36.217 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-07-18T03:20:36.240 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:36 +0000 (0:00:00.092) 0:00:00.092 ********* 2020-07-18T03:20:36.259 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:36 +0000 (0:00:00.019) 0:00:00.111 ********* 2020-07-18T03:20:37.812 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi018.front.sepia.ceph.com] 2020-07-18T03:20:37.856 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] 2020-07-18T03:20:37.902 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:37 +0000 (0:00:01.633) 0:00:01.744 ********* 2020-07-18T03:20:37.960 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:37 +0000 (0:00:00.068) 0:00:01.812 ********* 2020-07-18T03:20:38.425 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-07-18T03:20:38.427 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-07-18T03:20:38.463 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:38 +0000 (0:00:00.502) 0:00:02.315 ********* 2020-07-18T03:20:38.503 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:38 +0000 (0:00:00.039) 0:00:02.355 ********* 2020-07-18T03:20:39.079 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-07-18T03:20:39.081 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-07-18T03:20:39.119 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:39 +0000 (0:00:00.616) 0:00:02.971 ********* 2020-07-18T03:20:39.136 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-07-18T03:20:39.175 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:39 +0000 (0:00:00.054) 0:00:03.025 ********* 2020-07-18T03:20:39.183 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-07-18T03:20:39.185 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:39.227 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:39 +0000 (0:00:00.053) 0:00:03.079 ********* 2020-07-18T03:20:39.237 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:39.275 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:39 +0000 (0:00:00.047) 0:00:03.126 ********* 2020-07-18T03:20:39.755 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-07-18T03:20:39.757 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi018.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-07-18T03:20:39.795 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:39 +0000 (0:00:00.520) 0:00:03.647 ********* 2020-07-18T03:20:39.842 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:39 +0000 (0:00:00.043) 0:00:03.690 ********* 2020-07-18T03:20:40.908 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-07-18T03:20:40.910 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1595042439.860644-157453109200591/source", "state": "file", "uid": 0} changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1595042439.8210468-259341988588516/source", "state": "file", "uid": 0} 2020-07-18T03:20:40.955 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:40 +0000 (0:00:01.116) 0:00:04.807 ********* 2020-07-18T03:20:40.995 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:40 +0000 (0:00:00.039) 0:00:04.847 ********* 2020-07-18T03:20:41.425 INFO:teuthology.task.ansible.out: 2020-07-18T03:20:41.427 INFO:teuthology.task.ansible.out:TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCto 2020-07-18T03:20:41.430 INFO:teuthology.task.ansible.out:HDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-07-18T03:20:41.465 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-07-18T03:20:41.467 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-07-18T03:20:41.605 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-07-18T03:20:41.615 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-07-18T03:20:41.769 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:41 +0000 (0:00:00.774) 0:00:05.621 ********* 2020-07-18T03:20:41.806 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:41 +0000 (0:00:00.036) 0:00:05.658 ********* 2020-07-18T03:20:42.497 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi018.front.sepia.ceph.com] 2020-07-18T03:20:42.576 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] 2020-07-18T03:20:42.618 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-07-18T03:20:42.620 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:42 +0000 (0:00:00.812) 0:00:06.471 ********* 2020-07-18T03:20:42.700 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:42 +0000 (0:00:00.081) 0:00:06.552 ********* 2020-07-18T03:20:43.268 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi018.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.179525", "end": "2020-07-18 03:20:43.215428", "rc": 0, "start": "2020-07-18 03:20:43.035903", "stderr": "", "stderr_lines": [], "stdout": "d2c541f972f084049b707e8806b2858a9ab4fb83", "stdout_lines": ["d2c541f972f084049b707e8806b2858a9ab4fb83"]} 2020-07-18T03:20:43.329 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.629) 0:00:07.181 ********* 2020-07-18T03:20:43.340 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.209916", "end": "2020-07-18 03:20:43.239030", "rc": 0, "start": "2020-07-18 03:20:43.029114", "stderr": "", "stderr_lines": [], "stdout": "d2c541f972f084049b707e8806b2858a9ab4fb83", "stdout_lines": ["d2c541f972f084049b707e8806b2858a9ab4fb83"]} 2020-07-18T03:20:43.341 INFO:teuthology.task.ansible.out: 2020-07-18T03:20:43.387 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.058) 0:00:07.239 ********* 2020-07-18T03:20:43.584 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012137", "end": "2020-07-18 03:20:43.549670", "failed_when_result": false, "rc": 0, "start": "2020-07-18 03:20:43.537533", "stderr": "", "stderr_lines": [], "stdout": "d2c541f972f084049b707e8806b2858a9ab4fb83", "stdout_lines": ["d2c541f972f084049b707e8806b2858a9ab4fb83"]} 2020-07-18T03:20:43.642 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.249) 0:00:07.488 ********* 2020-07-18T03:20:43.648 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011290", "end": "2020-07-18 03:20:43.605886", "failed_when_result": false, "rc": 0, "start": "2020-07-18 03:20:43.594596", "stderr": "", "stderr_lines": [], "stdout": "d2c541f972f084049b707e8806b2858a9ab4fb83", "stdout_lines": ["d2c541f972f084049b707e8806b2858a9ab4fb83"]} 2020-07-18T03:20:43.687 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.050) 0:00:07.539 ********* 2020-07-18T03:20:43.703 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-07-18T03:20:43.705 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-07-18T03:20:43.746 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.059) 0:00:07.599 ********* 2020-07-18T03:20:43.756 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-07-18T03:20:43.800 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.053) 0:00:07.652 ********* 2020-07-18T03:20:43.811 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-07-18T03:20:43.814 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:43.847 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.046) 0:00:07.699 ********* 2020-07-18T03:20:43.858 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:43.895 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.047) 0:00:07.747 ********* 2020-07-18T03:20:43.904 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-07-18T03:20:43.914 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:43.943 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:43 +0000 (0:00:00.048) 0:00:07.795 ********* 2020-07-18T03:20:43.962 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.005 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.060) 0:00:07.856 ********* 2020-07-18T03:20:44.018 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-07-18T03:20:44.020 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.064 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.058) 0:00:07.915 ********* 2020-07-18T03:20:44.073 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.110 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.047) 0:00:07.962 ********* 2020-07-18T03:20:44.125 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.167 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.056) 0:00:08.019 ********* 2020-07-18T03:20:44.178 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.217 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.049) 0:00:08.069 ********* 2020-07-18T03:20:44.227 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-07-18T03:20:44.228 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.265 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.048) 0:00:08.117 ********* 2020-07-18T03:20:44.275 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.326 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:44 +0000 (0:00:00.059) 0:00:08.177 ********* 2020-07-18T03:20:44.346 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-07-18T03:20:44.348 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.358 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.361 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.368 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.370 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.380 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.391 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.393 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.403 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.405 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmV 2020-07-18T03:20:44.406 INFO:teuthology.task.ansible.out:f2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.415 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.417 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.427 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.438 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.440 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae9 2020-07-18T03:20:44.443 INFO:teuthology.task.ansible.out:8477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.451 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.463 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.465 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjam 2020-07-18T03:20:44.467 INFO:teuthology.task.ansible.out:in@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.470 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.487 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.487 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.488 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.490 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.501 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.503 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 2020-07-18T03:20:44.505 INFO:teuthology.task.ansible.out: C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.519 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a63 2020-07-18T03:20:44.521 INFO:teuthology.task.ansible.out:38e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.535 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.537 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.542 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.544 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.545 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name' 2020-07-18T03:20:44.546 INFO:teuthology.task.ansible.out:: 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.557 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.559 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.569 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.571 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.581 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.582 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162 2020-07-18T03:20:44.584 INFO:teuthology.task.ansible.out:\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.594 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.606 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.615 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.632 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.634 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.635 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eK 2020-07-18T03:20:44.636 INFO:teuthology.task.ansible.out:TfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_lo 2020-07-18T03:20:44.638 INFO:teuthology.task.ansible.out:op_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.639 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.639 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name 2020-07-18T03:20:44.641 INFO:teuthology.task.ansible.out:': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9 2020-07-18T03:20:44.643 INFO:teuthology.task.ansible.out:qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.652 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.663 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.665 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.690 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.692 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.693 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c 2020-07-18T03:20:44.694 INFO:teuthology.task.ansible.out:0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.695 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': ' 2020-07-18T03:20:44.696 INFO:teuthology.task.ansible.out:vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.705 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.707 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.708 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.724 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.726 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.737 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.739 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.750 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.761 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.763 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'o 2020-07-18T03:20:44.765 INFO:teuthology.task.ansible.out:vpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.776 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.786 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.795 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.797 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.808 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.810 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "ad 2020-07-18T03:20:44.812 INFO:teuthology.task.ansible.out:min1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.829 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.831 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c02 2020-07-18T03:20:44.833 INFO:teuthology.task.ansible.out:37a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.842 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.844 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.855 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.856 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.874 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.877 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.897 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.899 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.910 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.913 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.931 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.933 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item" 2020-07-18T03:20:44.935 INFO:teuthology.task.ansible.out:, "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.942 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.944 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon iDR2L3LMuqFG10t1dKdXtA 61f19801e4ab7abb7597df1606036f11b0cf6a27df853dca2df8f0bb071318e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon iDR2L3LMuqFG10t1dKdXtA 61f19801e4ab7abb7597df1606036f11b0cf6a27df853dca2df8f0bb071318e1"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.957 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.958 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.959 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.959 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.960 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ssesha 2020-07-18T03:20:44.963 INFO:teuthology.task.ansible.out:sa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.965 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.966 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.966 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.967 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false 2020-07-18T03:20:44.968 INFO:teuthology.task.ansible.out:, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.978 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.990 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.992 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:44.993 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon iDR2L3LMuqFG10t1dKdXtA 61f19801e4ab7abb7597df1606036f11b0cf6a27df853dca2df8f0bb071318e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon iDR2L3LMuqFG10t1dKdXtA 61f19801e4ab7abb7597df1606036f11b0cf6a27df853dca2df8f0bb071318e1"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.015 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.017 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.027 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.041 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.043 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.054 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.056 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.066 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.068 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.070 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.072 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.078 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khir 2020-07-18T03:20:45.080 INFO:teuthology.task.ansible.out:emat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.080 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.082 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.083 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.083 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.098 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.100 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.101 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.101 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.104 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.106 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.107 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.119 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.121 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.143 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.145 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.147 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.159 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.162 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.163 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.165 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.176 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.192 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.194 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.195 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.238 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.908) 0:00:09.085 ********* 2020-07-18T03:20:45.245 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.252 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (i 2020-07-18T03:20:45.254 INFO:teuthology.task.ansible.out:tem={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.272 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-07-18T03:20:45.308 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.074) 0:00:09.160 ********* 2020-07-18T03:20:45.352 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.044) 0:00:09.204 ********* 2020-07-18T03:20:45.369 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.415 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.063) 0:00:09.267 ********* 2020-07-18T03:20:45.427 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-07-18T03:20:45.472 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.056) 0:00:09.324 ********* 2020-07-18T03:20:45.489 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.523 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.050) 0:00:09.375 ********* 2020-07-18T03:20:45.532 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-07-18T03:20:45.543 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.580 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.057) 0:00:09.433 ********* 2020-07-18T03:20:45.591 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-07-18T03:20:45.593 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.631 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.050) 0:00:09.484 ********* 2020-07-18T03:20:45.642 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-07-18T03:20:45.644 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.679 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.047) 0:00:09.531 ********* 2020-07-18T03:20:45.691 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-07-18T03:20:45.693 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.727 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.048) 0:00:09.579 ********* 2020-07-18T03:20:45.736 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-07-18T03:20:45.748 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.778 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.047) 0:00:09.627 ********* 2020-07-18T03:20:45.784 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-07-18T03:20:45.786 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.822 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.046) 0:00:09.673 ********* 2020-07-18T03:20:45.830 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-07-18T03:20:45.832 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.866 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.044) 0:00:09.718 ********* 2020-07-18T03:20:45.876 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-07-18T03:20:45.878 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.891 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.904 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi018.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.952 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:45 +0000 (0:00:00.085) 0:00:09.804 ********* 2020-07-18T03:20:45.965 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.969 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.970 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.972 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.972 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.984 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.986 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:45.997 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.011 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.013 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.023 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.025 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.081 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.083 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.085 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.088 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.101 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.103 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.104 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.115 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.117 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.118 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.118 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.119 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.121 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.122 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.122 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.123 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.123 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.135 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.137 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.155 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.158 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.160 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.198 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:46 +0000 (0:00:00.245) 0:00:10.050 ********* 2020-07-18T03:20:46.218 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.220 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.230 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-07-18T03:20:46.279 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:46 +0000 (0:00:00.080) 0:00:10.131 ********* 2020-07-18T03:20:47.126 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-07-18T03:20:47.128 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "c6bfe134f92d5cc325c43b188dcccb7cdd63c6be", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "c6bfe134f92d5cc325c43b188dcccb7cdd63c6be", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-07-18T03:20:47.165 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.886) 0:00:11.017 ********* 2020-07-18T03:20:47.221 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.055) 0:00:11.073 ********* 2020-07-18T03:20:47.231 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-07-18T03:20:47.233 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => { "msg": "Host smithi018.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-07-18T03:20:47.276 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.055) 0:00:11.128 ********* 2020-07-18T03:20:47.291 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => { "msg": "Host smithi047.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-07-18T03:20:47.335 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.058) 0:00:11.187 ********* 2020-07-18T03:20:47.349 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-07-18T03:20:47.351 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-07-18T03:20:47.397 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.061) 0:00:11.249 ********* 2020-07-18T03:20:47.407 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-07-18T03:20:47.446 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.048) 0:00:11.298 ********* 2020-07-18T03:20:47.455 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-07-18T03:20:47.457 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:47.495 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.049) 0:00:11.347 ********* 2020-07-18T03:20:47.510 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:47.548 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:47 +0000 (0:00:00.052) 0:00:11.400 ********* 2020-07-18T03:20:48.063 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.337516", "end": "2020-07-18 03:20:48.037964", "rc": 0, "start": "2020-07-18 03:20:47.700448", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-07-18T03:20:48.098 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.550) 0:00:11.951 ********* 2020-07-18T03:20:48.100 INFO:teuthology.task.ansible.out: 2020-07-18T03:20:48.134 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.316383", "end": "2020-07-18 03:20:48.089263", "rc": 0, "start": "2020-07-18 03:20:47.772880", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-07-18T03:20:48.174 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.075) 0:00:12.027 ********* 2020-07-18T03:20:48.186 INFO:teuthology.task.ansible.out: 2020-07-18T03:20:48.195 INFO:teuthology.task.ansible.out:TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-07-18T03:20:48.197 INFO:teuthology.task.ansible.out: skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.238 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.063) 0:00:12.090 ********* 2020-07-18T03:20:48.251 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.302 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.063) 0:00:12.154 ********* 2020-07-18T03:20:48.313 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-07-18T03:20:48.315 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.351 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.049) 0:00:12.204 ********* 2020-07-18T03:20:48.362 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.412 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.060) 0:00:12.264 ********* 2020-07-18T03:20:48.423 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.461 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.048) 0:00:12.313 ********* 2020-07-18T03:20:48.471 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.507 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.046) 0:00:12.359 ********* 2020-07-18T03:20:48.516 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-07-18T03:20:48.518 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.554 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.046) 0:00:12.406 ********* 2020-07-18T03:20:48.564 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.621 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.066) 0:00:12.473 ********* 2020-07-18T03:20:48.632 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-07-18T03:20:48.634 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.682 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.060) 0:00:12.533 ********* 2020-07-18T03:20:48.692 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.739 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.057) 0:00:12.591 ********* 2020-07-18T03:20:48.757 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.807 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.068) 0:00:12.659 ********* 2020-07-18T03:20:48.820 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:48.874 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.066) 0:00:12.726 ********* 2020-07-18T03:20:48.895 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-07-18T03:20:48.898 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi018.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:48.944 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:48 +0000 (0:00:00.070) 0:00:12.797 ********* 2020-07-18T03:20:48.962 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi047.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:49.003 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.058) 0:00:12.855 ********* 2020-07-18T03:20:49.023 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-07-18T03:20:49.028 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.074 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.071) 0:00:12.927 ********* 2020-07-18T03:20:49.087 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.145 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.069) 0:00:12.996 ********* 2020-07-18T03:20:49.156 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-07-18T03:20:49.158 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.202 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.057) 0:00:13.054 ********* 2020-07-18T03:20:49.226 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.277 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.074) 0:00:13.129 ********* 2020-07-18T03:20:49.288 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-07-18T03:20:49.289 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:49.326 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.049) 0:00:13.178 ********* 2020-07-18T03:20:49.339 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:49.377 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.050) 0:00:13.229 ********* 2020-07-18T03:20:49.388 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-07-18T03:20:49.390 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.436 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.059) 0:00:13.288 ********* 2020-07-18T03:20:49.447 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.484 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.047) 0:00:13.336 ********* 2020-07-18T03:20:49.494 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-07-18T03:20:49.496 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:49.531 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.047) 0:00:13.384 ********* 2020-07-18T03:20:49.550 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:49.594 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.062) 0:00:13.446 ********* 2020-07-18T03:20:49.605 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-07-18T03:20:49.607 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.657 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.062) 0:00:13.509 ********* 2020-07-18T03:20:49.668 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.705 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.047) 0:00:13.557 ********* 2020-07-18T03:20:49.719 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.757 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.051) 0:00:13.609 ********* 2020-07-18T03:20:49.776 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.818 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.061) 0:00:13.670 ********* 2020-07-18T03:20:49.837 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.867 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.048) 0:00:13.719 ********* 2020-07-18T03:20:49.877 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.925 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.058) 0:00:13.777 ********* 2020-07-18T03:20:49.935 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-07-18T03:20:49.937 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:49.981 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:49 +0000 (0:00:00.055) 0:00:13.833 ********* 2020-07-18T03:20:49.991 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.031 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.050) 0:00:13.883 ********* 2020-07-18T03:20:50.042 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-07-18T03:20:50.050 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:50.079 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.048) 0:00:13.931 ********* 2020-07-18T03:20:50.089 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:50.127 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.047) 0:00:13.979 ********* 2020-07-18T03:20:50.136 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-07-18T03:20:50.138 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.173 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.046) 0:00:14.025 ********* 2020-07-18T03:20:50.185 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.224 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.050) 0:00:14.076 ********* 2020-07-18T03:20:50.234 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-07-18T03:20:50.237 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.272 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.047) 0:00:14.124 ********* 2020-07-18T03:20:50.283 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.320 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.047) 0:00:14.172 ********* 2020-07-18T03:20:50.330 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-07-18T03:20:50.332 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.378 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.057) 0:00:14.230 ********* 2020-07-18T03:20:50.387 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.431 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.053) 0:00:14.283 ********* 2020-07-18T03:20:50.441 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-07-18T03:20:50.443 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.485 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.053) 0:00:14.337 ********* 2020-07-18T03:20:50.496 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.537 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.051) 0:00:14.389 ********* 2020-07-18T03:20:50.548 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-07-18T03:20:50.550 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.589 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.051) 0:00:14.441 ********* 2020-07-18T03:20:50.601 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.642 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.052) 0:00:14.494 ********* 2020-07-18T03:20:50.660 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-07-18T03:20:50.663 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.704 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.062) 0:00:14.556 ********* 2020-07-18T03:20:50.716 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.754 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.049) 0:00:14.606 ********* 2020-07-18T03:20:50.764 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi018.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:50.804 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.049) 0:00:14.656 ********* 2020-07-18T03:20:50.814 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-07-18T03:20:50.851 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.047) 0:00:14.703 ********* 2020-07-18T03:20:50.870 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.904 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.053) 0:00:14.757 ********* 2020-07-18T03:20:50.915 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:50.967 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:50 +0000 (0:00:00.055) 0:00:14.812 ********* 2020-07-18T03:20:50.971 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-07-18T03:20:50.972 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:51.028 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:51 +0000 (0:00:00.068) 0:00:14.880 ********* 2020-07-18T03:20:51.040 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:51.098 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:51 +0000 (0:00:00.070) 0:00:14.950 ********* 2020-07-18T03:20:51.288 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi018.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-07-18T03:20:51.352 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:51 +0000 (0:00:00.253) 0:00:15.204 ********* 2020-07-18T03:20:51.370 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-07-18T03:20:51.409 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:51 +0000 (0:00:00.056) 0:00:15.261 ********* 2020-07-18T03:20:51.952 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-07-18T03:20:51.954 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-07-18T03:20:51.986 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-07-18T03:20:52.474 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-07-18T03:20:52.550 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:52 +0000 (0:00:01.141) 0:00:16.402 ********* 2020-07-18T03:20:52.573 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-07-18T03:20:52.631 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:52 +0000 (0:00:00.081) 0:00:16.483 ********* 2020-07-18T03:20:53.171 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi018.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-07-18T03:20:53.242 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-07-18T03:20:53.776 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-07-18T03:20:53.813 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:53 +0000 (0:00:01.181) 0:00:17.665 ********* 2020-07-18T03:20:53.835 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-07-18T03:20:53.874 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:53 +0000 (0:00:00.061) 0:00:17.726 ********* 2020-07-18T03:20:53.886 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-07-18T03:20:53.889 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:53.949 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:53 +0000 (0:00:00.075) 0:00:17.801 ********* 2020-07-18T03:20:53.961 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.001 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:53 +0000 (0:00:00.051) 0:00:17.853 ********* 2020-07-18T03:20:54.011 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-07-18T03:20:54.013 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.057 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.056) 0:00:17.909 ********* 2020-07-18T03:20:54.074 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.119 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.060) 0:00:17.970 ********* 2020-07-18T03:20:54.134 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.179 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.061) 0:00:18.031 ********* 2020-07-18T03:20:54.193 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.246 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.064) 0:00:18.096 ********* 2020-07-18T03:20:54.255 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-07-18T03:20:54.258 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.295 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.051) 0:00:18.147 ********* 2020-07-18T03:20:54.305 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.348 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.053) 0:00:18.200 ********* 2020-07-18T03:20:54.357 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-07-18T03:20:54.359 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.396 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.047) 0:00:18.247 ********* 2020-07-18T03:20:54.405 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.443 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.048) 0:00:18.295 ********* 2020-07-18T03:20:54.453 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-07-18T03:20:54.455 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.491 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.047) 0:00:18.343 ********* 2020-07-18T03:20:54.503 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.550 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.054) 0:00:18.397 ********* 2020-07-18T03:20:54.558 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.604 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.057) 0:00:18.455 ********* 2020-07-18T03:20:54.613 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.652 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.049) 0:00:18.504 ********* 2020-07-18T03:20:54.665 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-07-18T03:20:54.667 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.704 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.051) 0:00:18.556 ********* 2020-07-18T03:20:54.714 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.760 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.056) 0:00:18.612 ********* 2020-07-18T03:20:54.769 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-07-18T03:20:54.771 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.811 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.050) 0:00:18.663 ********* 2020-07-18T03:20:54.822 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:54.875 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:54 +0000 (0:00:00.064) 0:00:18.728 ********* 2020-07-18T03:20:55.527 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-07-18T03:20:55.529 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.425594", "end": "2020-07-18 03:20:55.500356", "rc": 0, "start": "2020-07-18 03:20:55.074762", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.483617", "end": "2020-07-18 03:20:55.500339", "rc": 0, "start": "2020-07-18 03:20:55.016722", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:20:55.576 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:55 +0000 (0:00:00.700) 0:00:19.428 ********* 2020-07-18T03:20:55.616 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:55 +0000 (0:00:00.039) 0:00:19.468 ********* 2020-07-18T03:20:56.940 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-07-18T03:20:56.942 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-07-18T03:20:56.980 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:56 +0000 (0:00:01.364) 0:00:20.833 ********* 2020-07-18T03:20:57.004 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-07-18T03:20:57.047 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.066) 0:00:20.899 ********* 2020-07-18T03:20:57.059 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-07-18T03:20:57.061 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.098 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.050) 0:00:20.950 ********* 2020-07-18T03:20:57.108 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.147 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.049) 0:00:20.999 ********* 2020-07-18T03:20:57.158 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-07-18T03:20:57.160 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.210 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.062) 0:00:21.062 ********* 2020-07-18T03:20:57.221 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.270 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.053) 0:00:21.115 ********* 2020-07-18T03:20:57.277 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-07-18T03:20:57.278 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.319 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.055) 0:00:21.171 ********* 2020-07-18T03:20:57.333 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.377 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.058) 0:00:21.229 ********* 2020-07-18T03:20:57.402 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-07-18T03:20:57.404 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.446 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.069) 0:00:21.298 ********* 2020-07-18T03:20:57.458 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:20:57.499 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:57 +0000 (0:00:00.052) 0:00:21.351 ********* 2020-07-18T03:20:58.026 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-07-18T03:20:58.028 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-07-18T03:20:58.078 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:58 +0000 (0:00:00.577) 0:00:21.929 ********* 2020-07-18T03:20:58.088 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-07-18T03:20:58.124 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:58 +0000 (0:00:00.046) 0:00:21.976 ********* 2020-07-18T03:20:59.043 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-07-18T03:20:59.045 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-07-18T03:20:59.082 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:59 +0000 (0:00:00.958) 0:00:22.934 ********* 2020-07-18T03:20:59.113 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-07-18T03:20:59.156 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:20:59 +0000 (0:00:00.074) 0:00:23.008 ********* 2020-07-18T03:20:59.646 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-07-18T03:20:59.648 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-07-18T03:20:59.745 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-07-18T03:21:00.158 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-07-18T03:21:00.305 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-07-18T03:21:00.703 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-07-18T03:21:00.743 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:00 +0000 (0:00:01.587) 0:00:24.596 ********* 2020-07-18T03:21:00.816 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-07-18T03:21:00.819 INFO:teuthology.task.ansible.out: 2020-07-18T03:21:00.878 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:00 +0000 (0:00:00.134) 0:00:24.730 ********* 2020-07-18T03:21:00.985 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-07-18T03:21:00.987 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-07-18T03:21:01.032 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:01 +0000 (0:00:00.154) 0:00:24.884 ********* 2020-07-18T03:21:01.122 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-07-18T03:21:01.163 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:01 +0000 (0:00:00.130) 0:00:25.015 ********* 2020-07-18T03:21:01.610 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-07-18T03:21:01.612 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-07-18T03:21:01.722 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-07-18T03:21:02.147 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-07-18T03:21:02.245 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-07-18T03:21:02.648 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-07-18T03:21:02.693 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:02 +0000 (0:00:01.530) 0:00:26.545 ********* 2020-07-18T03:21:02.757 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-07-18T03:21:02.759 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:02.801 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:02 +0000 (0:00:00.107) 0:00:26.653 ********* 2020-07-18T03:21:02.811 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-07-18T03:21:02.813 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-07-18T03:21:02.858 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:02 +0000 (0:00:00.057) 0:00:26.710 ********* 2020-07-18T03:21:02.869 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-07-18T03:21:02.871 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-07-18T03:21:02.934 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:02 +0000 (0:00:00.075) 0:00:26.786 ********* 2020-07-18T03:21:02.947 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-07-18T03:21:02.949 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.006 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:03 +0000 (0:00:00.071) 0:00:26.858 ********* 2020-07-18T03:21:03.062 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-07-18T03:21:03.064 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-07-18T03:21:03.105 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:03 +0000 (0:00:00.099) 0:00:26.957 ********* 2020-07-18T03:21:03.173 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-07-18T03:21:03.175 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-07-18T03:21:03.210 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:03 +0000 (0:00:00.104) 0:00:27.062 ********* 2020-07-18T03:21:03.289 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-07-18T03:21:03.291 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.303 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.305 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.315 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.380 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:03 +0000 (0:00:00.169) 0:00:27.232 ********* 2020-07-18T03:21:03.390 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-07-18T03:21:03.392 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-07-18T03:21:03.461 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:03 +0000 (0:00:00.080) 0:00:27.313 ********* 2020-07-18T03:21:03.538 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-07-18T03:21:03.540 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.563 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.574 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:03.614 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:03 +0000 (0:00:00.153) 0:00:27.466 ********* 2020-07-18T03:21:04.392 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-07-18T03:21:04.440 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:04 +0000 (0:00:00.825) 0:00:28.292 ********* 2020-07-18T03:21:04.491 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-07-18T03:21:04.492 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:04.546 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:04 +0000 (0:00:00.105) 0:00:28.398 ********* 2020-07-18T03:21:04.626 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-07-18T03:21:04.629 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-07-18T03:21:04.678 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:04 +0000 (0:00:00.131) 0:00:28.530 ********* 2020-07-18T03:21:04.728 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-07-18T03:21:04.731 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:04.786 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:04 +0000 (0:00:00.097) 0:00:28.627 ********* 2020-07-18T03:21:05.177 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-07-18T03:21:05.260 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:05 +0000 (0:00:00.484) 0:00:29.112 ********* 2020-07-18T03:21:05.389 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-07-18T03:21:05.437 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:05 +0000 (0:00:00.176) 0:00:29.288 ********* 2020-07-18T03:21:05.878 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-07-18T03:21:05.880 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-07-18T03:21:05.918 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:05 +0000 (0:00:00.481) 0:00:29.770 ********* 2020-07-18T03:21:05.975 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi047.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:06.013 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.094) 0:00:29.864 ********* 2020-07-18T03:21:06.028 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-07-18T03:21:06.030 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-07-18T03:21:06.078 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.065) 0:00:29.930 ********* 2020-07-18T03:21:06.139 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-07-18T03:21:06.141 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:06.151 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-07-18T03:21:06.190 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.112) 0:00:30.042 ********* 2020-07-18T03:21:06.282 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-07-18T03:21:06.284 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-07-18T03:21:06.322 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.131) 0:00:30.174 ********* 2020-07-18T03:21:06.421 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-07-18T03:21:06.462 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.139) 0:00:30.314 ********* 2020-07-18T03:21:06.612 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-07-18T03:21:06.614 INFO:teuthology.task.ansible.out:fatal: [smithi047.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-07-18T03:21:06.648 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.186) 0:00:30.500 ********* 2020-07-18T03:21:06.689 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-07-18T03:21:06.691 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:06.737 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.089) 0:00:30.589 ********* 2020-07-18T03:21:06.758 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-07-18T03:21:06.766 INFO:teuthology.task.ansible.out:fatal: [smithi018.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-07-18T03:21:06.805 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.067) 0:00:30.657 ********* 2020-07-18T03:21:06.853 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-07-18T03:21:06.855 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:06.888 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:06 +0000 (0:00:00.083) 0:00:30.740 ********* 2020-07-18T03:21:07.320 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-07-18T03:21:07.322 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-07-18T03:21:07.380 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:07 +0000 (0:00:00.491) 0:00:31.232 ********* 2020-07-18T03:21:07.507 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-07-18T03:21:07.546 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:07 +0000 (0:00:00.165) 0:00:31.398 ********* 2020-07-18T03:21:08.409 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-07-18T03:21:08.410 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ActiveEnterTimestampMonotonic": "20001676", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice basic.target systemd-journald.socket nss-lookup.target network.target systemd-tmpfiles-setup.service -.mount var-run.mount sysinit.target tmp.mount local-fs.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-07-18 03:19:34 UTC", "AssertTimestampMonotonic": "20000780", "Before": "multi-user.target plymouth-quit.service getty@tty1.service shutdown.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ConditionTimestampMonotonic": "20000780", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-07-18T03:21:08.415 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2515", "ExecMainStartTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ExecMainStartTimestampMonotonic": "20001651", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[Sat 2020-07-18 03:19:34 UTC] ; stop_time=[n/a] ; pid=2515 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-07-18 03:19:34 UTC", "InactiveExitTimestampMonotonic": "20001676", "InvocationID": "9afb3bde9d92443ab05ace6140c6ffba", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "Limi 2020-07-18T03:21:08.417 INFO:teuthology.task.ansible.out:tDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2515", "MemoryAccounting": "yes", "MemoryCurrent": "1228800", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/run/nrpe /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "n 2020-07-18T03:21:08.419 INFO:teuthology.task.ansible.out:rpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-07-18 03:19:34 UTC", "StateChangeTimestampMonotonic": "20001676", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-07-18 03:19:34 UTC", "WatchdogTimestampMonotonic": "20001675", "WatchdogUSec": "0"}} 2020-07-18T03:21:08.454 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:08 +0000 (0:00:00.905) 0:00:32.304 ********* 2020-07-18T03:21:08.466 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ActiveEnterTimestampMonotonic": "19578480", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "tmp.mount nss-lookup.target systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket local-fs.target network.target basic.target -.mount time-sync.target var-run.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-07-18 03:19:34 UTC", "AssertTimestampMonotonic": "19577707", "Before": "shutdown.target multi-user.target plymouth-quit.service getty@tty1.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ConditionTimestampMonotonic": "19577707", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-07-18T03:21:08.468 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2485", "ExecMainStartTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ExecMainStartTimestampMonotonic": "19578446", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[Sat 2020-07-18 03:19:34 UTC] ; stop_time=[n/a] ; pid=2485 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-07-18 03:19:34 UTC", "InactiveExitTimestampMonotonic": "19578480", "InvocationID": "ad9b89e9e89b43bab9b8ad4d6c1ad26b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "Limi 2020-07-18T03:21:08.470 INFO:teuthology.task.ansible.out:tDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2485", "MemoryAccounting": "yes", "MemoryCurrent": "1187840", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "n 2020-07-18T03:21:08.472 INFO:teuthology.task.ansible.out:rpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-07-18 03:19:34 UTC", "StateChangeTimestampMonotonic": "19578480", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-07-18 03:19:34 UTC", "WatchdogTimestampMonotonic": "19578479", "WatchdogUSec": "0"}} 2020-07-18T03:21:08.506 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:08 +0000 (0:00:00.054) 0:00:32.358 ********* 2020-07-18T03:21:08.737 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-07-18T03:21:08.739 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001740", "end": "2020-07-18 03:21:08.709256", "rc": 0, "start": "2020-07-18 03:21:08.707516", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-07-18T03:21:08.777 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:08 +0000 (0:00:00.270) 0:00:32.629 ********* 2020-07-18T03:21:08.790 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001845", "end": "2020-07-18 03:21:08.759919", "rc": 0, "start": "2020-07-18 03:21:08.758074", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-07-18T03:21:08.831 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:08 +0000 (0:00:00.054) 0:00:32.683 ********* 2020-07-18T03:21:09.768 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-07-18T03:21:09.770 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-07-18T03:21:09.804 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:09 +0000 (0:00:00.972) 0:00:33.656 ********* 2020-07-18T03:21:09.819 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-07-18T03:21:09.889 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:09 +0000 (0:00:00.084) 0:00:33.740 ********* 2020-07-18T03:21:10.902 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-07-18T03:21:10.905 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-07-18T03:21:10.938 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:10 +0000 (0:00:01.049) 0:00:34.790 ********* 2020-07-18T03:21:10.954 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-07-18T03:21:10.995 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:10 +0000 (0:00:00.057) 0:00:34.847 ********* 2020-07-18T03:21:12.467 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-07-18T03:21:12.554 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:12 +0000 (0:00:01.557) 0:00:36.405 ********* 2020-07-18T03:21:12.569 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-07-18T03:21:12.626 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:12 +0000 (0:00:00.073) 0:00:36.478 ********* 2020-07-18T03:21:29.853 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-07-18T03:21:29.857 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:17.025680", "end": "2020-07-18 03:21:29.813196", "failed_when_result": false, "rc": 0, "start": "2020-07-18 03:21:12.787516", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-07-18T03:21:29.934 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:29 +0000 (0:00:17.307) 0:00:53.786 ********* 2020-07-18T03:21:30.062 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:17.120166", "end": "2020-07-18 03:21:29.995098", "failed_when_result": false, "rc": 0, "start": "2020-07-18 03:21:12.874932", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-07-18T03:21:30.130 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:30 +0000 (0:00:00.195) 0:00:53.982 ********* 2020-07-18T03:21:30.640 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-07-18T03:21:30.642 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-07-18T03:21:30.694 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:30 +0000 (0:00:00.563) 0:00:54.546 ********* 2020-07-18T03:21:30.797 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-07-18T03:21:30.850 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:30 +0000 (0:00:00.156) 0:00:54.702 ********* 2020-07-18T03:21:30.961 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-07-18T03:21:30.964 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.013165", "end": "2020-07-18 03:21:30.935401", "rc": 0, "start": "2020-07-18 03:21:30.922236", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:31.011 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:31 +0000 (0:00:00.160) 0:00:54.863 ********* 2020-07-18T03:21:31.138 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016662", "end": "2020-07-18 03:21:31.095075", "rc": 0, "start": "2020-07-18 03:21:31.078413", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:31.214 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:31 +0000 (0:00:00.202) 0:00:55.066 ********* 2020-07-18T03:21:31.217 INFO:teuthology.task.ansible.out: 2020-07-18T03:21:31.254 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-07-18T03:21:31.256 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.010798", "end": "2020-07-18 03:21:31.226419", "rc": 0, "start": "2020-07-18 03:21:31.215621", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:31.306 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:31 +0000 (0:00:00.091) 0:00:55.158 ********* 2020-07-18T03:21:31.456 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.007819", "end": "2020-07-18 03:21:31.437627", "rc": 0, "start": "2020-07-18 03:21:31.429808", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:31.511 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:31 +0000 (0:00:00.205) 0:00:55.363 ********* 2020-07-18T03:21:48.419 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-07-18T03:21:48.423 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.860426", "end": "2020-07-18 03:21:48.389891", "rc": 0, "start": "2020-07-18 03:21:31.529465", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:48.475 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:48 +0000 (0:00:16.964) 0:01:12.327 ********* 2020-07-18T03:21:48.645 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.880826", "end": "2020-07-18 03:21:48.610873", "rc": 0, "start": "2020-07-18 03:21:31.730047", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:48.687 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:48 +0000 (0:00:00.211) 0:01:12.539 ********* 2020-07-18T03:21:48.715 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-07-18T03:21:48.757 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:48 +0000 (0:00:00.070) 0:01:12.609 ********* 2020-07-18T03:21:48.806 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-07-18T03:21:48.808 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-07-18T03:21:48.866 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:48 +0000 (0:00:00.108) 0:01:12.718 ********* 2020-07-18T03:21:48.935 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi018.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-07-18T03:21:48.993 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:48 +0000 (0:00:00.125) 0:01:12.844 ********* 2020-07-18T03:21:49.030 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.037) 0:01:12.882 ********* 2020-07-18T03:21:49.057 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-07-18T03:21:49.058 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-07-18T03:21:49.071 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-07-18T03:21:49.078 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-07-18T03:21:49.105 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.075) 0:01:12.957 ********* 2020-07-18T03:21:49.148 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.042) 0:01:12.999 ********* 2020-07-18T03:21:49.168 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-07-18T03:21:49.170 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-07-18T03:21:49.208 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.060) 0:01:13.060 ********* 2020-07-18T03:21:49.220 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi047.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-07-18T03:21:49.259 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.051) 0:01:13.111 ********* 2020-07-18T03:21:49.339 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-07-18T03:21:49.341 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-07-18T03:21:49.376 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.116) 0:01:13.228 ********* 2020-07-18T03:21:49.430 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-07-18T03:21:49.432 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-07-18T03:21:49.466 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.089) 0:01:13.318 ********* 2020-07-18T03:21:49.515 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.048) 0:01:13.367 ********* 2020-07-18T03:21:49.710 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 1224, "name": "ubuntu", "state": "present", "system": false} 2020-07-18T03:21:49.762 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.247) 0:01:13.614 ********* 2020-07-18T03:21:49.776 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-07-18T03:21:49.778 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-07-18T03:21:49.818 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.055) 0:01:13.670 ********* 2020-07-18T03:21:49.828 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-07-18T03:21:49.830 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:49.868 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.049) 0:01:13.720 ********* 2020-07-18T03:21:49.901 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:49.938 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:49 +0000 (0:00:00.070) 0:01:13.790 ********* 2020-07-18T03:21:50.039 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-07-18T03:21:50.041 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 1224, "name": "ubuntu", "state": "present", "system": false} 2020-07-18T03:21:50.085 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:50 +0000 (0:00:00.146) 0:01:13.937 ********* 2020-07-18T03:21:50.149 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:50.203 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:50 +0000 (0:00:00.117) 0:01:14.055 ********* 2020-07-18T03:21:50.215 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-07-18T03:21:50.217 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-07-18T03:21:50.269 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:50 +0000 (0:00:00.065) 0:01:14.121 ********* 2020-07-18T03:21:50.280 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-07-18T03:21:50.282 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:21:50.326 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:50 +0000 (0:00:00.057) 0:01:14.178 ********* 2020-07-18T03:21:50.598 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-07-18T03:21:50.600 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-07-18T03:21:50.650 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:50 +0000 (0:00:00.323) 0:01:14.502 ********* 2020-07-18T03:21:50.959 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-07-18T03:21:50.961 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-07-18T03:21:51.002 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:51 +0000 (0:00:00.352) 0:01:14.854 ********* 2020-07-18T03:21:51.058 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-07-18T03:21:51.060 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi018"}, "changed": false} 2020-07-18T03:21:51.112 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:51 +0000 (0:00:00.110) 0:01:14.965 ********* 2020-07-18T03:21:51.357 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-07-18T03:21:51.359 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-07-18T03:21:51.441 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:51 +0000 (0:00:00.328) 0:01:15.293 ********* 2020-07-18T03:21:51.495 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-07-18T03:21:51.497 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi047"}, "changed": false} 2020-07-18T03:21:51.537 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:51 +0000 (0:00:00.096) 0:01:15.389 ********* 2020-07-18T03:21:52.366 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi018", "ansible_hostname": "smithi018", "ansible_nodename": "smithi018"}, "changed": false, "name": "smithi018"} ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi047", "ansible_hostname": "smithi047", "ansible_nodename": "smithi047"}, "changed": false, "name": "smithi047"} 2020-07-18T03:21:52.413 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:52 +0000 (0:00:00.873) 0:01:16.262 ********* 2020-07-18T03:21:52.459 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:52 +0000 (0:00:00.045) 0:01:16.308 ********* 2020-07-18T03:21:52.955 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-07-18T03:21:52.957 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-07-18T03:21:53.025 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:53 +0000 (0:00:00.568) 0:01:16.877 ********* 2020-07-18T03:21:53.045 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-07-18T03:21:53.091 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:53 +0000 (0:00:00.066) 0:01:16.944 ********* 2020-07-18T03:21:53.578 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-07-18T03:21:53.580 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-07-18T03:21:53.628 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:53 +0000 (0:00:00.536) 0:01:17.480 ********* 2020-07-18T03:21:53.651 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-07-18T03:21:53.694 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:53 +0000 (0:00:00.066) 0:01:17.546 ********* 2020-07-18T03:21:54.293 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** ok: [smithi018.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-07-18T03:21:54.342 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:54 +0000 (0:00:00.647) 0:01:18.194 ********* 2020-07-18T03:21:54.354 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-07-18T03:21:54.401 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:54 +0000 (0:00:00.055) 0:01:18.249 ********* 2020-07-18T03:21:54.941 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-07-18T03:21:54.943 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-07-18T03:21:54.997 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:54 +0000 (0:00:00.599) 0:01:18.849 ********* 2020-07-18T03:21:55.066 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-07-18T03:21:55.111 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.114) 0:01:18.964 ********* 2020-07-18T03:21:55.248 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-07-18T03:21:55.251 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-07-18T03:21:55.299 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.187) 0:01:19.151 ********* 2020-07-18T03:21:55.370 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-07-18T03:21:55.410 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.110) 0:01:19.261 ********* 2020-07-18T03:21:55.539 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-07-18T03:21:55.541 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-07-18T03:21:55.579 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.170) 0:01:19.431 ********* 2020-07-18T03:21:55.644 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-07-18T03:21:55.694 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.113) 0:01:19.545 ********* 2020-07-18T03:21:55.829 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-07-18T03:21:55.831 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1224, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-07-18T03:21:55.865 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.172) 0:01:19.717 ********* 2020-07-18T03:21:55.935 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1224, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-07-18T03:21:55.977 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:55 +0000 (0:00:00.111) 0:01:19.829 ********* 2020-07-18T03:21:56.439 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-07-18T03:21:56.441 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-07-18T03:21:56.496 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:56 +0000 (0:00:00.509) 0:01:20.348 ********* 2020-07-18T03:21:56.554 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-07-18T03:21:56.593 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:56 +0000 (0:00:00.097) 0:01:20.445 ********* 2020-07-18T03:21:57.057 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-07-18T03:21:57.059 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-07-18T03:21:57.096 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:57 +0000 (0:00:00.502) 0:01:20.948 ********* 2020-07-18T03:21:57.182 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-07-18T03:21:57.220 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:57 +0000 (0:00:00.123) 0:01:21.072 ********* 2020-07-18T03:21:59.192 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-07-18T03:21:59.198 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.851212", "end": "2020-07-18 03:21:59.162907", "rc": 0, "start": "2020-07-18 03:21:57.311695", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:59.229 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:59 +0000 (0:00:02.008) 0:01:23.081 ********* 2020-07-18T03:21:59.297 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.846427", "end": "2020-07-18 03:21:59.266028", "rc": 0, "start": "2020-07-18 03:21:57.419601", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:21:59.349 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:21:59 +0000 (0:00:00.120) 0:01:23.201 ********* 2020-07-18T03:21:59.855 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-07-18T03:21:59.856 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-07-18T03:21:59.984 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-07-18T03:22:00.369 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-07-18T03:22:00.500 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-07-18T03:22:00.915 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-07-18T03:22:00.969 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:00 +0000 (0:00:01.619) 0:01:24.821 ********* 2020-07-18T03:22:01.055 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-07-18T03:22:01.100 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:01 +0000 (0:00:00.131) 0:01:24.952 ********* 2020-07-18T03:22:01.634 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* ok: [smithi018.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-07-18T03:22:01.677 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:01 +0000 (0:00:00.571) 0:01:25.524 ********* 2020-07-18T03:22:01.731 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-07-18T03:22:01.775 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:01 +0000 (0:00:00.102) 0:01:25.627 ********* 2020-07-18T03:22:02.287 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-07-18T03:22:02.289 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-07-18T03:22:02.384 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-07-18T03:22:02.816 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-07-18T03:22:02.940 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-07-18T03:22:03.363 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-07-18T03:22:03.401 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:01.626) 0:01:27.253 ********* 2020-07-18T03:22:03.432 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-07-18T03:22:03.476 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.074) 0:01:27.328 ********* 2020-07-18T03:22:03.496 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-07-18T03:22:03.498 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-07-18T03:22:03.535 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.059) 0:01:27.387 ********* 2020-07-18T03:22:03.544 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-07-18T03:22:03.546 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.581 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.046) 0:01:27.433 ********* 2020-07-18T03:22:03.591 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-07-18T03:22:03.629 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.047) 0:01:27.481 ********* 2020-07-18T03:22:03.639 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-07-18T03:22:03.641 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.662 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.698 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.069) 0:01:27.550 ********* 2020-07-18T03:22:03.709 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-07-18T03:22:03.711 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.749 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.049) 0:01:27.599 ********* 2020-07-18T03:22:03.757 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-07-18T03:22:03.759 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.797 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.049) 0:01:27.649 ********* 2020-07-18T03:22:03.808 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-07-18T03:22:03.809 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.847 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.049) 0:01:27.699 ********* 2020-07-18T03:22:03.856 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:03.858 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-07-18T03:22:03.893 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.046) 0:01:27.745 ********* 2020-07-18T03:22:03.904 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-07-18T03:22:03.906 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:03.943 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:03 +0000 (0:00:00.047) 0:01:27.793 ********* 2020-07-18T03:22:03.975 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-07-18T03:22:03.979 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:04.015 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.074) 0:01:27.868 ********* 2020-07-18T03:22:04.025 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:04.027 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-07-18T03:22:04.071 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.055) 0:01:27.923 ********* 2020-07-18T03:22:04.082 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:04.119 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.047) 0:01:27.971 ********* 2020-07-18T03:22:04.144 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:04.199 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.077) 0:01:28.048 ********* 2020-07-18T03:22:04.253 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-07-18T03:22:04.255 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:04.323 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.126) 0:01:28.175 ********* 2020-07-18T03:22:04.372 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-07-18T03:22:04.374 INFO:teuthology.task.ansible.out:fatal: [smithi018.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007430", "end": "2020-07-18 03:22:04.337137", "msg": "non-zero return code", "rc": 1, "start": "2020-07-18 03:22:04.329707", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-07-18T03:22:04.421 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.097) 0:01:28.273 ********* 2020-07-18T03:22:04.513 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-07-18T03:22:04.515 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:04.570 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.149) 0:01:28.422 ********* 2020-07-18T03:22:04.602 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* fatal: [smithi047.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007075", "end": "2020-07-18 03:22:04.548427", "msg": "non-zero return code", "rc": 1, "start": "2020-07-18 03:22:04.541352", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-07-18T03:22:04.655 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.084) 0:01:28.507 ********* 2020-07-18T03:22:04.711 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-07-18T03:22:04.712 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:04.755 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:04 +0000 (0:00:00.100) 0:01:28.607 ********* 2020-07-18T03:22:07.093 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-07-18T03:22:07.094 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-07-18T03:22:07.137 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:07 +0000 (0:00:02.381) 0:01:30.989 ********* 2020-07-18T03:22:07.157 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-07-18T03:22:07.195 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:07 +0000 (0:00:00.058) 0:01:31.047 ********* 2020-07-18T03:22:09.192 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-07-18T03:22:09.232 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:09 +0000 (0:00:02.036) 0:01:33.084 ********* 2020-07-18T03:22:09.253 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-07-18T03:22:09.295 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:09 +0000 (0:00:00.062) 0:01:33.147 ********* 2020-07-18T03:22:12.416 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-07-18T03:22:12.418 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-07-18T03:22:12.464 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:12 +0000 (0:00:03.169) 0:01:36.316 ********* 2020-07-18T03:22:12.475 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-07-18T03:22:12.513 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:12 +0000 (0:00:00.049) 0:01:36.365 ********* 2020-07-18T03:22:12.523 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-07-18T03:22:12.525 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:12.572 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:12 +0000 (0:00:00.058) 0:01:36.424 ********* 2020-07-18T03:22:12.583 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:12.621 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:12 +0000 (0:00:00.049) 0:01:36.473 ********* 2020-07-18T03:22:13.570 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-07-18T03:22:13.616 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:13 +0000 (0:00:00.994) 0:01:37.468 ********* 2020-07-18T03:22:13.632 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-07-18T03:22:13.663 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:13 +0000 (0:00:00.047) 0:01:37.515 ********* 2020-07-18T03:22:13.682 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-07-18T03:22:13.684 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:13.721 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:13 +0000 (0:00:00.057) 0:01:37.573 ********* 2020-07-18T03:22:13.730 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:13.767 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:13 +0000 (0:00:00.046) 0:01:37.619 ********* 2020-07-18T03:22:14.086 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-07-18T03:22:14.087 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service system.slice sysinit.target dbus.service dbus.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ebtables.service shutdown.target iptables.service ipset.service ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-07-18T03:22:14.088 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", 2020-07-18T03:22:14.090 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target dbus.socket", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-07-18T03:22:14.092 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-07-18T03:22:14.124 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:14 +0000 (0:00:00.355) 0:01:37.975 ********* 2020-07-18T03:22:14.136 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.socket dbus.service sysinit.target basic.target system.slice polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ip6tables.service shutdown.target ipset.service iptables.service ebtables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-07-18T03:22:14.138 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", 2020-07-18T03:22:14.139 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice dbus.socket", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-07-18T03:22:14.141 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-07-18T03:22:14.175 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:14 +0000 (0:00:00.051) 0:01:38.027 ********* 2020-07-18T03:22:14.462 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-07-18T03:22:14.464 INFO:teuthology.task.ansible.out:fatal: [smithi047.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-07-18T03:22:14.509 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:14 +0000 (0:00:00.334) 0:01:38.361 ********* 2020-07-18T03:22:14.519 INFO:teuthology.task.ansible.out:fatal: [smithi018.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-07-18T03:22:14.555 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:14 +0000 (0:00:00.046) 0:01:38.407 ********* 2020-07-18T03:22:15.432 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-07-18T03:22:15.434 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-07-18T03:22:15.435 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-07-18T03:22:15.479 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.923) 0:01:39.331 ********* 2020-07-18T03:22:15.532 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.052) 0:01:39.384 ********* 2020-07-18T03:22:15.542 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-07-18T03:22:15.544 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.581 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.049) 0:01:39.433 ********* 2020-07-18T03:22:15.592 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.633 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.051) 0:01:39.485 ********* 2020-07-18T03:22:15.643 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-07-18T03:22:15.645 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.681 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.047) 0:01:39.533 ********* 2020-07-18T03:22:15.698 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.731 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.050) 0:01:39.583 ********* 2020-07-18T03:22:15.741 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-07-18T03:22:15.743 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.785 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.054) 0:01:39.637 ********* 2020-07-18T03:22:15.798 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.861 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.075) 0:01:39.713 ********* 2020-07-18T03:22:15.875 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-07-18T03:22:15.877 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.917 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.056) 0:01:39.769 ********* 2020-07-18T03:22:15.928 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:15.971 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:15 +0000 (0:00:00.053) 0:01:39.823 ********* 2020-07-18T03:22:15.980 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-07-18T03:22:15.983 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.018 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.047) 0:01:39.870 ********* 2020-07-18T03:22:16.028 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.065 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.047) 0:01:39.918 ********* 2020-07-18T03:22:16.075 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-07-18T03:22:16.077 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.112 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.046) 0:01:39.964 ********* 2020-07-18T03:22:16.131 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.169 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.056) 0:01:40.021 ********* 2020-07-18T03:22:16.180 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-07-18T03:22:16.225 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.056) 0:01:40.077 ********* 2020-07-18T03:22:16.280 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.054) 0:01:40.132 ********* 2020-07-18T03:22:16.290 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-07-18T03:22:16.292 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.349 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.067) 0:01:40.199 ********* 2020-07-18T03:22:16.360 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.396 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.048) 0:01:40.248 ********* 2020-07-18T03:22:16.407 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-07-18T03:22:16.410 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.446 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.049) 0:01:40.298 ********* 2020-07-18T03:22:16.456 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.495 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.049) 0:01:40.347 ********* 2020-07-18T03:22:16.504 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-07-18T03:22:16.506 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.550 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.054) 0:01:40.402 ********* 2020-07-18T03:22:16.578 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.609 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.058) 0:01:40.461 ********* 2020-07-18T03:22:16.619 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-07-18T03:22:16.620 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.662 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.052) 0:01:40.514 ********* 2020-07-18T03:22:16.673 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.717 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.055) 0:01:40.569 ********* 2020-07-18T03:22:16.727 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-07-18T03:22:16.767 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.049) 0:01:40.619 ********* 2020-07-18T03:22:16.812 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.045) 0:01:40.664 ********* 2020-07-18T03:22:16.822 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-07-18T03:22:16.825 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.860 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.047) 0:01:40.712 ********* 2020-07-18T03:22:16.870 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.911 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.047) 0:01:40.759 ********* 2020-07-18T03:22:16.918 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-07-18T03:22:16.920 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:16.965 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:16 +0000 (0:00:00.057) 0:01:40.817 ********* 2020-07-18T03:22:16.976 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.015 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.050) 0:01:40.867 ********* 2020-07-18T03:22:17.024 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-07-18T03:22:17.033 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.066 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.049) 0:01:40.917 ********* 2020-07-18T03:22:17.075 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.114 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.049) 0:01:40.966 ********* 2020-07-18T03:22:17.124 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-07-18T03:22:17.127 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.184 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.064) 0:01:41.031 ********* 2020-07-18T03:22:17.198 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.235 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.055) 0:01:41.086 ********* 2020-07-18T03:22:17.244 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-07-18T03:22:17.246 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.283 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.048) 0:01:41.135 ********* 2020-07-18T03:22:17.294 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.330 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.047) 0:01:41.182 ********* 2020-07-18T03:22:17.340 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-07-18T03:22:17.342 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.379 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.048) 0:01:41.231 ********* 2020-07-18T03:22:17.390 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.429 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.049) 0:01:41.281 ********* 2020-07-18T03:22:17.439 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-07-18T03:22:17.441 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.478 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.048) 0:01:41.330 ********* 2020-07-18T03:22:17.488 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.526 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.048) 0:01:41.379 ********* 2020-07-18T03:22:17.536 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-07-18T03:22:17.538 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.576 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.047) 0:01:41.426 ********* 2020-07-18T03:22:17.584 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.625 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.050) 0:01:41.477 ********* 2020-07-18T03:22:17.635 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-07-18T03:22:17.636 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.677 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.052) 0:01:41.530 ********* 2020-07-18T03:22:17.688 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.728 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.047) 0:01:41.577 ********* 2020-07-18T03:22:17.734 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-07-18T03:22:17.736 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.920 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.195) 0:01:41.772 ********* 2020-07-18T03:22:17.930 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:17.969 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:17 +0000 (0:00:00.049) 0:01:41.821 ********* 2020-07-18T03:22:17.979 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-07-18T03:22:17.980 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.017 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.048) 0:01:41.870 ********* 2020-07-18T03:22:18.028 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.064 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.046) 0:01:41.916 ********* 2020-07-18T03:22:18.075 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-07-18T03:22:18.076 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.112 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.048) 0:01:41.964 ********* 2020-07-18T03:22:18.124 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.159 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.047) 0:01:42.012 ********* 2020-07-18T03:22:18.188 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.215 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.055) 0:01:42.067 ********* 2020-07-18T03:22:18.230 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.272 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.056) 0:01:42.124 ********* 2020-07-18T03:22:18.282 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-07-18T03:22:18.284 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.323 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.050) 0:01:42.175 ********* 2020-07-18T03:22:18.333 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.389 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.065) 0:01:42.241 ********* 2020-07-18T03:22:18.400 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-07-18T03:22:18.400 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.441 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.051) 0:01:42.293 ********* 2020-07-18T03:22:18.451 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.493 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.052) 0:01:42.345 ********* 2020-07-18T03:22:18.504 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-07-18T03:22:18.508 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.544 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.050) 0:01:42.396 ********* 2020-07-18T03:22:18.557 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.605 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.060) 0:01:42.457 ********* 2020-07-18T03:22:18.618 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.653 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.048) 0:01:42.505 ********* 2020-07-18T03:22:18.664 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.715 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.062) 0:01:42.567 ********* 2020-07-18T03:22:18.730 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.767 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.051) 0:01:42.619 ********* 2020-07-18T03:22:18.778 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.821 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.054) 0:01:42.673 ********* 2020-07-18T03:22:18.835 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-07-18T03:22:18.836 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.874 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.052) 0:01:42.726 ********* 2020-07-18T03:22:18.886 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.929 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.055) 0:01:42.781 ********* 2020-07-18T03:22:18.947 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:18.997 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:18 +0000 (0:00:00.067) 0:01:42.848 ********* 2020-07-18T03:22:19.034 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.073 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.076) 0:01:42.925 ********* 2020-07-18T03:22:19.084 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.125 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.051) 0:01:42.977 ********* 2020-07-18T03:22:19.138 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.177 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.052) 0:01:43.029 ********* 2020-07-18T03:22:19.190 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.239 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.062) 0:01:43.091 ********* 2020-07-18T03:22:19.250 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.298 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.058) 0:01:43.150 ********* 2020-07-18T03:22:19.309 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:19.311 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.347 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.049) 0:01:43.199 ********* 2020-07-18T03:22:19.360 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.400 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.052) 0:01:43.252 ********* 2020-07-18T03:22:19.424 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-07-18T03:22:19.430 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.462 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.062) 0:01:43.314 ********* 2020-07-18T03:22:19.472 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.510 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.047) 0:01:43.362 ********* 2020-07-18T03:22:19.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-07-18T03:22:19.522 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.572 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.056) 0:01:43.418 ********* 2020-07-18T03:22:19.578 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.617 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.050) 0:01:43.469 ********* 2020-07-18T03:22:19.626 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-07-18T03:22:19.628 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.668 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.051) 0:01:43.520 ********* 2020-07-18T03:22:19.679 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.725 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.056) 0:01:43.577 ********* 2020-07-18T03:22:19.738 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.778 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.053) 0:01:43.631 ********* 2020-07-18T03:22:19.793 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:19.831 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:19 +0000 (0:00:00.052) 0:01:43.683 ********* 2020-07-18T03:22:20.032 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-07-18T03:22:20.034 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-07-18T03:22:20.080 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.246) 0:01:43.930 ********* 2020-07-18T03:22:20.090 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-07-18T03:22:20.138 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.060) 0:01:43.991 ********* 2020-07-18T03:22:20.363 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-07-18T03:22:20.400 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.261) 0:01:44.252 ********* 2020-07-18T03:22:20.425 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-07-18T03:22:20.467 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.066) 0:01:44.319 ********* 2020-07-18T03:22:20.638 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-07-18T03:22:20.640 INFO:teuthology.task.ansible.out:[WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-07-18T03:22:20.682 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.214) 0:01:44.534 ********* 2020-07-18T03:22:20.722 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-07-18T03:22:20.753 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.071) 0:01:44.605 ********* 2020-07-18T03:22:20.769 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:20.804 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.051) 0:01:44.656 ********* 2020-07-18T03:22:20.816 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:20.859 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.054) 0:01:44.711 ********* 2020-07-18T03:22:20.868 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-07-18T03:22:20.870 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:20.908 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.049) 0:01:44.761 ********* 2020-07-18T03:22:20.919 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:20.959 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:20 +0000 (0:00:00.050) 0:01:44.811 ********* 2020-07-18T03:22:20.969 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-07-18T03:22:20.971 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.027 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.061) 0:01:44.873 ********* 2020-07-18T03:22:21.032 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.074 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.052) 0:01:44.926 ********* 2020-07-18T03:22:21.091 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.146 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.072) 0:01:44.998 ********* 2020-07-18T03:22:21.161 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.200 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.054) 0:01:45.052 ********* 2020-07-18T03:22:21.210 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-07-18T03:22:21.212 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.250 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.049) 0:01:45.102 ********* 2020-07-18T03:22:21.260 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.297 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.046) 0:01:45.149 ********* 2020-07-18T03:22:21.306 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-07-18T03:22:21.308 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.344 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.047) 0:01:45.196 ********* 2020-07-18T03:22:21.354 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.394 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.049) 0:01:45.246 ********* 2020-07-18T03:22:21.404 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-07-18T03:22:21.406 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.447 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.049) 0:01:45.296 ********* 2020-07-18T03:22:21.455 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.493 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.048) 0:01:45.345 ********* 2020-07-18T03:22:21.503 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-07-18T03:22:21.505 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.544 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.051) 0:01:45.396 ********* 2020-07-18T03:22:21.556 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.596 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.052) 0:01:45.449 ********* 2020-07-18T03:22:21.607 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-07-18T03:22:21.609 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.650 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.050) 0:01:45.499 ********* 2020-07-18T03:22:21.658 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.702 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.049) 0:01:45.549 ********* 2020-07-18T03:22:21.712 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-07-18T03:22:21.713 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.758 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.060) 0:01:45.609 ********* 2020-07-18T03:22:21.767 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.809 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.051) 0:01:45.661 ********* 2020-07-18T03:22:21.818 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-07-18T03:22:21.820 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.858 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.049) 0:01:45.710 ********* 2020-07-18T03:22:21.871 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.910 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.052) 0:01:45.762 ********* 2020-07-18T03:22:21.920 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-07-18T03:22:21.926 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:21.960 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:21 +0000 (0:00:00.049) 0:01:45.811 ********* 2020-07-18T03:22:21.970 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.009 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.049) 0:01:45.861 ********* 2020-07-18T03:22:22.019 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-07-18T03:22:22.023 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.064 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.054) 0:01:45.916 ********* 2020-07-18T03:22:22.074 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.114 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.049) 0:01:45.966 ********* 2020-07-18T03:22:22.124 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-07-18T03:22:22.126 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.164 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.050) 0:01:46.016 ********* 2020-07-18T03:22:22.175 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.210 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.046) 0:01:46.062 ********* 2020-07-18T03:22:22.220 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-07-18T03:22:22.222 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.260 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.049) 0:01:46.112 ********* 2020-07-18T03:22:22.273 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.310 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.049) 0:01:46.162 ********* 2020-07-18T03:22:22.325 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-07-18T03:22:22.327 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.369 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.059) 0:01:46.221 ********* 2020-07-18T03:22:22.380 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.424 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.055) 0:01:46.276 ********* 2020-07-18T03:22:22.438 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-07-18T03:22:22.441 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.480 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.055) 0:01:46.332 ********* 2020-07-18T03:22:22.493 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.529 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.048) 0:01:46.381 ********* 2020-07-18T03:22:22.541 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-07-18T03:22:22.543 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.580 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.050) 0:01:46.432 ********* 2020-07-18T03:22:22.590 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.628 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.048) 0:01:46.480 ********* 2020-07-18T03:22:22.637 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-07-18T03:22:22.639 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.677 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.048) 0:01:46.529 ********* 2020-07-18T03:22:22.689 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.730 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.053) 0:01:46.582 ********* 2020-07-18T03:22:22.739 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-07-18T03:22:22.740 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.786 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.056) 0:01:46.638 ********* 2020-07-18T03:22:22.796 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.837 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.050) 0:01:46.689 ********* 2020-07-18T03:22:22.848 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-07-18T03:22:22.850 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.894 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.056) 0:01:46.745 ********* 2020-07-18T03:22:22.905 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.945 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.052) 0:01:46.798 ********* 2020-07-18T03:22:22.957 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-07-18T03:22:22.959 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:22.995 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:22 +0000 (0:00:00.049) 0:01:46.847 ********* 2020-07-18T03:22:23.006 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.046 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.051) 0:01:46.898 ********* 2020-07-18T03:22:23.061 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-07-18T03:22:23.098 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.052) 0:01:46.951 ********* 2020-07-18T03:22:23.112 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-07-18T03:22:23.159 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.059) 0:01:47.010 ********* 2020-07-18T03:22:23.168 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-07-18T03:22:23.170 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.181 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-07-18T03:22:23.192 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.194 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.204 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.215 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.217 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.281 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.122) 0:01:47.133 ********* 2020-07-18T03:22:23.292 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-07-18T03:22:23.294 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.301 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.345 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.063) 0:01:47.197 ********* 2020-07-18T03:22:23.564 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-07-18T03:22:23.566 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004485", "end": "2020-07-18 03:22:23.526196", "rc": 0, "start": "2020-07-18 03:22:23.521711", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-07-18T03:22:23.615 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.269) 0:01:47.467 ********* 2020-07-18T03:22:23.637 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004680", "end": "2020-07-18 03:22:23.591640", "rc": 0, "start": "2020-07-18 03:22:23.586960", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-07-18T03:22:23.689 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.073) 0:01:47.541 ********* 2020-07-18T03:22:23.701 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-07-18T03:22:23.703 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.754 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.065) 0:01:47.606 ********* 2020-07-18T03:22:23.765 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.811 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.056) 0:01:47.663 ********* 2020-07-18T03:22:23.821 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-07-18T03:22:23.823 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.859 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.047) 0:01:47.711 ********* 2020-07-18T03:22:23.869 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.913 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.053) 0:01:47.765 ********* 2020-07-18T03:22:23.930 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:23.964 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:23 +0000 (0:00:00.051) 0:01:47.816 ********* 2020-07-18T03:22:23.975 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:24.016 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:24 +0000 (0:00:00.052) 0:01:47.868 ********* 2020-07-18T03:22:24.026 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-07-18T03:22:24.027 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:24.065 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:24 +0000 (0:00:00.048) 0:01:47.917 ********* 2020-07-18T03:22:24.078 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:24.125 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:24 +0000 (0:00:00.060) 0:01:47.977 ********* 2020-07-18T03:22:25.085 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-07-18T03:22:25.087 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-07-18T03:22:25.126 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:01.000) 0:01:48.978 ********* 2020-07-18T03:22:25.147 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-07-18T03:22:25.194 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.068) 0:01:49.047 ********* 2020-07-18T03:22:25.206 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-07-18T03:22:25.208 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930532024320, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180670, 'block_used': 12914438, 'inode_total': 61054976, 'inode_available': 60990206, 'inode_used': 64770, 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180670, "block_size": 4096, "block_total": 240095108, "block_used": 12914438, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990206, "inode_total": 61054976, "inode_used": 64770, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930532024320, "size_total": 983429562368, "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}} 2020-07-18T03:22:25.248 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.053) 0:01:49.100 ********* 2020-07-18T03:22:25.273 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930532028416, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180671, 'block_used': 12914437, 'inode_total': 61054976, 'inode_available': 60990206, 'inode_used': 64770, 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180671, "block_size": 4096, "block_total": 240095108, "block_used": 12914437, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990206, "inode_total": 61054976, "inode_used": 64770, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930532028416, "size_total": 983429562368, "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}} 2020-07-18T03:22:25.314 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.066) 0:01:49.166 ********* 2020-07-18T03:22:25.485 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-07-18T03:22:25.487 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005439", "end": "2020-07-18 03:22:25.459179", "rc": 0, "start": "2020-07-18 03:22:25.453740", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:25.531 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.216) 0:01:49.383 ********* 2020-07-18T03:22:25.554 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005230", "end": "2020-07-18 03:22:25.521308", "rc": 0, "start": "2020-07-18 03:22:25.516078", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:25.596 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.065) 0:01:49.448 ********* 2020-07-18T03:22:25.622 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-07-18T03:22:25.624 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930532024320, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180670, 'block_used': 12914438, 'inode_total': 61054976, 'inode_available': 60990206, 'inode_used': 64770, 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180670, "block_size": 4096, "block_total": 240095108, "block_used": 12914438, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990206, "inode_total": 61054976, "inode_used": 64770, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930532024320, "size_total": 983429562368, "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:25.681 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.085) 0:01:49.534 ********* 2020-07-18T03:22:25.692 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930532028416, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180671, 'block_used': 12914437, 'inode_total': 61054976, 'inode_available': 60990206, 'inode_used': 64770, 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180671, "block_size": 4096, "block_total": 240095108, "block_used": 12914437, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990206, "inode_total": 61054976, "inode_used": 64770, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930532028416, "size_total": 983429562368, "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:25.731 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.049) 0:01:49.583 ********* 2020-07-18T03:22:25.741 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-07-18T03:22:25.780 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.048) 0:01:49.632 ********* 2020-07-18T03:22:25.843 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:25 +0000 (0:00:00.062) 0:01:49.695 ********* 2020-07-18T03:22:27.408 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-07-18T03:22:27.410 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9D4LDvV1NU3K9HU3fxzA4dHXzjO0jqTBM'], 'uuids': ['10a28e77-ee6c-4f6c-ac44-2019ca5dfb64'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.358126", "end": "2020-07-18 03:22:27.359497", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9D4LDvV1NU3K9HU3fxzA4dHXzjO0jqTBM"], "labels": [], "masters": [], "uuids": ["10a28e77-ee6c-4f6c-ac44-2019ca5dfb64"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:26.001371", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:27.473 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhNMhi0OI0o9hEhu4xS3OENMKa6VMG0h22'], 'uuids': ['f9bfad5e-5d82-4761-ad17-23970bfb005a'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.362832", "end": "2020-07-18 03:22:27.436168", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhNMhi0OI0o9hEhu4xS3OENMKa6VMG0h22"], "labels": [], "masters": [], "uuids": ["f9bfad5e-5d82-4761-ad17-23970bfb005a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:26.073336", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:28.706 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-4mB2EW-jxTJ-F6JD-2yKN-2XVR-HDH3-XB90vk', 'nvme-INTEL_SSDPED1D480GA_PHMB75120091480DGN', 'nvme-nvme.8086-50484d42373531323030393134383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPED1D480GA', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '937703088', 'sectorsize': '512', 'size': '447.13 GB', 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.061552", "end": "2020-07-18 03:22:28.663611", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-4mB2EW-jxTJ-F6JD-2yKN-2XVR-HDH3-XB90vk", "nvme-INTEL_SSDPED1D480GA_PHMB75120091480DGN", "nvme-nvme.8086-50484d42373531323030393134383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:27.602059", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2020-07-18T03:22:28.709 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:28.809 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-stxYcT-3n0o-Wf1W-bB2l-fgrF-A2wX-ipqjgX', 'nvme-INTEL_SSDPED1D480GA_PHMB75130022480DGN', 'nvme-nvme.8086-50484d42373531333030323234383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPED1D480GA', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '937703088', 'sectorsize': '512', 'size': '447.13 GB', 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.092148", "end": "2020-07-18 03:22:28.760460", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-stxYcT-3n0o-Wf1W-bB2l-fgrF-A2wX-ipqjgX", "nvme-INTEL_SSDPED1D480GA_PHMB75130022480DGN", "nvme-nvme.8086-50484d42373531333030323234383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:27.668312", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2020-07-18T03:22:28.810 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:30.011 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd92Dy7oVTMA7LclThTt833o6KjhXbLBvQ0'], 'uuids': ['03ec8be4-d2ca-4c32-bf19-cac3e5eedf71'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.063434", "end": "2020-07-18 03:22:29.960225", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd92Dy7oVTMA7LclThTt833o6KjhXbLBvQ0"], "labels": [], "masters": [], "uuids": ["03ec8be4-d2ca-4c32-bf19-cac3e5eedf71"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:28.896791", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:30.079 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhJI0IO1sIhoafrcOSuC66Hk1t7WotJKC7'], 'uuids': ['ee9b5ccc-84e5-420a-9e19-ea4bc70ec74d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.043189", "end": "2020-07-18 03:22:30.041595", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhJI0IO1sIhoafrcOSuC66Hk1t7WotJKC7"], "labels": [], "masters": [], "uuids": ["ee9b5ccc-84e5-420a-9e19-ea4bc70ec74d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:28.998406", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:31.250 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9UZ2OAyjIF1HLj01eVUcvt4mhtQIv6u3j'], 'uuids': ['0389221f-6a3c-4407-9458-747e72019dab'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.043758", "end": "2020-07-18 03:22:31.217121", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9UZ2OAyjIF1HLj01eVUcvt4mhtQIv6u3j"], "labels": [], "masters": [], "uuids": ["0389221f-6a3c-4407-9458-747e72019dab"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:30.173363", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:31.349 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhebSAtKVDvexBxq8kTg0Js2ZBMVYwmATV'], 'uuids': ['eca3bb01-a57f-4cda-8226-445440e4d359'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.044171", "end": "2020-07-18 03:22:31.317320", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhebSAtKVDvexBxq8kTg0Js2ZBMVYwmATV"], "labels": [], "masters": [], "uuids": ["eca3bb01-a57f-4cda-8226-445440e4d359"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:30.273149", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:32.552 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9RTZYfcUrWP5n7hBZtSKC3UnoPuozg5BQ'], 'uuids': ['d3126e39-9bc5-4437-97f3-89315c91f5f9'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.057308", "end": "2020-07-18 03:22:32.517391", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9RTZYfcUrWP5n7hBZtSKC3UnoPuozg5BQ"], "labels": [], "masters": [], "uuids": ["d3126e39-9bc5-4437-97f3-89315c91f5f9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:31.460083", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:32.568 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQDQ', 'wwn-0x5000c500871a8c35'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HQDQ', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a8c35', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQDQ-part1', 'wwn-0x5000c500871a8c35-part1'], 'uuids': ['7d7ca31c-77d6-4023-896d-a1d87c0f49ac'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQDQ", "wwn-0x5000c500871a8c35"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQDQ-part1", "wwn-0x5000c500871a8c35-part1"], "labels": [], "masters": [], "uuids": ["7d7ca31c-77d6-4023-896d-a1d87c0f49ac"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQDQ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a8c35"}}, 2020-07-18T03:22:32.570 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-07-18T03:22:32.644 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhZTGuzu0ACqYrvxWffdA9EbltFdq46L0u'], 'uuids': ['82598bda-40d1-43eb-b79d-7536dd53d718'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.047365", "end": "2020-07-18 03:22:32.582077", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhZTGuzu0ACqYrvxWffdA9EbltFdq46L0u"], "labels": [], "masters": [], "uuids": ["82598bda-40d1-43eb-b79d-7536dd53d718"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:31.534712", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:32.701 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HN80', 'wwn-0x5000c500871ae812'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HN80', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871ae812', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HN80-part1', 'wwn-0x5000c500871ae812-part1'], 'uuids': ['7d7ca31c-77d6-4023-896d-a1d87c0f49ac'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HN80", "wwn-0x5000c500871ae812"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HN80-part1", "wwn-0x5000c500871ae812-part1"], "labels": [], "masters": [], "uuids": ["7d7ca31c-77d6-4023-896d-a1d87c0f49ac"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HN80", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871ae812"}}, 2020-07-18T03:22:32.706 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-07-18T03:22:33.823 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9IfZs0gP3nM7oCJScVtthe8T6jRs2kKmk'], 'uuids': ['c7352e59-9f27-4ae7-873c-fbf916de5d04'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.043539", "end": "2020-07-18 03:22:33.784666", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9IfZs0gP3nM7oCJScVtthe8T6jRs2kKmk"], "labels": [], "masters": [], "uuids": ["c7352e59-9f27-4ae7-873c-fbf916de5d04"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:32.741127", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:33.870 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:33 +0000 (0:00:08.026) 0:01:57.722 ********* 2020-07-18T03:22:33.993 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2Wh3zSjB3DnsF99xFrBYpqLPOM2eJTzkU7z'], 'uuids': ['fe89a6bb-a0d7-45a2-9553-4b012ccbaa72'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.050009", "end": "2020-07-18 03:22:33.951397", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2Wh3zSjB3DnsF99xFrBYpqLPOM2eJTzkU7z"], "labels": [], "masters": [], "uuids": ["fe89a6bb-a0d7-45a2-9553-4b012ccbaa72"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:32.901388", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-07-18T03:22:34.079 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:34 +0000 (0:00:00.209) 0:01:57.931 ********* 2020-07-18T03:22:34.248 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9D4LDvV1NU3K9HU3fxzA4dHXzjO0jqTBM'], 'uuids': ['10a28e77-ee6c-4f6c-ac44-2019ca5dfb64'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.095566", "end": "2020-07-18 03:22:34.212763", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9D4LDvV1NU3K9HU3fxzA4dHXzjO0jqTBM"], "labels": [], "masters": [], "uuids": ["10a28e77-ee6c-4f6c-ac44-2019ca5dfb64"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.117197", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0928679 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0928679 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:34.486 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhNMhi0OI0o9hEhu4xS3OENMKa6VMG0h22'], 'uuids': ['f9bfad5e-5d82-4761-ad17-23970bfb005a'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.096355", "end": "2020-07-18 03:22:34.433876", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhNMhi0OI0o9hEhu4xS3OENMKa6VMG0h22"], "labels": [], "masters": [], "uuids": ["f9bfad5e-5d82-4761-ad17-23970bfb005a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.337521", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.093258 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.093258 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:34.502 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-4mB2EW-jxTJ-F6JD-2yKN-2XVR-HDH3-XB90vk', 'nvme-INTEL_SSDPED1D480GA_PHMB75120091480DGN', 'nvme-nvme.8086-50484d42373531323030393134383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPED1D480GA', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '937703088', 'sectorsize': '512', 'size': '447.13 GB', 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036234", "end": "2020-07-18 03:22:34.458148", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-4mB2EW-jxTJ-F6JD-2yKN-2XVR-HDH3-XB90vk", "nvme-INTEL_SSDPED1D480GA_PHMB75120091480DGN", "nvme-nvme.8086-50484d42373531323030393134383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.421914", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0335585 s, 3.4 GB/s", "stderr_lines": [" 2020-07-18T03:22:34.504 INFO:teuthology.task.ansible.out:110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0335585 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:34.747 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-stxYcT-3n0o-Wf1W-bB2l-fgrF-A2wX-ipqjgX', 'nvme-INTEL_SSDPED1D480GA_PHMB75130022480DGN', 'nvme-nvme.8086-50484d42373531333030323234383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPED1D480GA', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '937703088', 'sectorsize': '512', 'size': '447.13 GB', 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036359", "end": "2020-07-18 03:22:34.701773", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-stxYcT-3n0o-Wf1W-bB2l-fgrF-A2wX-ipqjgX", "nvme-INTEL_SSDPED1D480GA_PHMB75130022480DGN", "nvme-nvme.8086-50484d42373531333030323234383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.665414", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.033676 s, 3.4 GB/s", "stderr_lines": ["1 2020-07-18T03:22:34.749 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.033676 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:34.791 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd92Dy7oVTMA7LclThTt833o6KjhXbLBvQ0'], 'uuids': ['03ec8be4-d2ca-4c32-bf19-cac3e5eedf71'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.094714", "end": "2020-07-18 03:22:34.764837", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd92Dy7oVTMA7LclThTt833o6KjhXbLBvQ0"], "labels": [], "masters": [], "uuids": ["03ec8be4-d2ca-4c32-bf19-cac3e5eedf71"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.670123", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0920473 s, 1.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0920473 s, 1.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.060 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhJI0IO1sIhoafrcOSuC66Hk1t7WotJKC7'], 'uuids': ['ee9b5ccc-84e5-420a-9e19-ea4bc70ec74d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.094313", "end": "2020-07-18 03:22:35.030781", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhJI0IO1sIhoafrcOSuC66Hk1t7WotJKC7"], "labels": [], "masters": [], "uuids": ["ee9b5ccc-84e5-420a-9e19-ea4bc70ec74d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.936468", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0916338 s, 1.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0916338 s, 1.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.095 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9UZ2OAyjIF1HLj01eVUcvt4mhtQIv6u3j'], 'uuids': ['0389221f-6a3c-4407-9458-747e72019dab'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.095570", "end": "2020-07-18 03:22:35.060703", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9UZ2OAyjIF1HLj01eVUcvt4mhtQIv6u3j"], "labels": [], "masters": [], "uuids": ["0389221f-6a3c-4407-9458-747e72019dab"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:34.965133", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0928873 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0928873 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.370 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhebSAtKVDvexBxq8kTg0Js2ZBMVYwmATV'], 'uuids': ['eca3bb01-a57f-4cda-8226-445440e4d359'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.094892", "end": "2020-07-18 03:22:35.335880", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhebSAtKVDvexBxq8kTg0Js2ZBMVYwmATV"], "labels": [], "masters": [], "uuids": ["eca3bb01-a57f-4cda-8226-445440e4d359"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:35.240988", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0922023 s, 1.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0922023 s, 1.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.393 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9RTZYfcUrWP5n7hBZtSKC3UnoPuozg5BQ'], 'uuids': ['d3126e39-9bc5-4437-97f3-89315c91f5f9'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.094315", "end": "2020-07-18 03:22:35.360614", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9RTZYfcUrWP5n7hBZtSKC3UnoPuozg5BQ"], "labels": [], "masters": [], "uuids": ["d3126e39-9bc5-4437-97f3-89315c91f5f9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:35.266299", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0916498 s, 1.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0916498 s, 1.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.439 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQDQ', 'wwn-0x5000c500871a8c35'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HQDQ', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a8c35', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQDQ-part1', 'wwn-0x5000c500871a8c35-part1'], 'uuids': ['7d7ca31c-77d6-4023-896d-a1d87c0f49ac'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQDQ", "wwn-0x5000c500871a8c35"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQDQ-part1", "wwn-0x5000c500871a8c35-part1"], "labels": [], "masters": [], "uuids": ["7d7ca31c-77d6-4023-896d-a1d87c0f49ac"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQDQ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a8c35"}}, 2020-07-18T03:22:35.442 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-07-18T03:22:35.682 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhZTGuzu0ACqYrvxWffdA9EbltFdq46L0u'], 'uuids': ['82598bda-40d1-43eb-b79d-7536dd53d718'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.095400", "end": "2020-07-18 03:22:35.647678", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2WhZTGuzu0ACqYrvxWffdA9EbltFdq46L0u"], "labels": [], "masters": [], "uuids": ["82598bda-40d1-43eb-b79d-7536dd53d718"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:35.552278", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0927678 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0927678 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.696 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HN80', 'wwn-0x5000c500871ae812'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HN80', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871ae812', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HN80-part1', 'wwn-0x5000c500871ae812-part1'], 'uuids': ['7d7ca31c-77d6-4023-896d-a1d87c0f49ac'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HN80", "wwn-0x5000c500871ae812"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HN80-part1", "wwn-0x5000c500871ae812-part1"], "labels": [], "masters": [], "uuids": ["7d7ca31c-77d6-4023-896d-a1d87c0f49ac"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HN80", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871ae812"}}, 2020-07-18T03:22:35.698 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-07-18T03:22:35.752 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9IfZs0gP3nM7oCJScVtthe8T6jRs2kKmk'], 'uuids': ['c7352e59-9f27-4ae7-873c-fbf916de5d04'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.095155", "end": "2020-07-18 03:22:35.716630", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zVwZu7DjdWQxbLboI1EAxSTfphvC6kd9IfZs0gP3nM7oCJScVtthe8T6jRs2kKmk"], "labels": [], "masters": [], "uuids": ["c7352e59-9f27-4ae7-873c-fbf916de5d04"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:35.621475", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0924963 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0924963 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:35.808 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:35 +0000 (0:00:01.728) 0:01:59.660 ********* 2020-07-18T03:22:35.998 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2Wh3zSjB3DnsF99xFrBYpqLPOM2eJTzkU7z'], 'uuids': ['fe89a6bb-a0d7-45a2-9553-4b012ccbaa72'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.093823", "end": "2020-07-18 03:22:35.958540", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-XZUekCWeDVM4heq74OJfvATfFRnaX2Wh3zSjB3DnsF99xFrBYpqLPOM2eJTzkU7z"], "labels": [], "masters": [], "uuids": ["fe89a6bb-a0d7-45a2-9553-4b012ccbaa72"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-07-18 03:22:35.864717", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0911752 s, 1.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0911752 s, 1.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:36.045 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.237) 0:01:59.897 ********* 2020-07-18T03:22:36.104 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-07-18T03:22:36.106 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.069312", "end": "2020-07-18 03:22:36.070539", "rc": 0, "start": "2020-07-18 03:22:36.001227", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:36.147 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.101) 0:01:59.999 ********* 2020-07-18T03:22:36.370 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.074203", "end": "2020-07-18 03:22:36.327485", "rc": 0, "start": "2020-07-18 03:22:36.253282", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:36.408 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.260) 0:02:00.260 ********* 2020-07-18T03:22:36.429 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-07-18T03:22:36.431 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.057872", "end": "2020-07-18 03:22:36.407289", "rc": 0, "start": "2020-07-18 03:22:36.349417", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:36.469 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.061) 0:02:00.321 ********* 2020-07-18T03:22:36.501 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-07-18T03:22:36.542 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.073) 0:02:00.394 ********* 2020-07-18T03:22:36.600 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-07-18T03:22:36.602 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:36.654 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.111) 0:02:00.506 ********* 2020-07-18T03:22:36.685 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-07-18T03:22:36.687 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.054252", "end": "2020-07-18 03:22:36.657971", "rc": 0, "start": "2020-07-18 03:22:36.603719", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:36.723 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.069) 0:02:00.575 ********* 2020-07-18T03:22:36.772 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.048) 0:02:00.624 ********* 2020-07-18T03:22:36.791 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:36.835 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.063) 0:02:00.688 ********* 2020-07-18T03:22:36.847 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-07-18T03:22:36.888 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.052) 0:02:00.740 ********* 2020-07-18T03:22:36.889 INFO:teuthology.task.ansible.out: 2020-07-18T03:22:36.901 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-07-18T03:22:36.902 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:36.941 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.052) 0:02:00.793 ********* 2020-07-18T03:22:36.951 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:36.998 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:36 +0000 (0:00:00.057) 0:02:00.850 ********* 2020-07-18T03:22:37.052 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-07-18T03:22:37.054 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.093 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.095) 0:02:00.946 ********* 2020-07-18T03:22:37.124 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-07-18T03:22:37.126 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.162 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.068) 0:02:01.014 ********* 2020-07-18T03:22:37.213 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-07-18T03:22:37.215 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.250 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.087) 0:02:01.102 ********* 2020-07-18T03:22:37.287 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.037) 0:02:01.139 ********* 2020-07-18T03:22:37.308 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-07-18T03:22:37.317 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930532024320, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180670, 'block_used': 12914438, 'inode_total': 61054976, 'inode_available': 60990206, 'inode_used': 64770, 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180670, "block_size": 4096, "block_total": 240095108, "block_used": 12914438, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990206, "inode_total": 61054976, "inode_used": 64770, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930532024320, "size_total": 983429562368, "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.358 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.070) 0:02:01.210 ********* 2020-07-18T03:22:37.401 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.439 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.080) 0:02:01.291 ********* 2020-07-18T03:22:37.441 INFO:teuthology.task.ansible.out: 2020-07-18T03:22:37.485 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:37.487 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.522 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.082) 0:02:01.374 ********* 2020-07-18T03:22:37.534 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-07-18T03:22:37.538 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-07-18T03:22:37.572 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.050) 0:02:01.424 ********* 2020-07-18T03:22:37.585 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.643 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.070) 0:02:01.495 ********* 2020-07-18T03:22:37.655 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-07-18T03:22:37.656 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930532028416, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180671, 'block_used': 12914437, 'inode_total': 61054976, 'inode_available': 60990206, 'inode_used': 64770, 'uuid': '7d7ca31c-77d6-4023-896d-a1d87c0f49ac'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180671, "block_size": 4096, "block_total": 240095108, "block_used": 12914437, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990206, "inode_total": 61054976, "inode_used": 64770, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930532028416, "size_total": 983429562368, "uuid": "7d7ca31c-77d6-4023-896d-a1d87c0f49ac"}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.703 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.057) 0:02:01.552 ********* 2020-07-18T03:22:37.713 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-07-18T03:22:37.714 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.752 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.051) 0:02:01.604 ********* 2020-07-18T03:22:37.763 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-07-18T03:22:37.766 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.807 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.048) 0:02:01.653 ********* 2020-07-18T03:22:37.822 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-07-18T03:22:37.824 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.866 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.065) 0:02:01.718 ********* 2020-07-18T03:22:37.882 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.930 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.063) 0:02:01.782 ********* 2020-07-18T03:22:37.941 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-07-18T03:22:37.943 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:37.979 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:37 +0000 (0:00:00.049) 0:02:01.831 ********* 2020-07-18T03:22:37.992 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:38.038 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:38 +0000 (0:00:00.058) 0:02:01.890 ********* 2020-07-18T03:22:38.086 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:38.126 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:38 +0000 (0:00:00.085) 0:02:01.976 ********* 2020-07-18T03:22:38.192 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-07-18T03:22:38.194 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:38.246 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:38 +0000 (0:00:00.121) 0:02:02.098 ********* 2020-07-18T03:22:38.293 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:38.334 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:38 +0000 (0:00:00.087) 0:02:02.186 ********* 2020-07-18T03:22:38.653 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-07-18T03:22:38.656 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-07-18T03:22:38.702 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:38 +0000 (0:00:00.368) 0:02:02.554 ********* 2020-07-18T03:22:38.877 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-07-18T03:22:38.937 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:38 +0000 (0:00:00.234) 0:02:02.789 ********* 2020-07-18T03:22:39.500 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-07-18T03:22:39.502 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:39.528 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:39.843 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:39.876 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:40.207 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:40.262 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:40.583 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:40.648 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:40.938 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:40.996 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:40 +0000 (0:00:02.058) 0:02:04.848 ********* 2020-07-18T03:22:41.010 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-07-18T03:22:41.063 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:41 +0000 (0:00:00.066) 0:02:04.915 ********* 2020-07-18T03:22:41.226 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-07-18T03:22:41.228 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-07-18T03:22:41.265 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:41 +0000 (0:00:00.202) 0:02:05.117 ********* 2020-07-18T03:22:41.314 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-07-18T03:22:41.354 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:41 +0000 (0:00:00.089) 0:02:05.206 ********* 2020-07-18T03:22:41.530 INFO:teuthology.task.ansible.out: 2020-07-18T03:22:41.532 INFO:teuthology.task.ansible.out:TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:41.611 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:41.764 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:41.824 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:41.962 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:42.030 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:42.165 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:42.187 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:42.229 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:42 +0000 (0:00:00.874) 0:02:06.081 ********* 2020-07-18T03:22:42.254 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-07-18T03:22:42.255 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:42.301 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:42 +0000 (0:00:00.072) 0:02:06.154 ********* 2020-07-18T03:22:42.458 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-07-18T03:22:42.460 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-07-18T03:22:42.509 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:42 +0000 (0:00:00.207) 0:02:06.361 ********* 2020-07-18T03:22:42.552 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-07-18T03:22:42.589 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:42 +0000 (0:00:00.078) 0:02:06.439 ********* 2020-07-18T03:22:42.599 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-07-18T03:22:42.600 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-07-18T03:22:42.639 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:42 +0000 (0:00:00.050) 0:02:06.490 ********* 2020-07-18T03:22:42.650 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-07-18T03:22:42.691 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:42 +0000 (0:00:00.053) 0:02:06.543 ********* 2020-07-18T03:22:43.274 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-07-18T03:22:43.277 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true} changed: [smithi047.front.sepia.ceph.com] => {"changed": true} 2020-07-18T03:22:43.322 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:43 +0000 (0:00:00.631) 0:02:07.174 ********* 2020-07-18T03:22:43.385 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:43 +0000 (0:00:00.062) 0:02:07.237 ********* 2020-07-18T03:22:44.047 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-07-18T03:22:44.083 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.698) 0:02:07.935 ********* 2020-07-18T03:22:44.094 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-07-18T03:22:44.144 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.060) 0:02:07.996 ********* 2020-07-18T03:22:44.170 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.213 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.069) 0:02:08.065 ********* 2020-07-18T03:22:44.224 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.270 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.053) 0:02:08.119 ********* 2020-07-18T03:22:44.278 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-07-18T03:22:44.279 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.331 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.064) 0:02:08.183 ********* 2020-07-18T03:22:44.344 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.403 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.072) 0:02:08.255 ********* 2020-07-18T03:22:44.420 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.470 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.066) 0:02:08.322 ********* 2020-07-18T03:22:44.489 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.529 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.058) 0:02:08.381 ********* 2020-07-18T03:22:44.539 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-07-18T03:22:44.541 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.585 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.056) 0:02:08.437 ********* 2020-07-18T03:22:44.597 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.642 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.057) 0:02:08.494 ********* 2020-07-18T03:22:44.653 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-07-18T03:22:44.655 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.701 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.058) 0:02:08.553 ********* 2020-07-18T03:22:44.711 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.750 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.049) 0:02:08.603 ********* 2020-07-18T03:22:44.761 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-07-18T03:22:44.763 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.805 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.054) 0:02:08.657 ********* 2020-07-18T03:22:44.815 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.861 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.055) 0:02:08.713 ********* 2020-07-18T03:22:44.871 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.914 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.053) 0:02:08.766 ********* 2020-07-18T03:22:44.925 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:44.986 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:44 +0000 (0:00:00.071) 0:02:08.838 ********* 2020-07-18T03:22:44.997 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-07-18T03:22:44.999 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.034 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.048) 0:02:08.887 ********* 2020-07-18T03:22:45.048 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.089 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.054) 0:02:08.941 ********* 2020-07-18T03:22:45.102 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-07-18T03:22:45.104 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.144 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.054) 0:02:08.996 ********* 2020-07-18T03:22:45.162 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.220 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.076) 0:02:09.072 ********* 2020-07-18T03:22:45.234 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.289 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.069) 0:02:09.142 ********* 2020-07-18T03:22:45.302 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.357 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.066) 0:02:09.208 ********* 2020-07-18T03:22:45.367 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-07-18T03:22:45.369 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.433 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.077) 0:02:09.285 ********* 2020-07-18T03:22:45.444 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:45.488 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:45 +0000 (0:00:00.054) 0:02:09.340 ********* 2020-07-18T03:22:46.032 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-07-18T03:22:46.038 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-07-18T03:22:46.113 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.624) 0:02:09.964 ********* 2020-07-18T03:22:46.123 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-07-18T03:22:46.164 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.051) 0:02:10.016 ********* 2020-07-18T03:22:46.472 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-07-18T03:22:46.473 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ActiveEnterTimestampMonotonic": "19372412", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target ntpdate.service sntp.service systemd-tmpfiles-setup.service ntpd.service sysinit.target systemd-journald.socket tmp.mount -.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-07-18 03:19:33 UTC", "AssertTimestampMonotonic": "18328712", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-07-18 03:19:33 UTC", "ConditionTimestampMonotonic": "18328688", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-07-18T03:22:46.475 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2404", "ExecMainStartTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ExecMainStartTimestampMonotonic": "19332535", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[Sat 2020-07-18 03:19:33 UTC] ; stop_time=[Sat 2020-07-18 03:19:34 UTC] ; pid=2401 ; code=exited ; status=0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[Sat 2020-07-18 03:19:34 UTC] ; stop_time=[Sat 2020-07-18 03:19:34 UTC] ; pid=2476 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-07-18 03:19:33 UTC", "InactiveExitTimestampMonotonic": "18415205", "InvocationID": "abf23437c887414ab4a82729cda0cdac", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZE 2020-07-18T03:22:46.476 INFO:teuthology.task.ansible.out:Soft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2404", "MemoryAccounting": "yes", "MemoryCurrent": "2715648", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no" 2020-07-18T03:22:46.478 INFO:teuthology.task.ansible.out:, "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-07-18 03:19:34 UTC", "StateChangeTimestampMonotonic": "19372412", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-07-18 03:19:34 UTC", "WatchdogTimestampMonotonic": "19332550", "WatchdogUSec": "0"}} 2020-07-18T03:22:46.512 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.348) 0:02:10.364 ********* 2020-07-18T03:22:46.526 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ActiveEnterTimestampMonotonic": "20007482", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket system.slice tmp.mount ntpdate.service sysinit.target basic.target systemd-tmpfiles-setup.service sntp.service -.mount ntpd.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-07-18 03:19:33 UTC", "AssertTimestampMonotonic": "18876076", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-07-18 03:19:33 UTC", "ConditionTimestampMonotonic": "18876047", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service ntpd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-07-18T03:22:46.528 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2450", "ExecMainStartTimestamp": "Sat 2020-07-18 03:19:34 UTC", "ExecMainStartTimestampMonotonic": "19881241", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[Sat 2020-07-18 03:19:33 UTC] ; stop_time=[Sat 2020-07-18 03:19:34 UTC] ; pid=2444 ; code=exited ; status=0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[Sat 2020-07-18 03:19:34 UTC] ; stop_time=[Sat 2020-07-18 03:19:34 UTC] ; pid=2509 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-07-18 03:19:33 UTC", "InactiveExitTimestampMonotonic": "18884400", "InvocationID": "ca906dad068c47388d10fd59bda29b75", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZE 2020-07-18T03:22:46.530 INFO:teuthology.task.ansible.out:Soft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2450", "MemoryAccounting": "yes", "MemoryCurrent": "2543616", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no" 2020-07-18T03:22:46.531 INFO:teuthology.task.ansible.out:, "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-07-18 03:19:34 UTC", "StateChangeTimestampMonotonic": "20007482", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-07-18 03:19:34 UTC", "WatchdogTimestampMonotonic": "19881255", "WatchdogUSec": "0"}} 2020-07-18T03:22:46.570 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.058) 0:02:10.422 ********* 2020-07-18T03:22:46.581 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:46.583 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-07-18T03:22:46.635 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.064) 0:02:10.487 ********* 2020-07-18T03:22:46.647 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-07-18T03:22:46.686 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.051) 0:02:10.538 ********* 2020-07-18T03:22:46.698 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:46.701 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-07-18T03:22:46.741 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.055) 0:02:10.593 ********* 2020-07-18T03:22:46.765 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-07-18T03:22:46.800 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.058) 0:02:10.652 ********* 2020-07-18T03:22:46.820 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-07-18T03:22:46.822 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:46.863 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.060) 0:02:10.712 ********* 2020-07-18T03:22:46.870 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:46.907 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.047) 0:02:10.760 ********* 2020-07-18T03:22:46.918 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-07-18T03:22:46.919 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:46.974 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:46 +0000 (0:00:00.064) 0:02:10.825 ********* 2020-07-18T03:22:46.984 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:47.024 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:47 +0000 (0:00:00.051) 0:02:10.876 ********* 2020-07-18T03:22:47.035 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-07-18T03:22:47.037 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:47.074 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:47 +0000 (0:00:00.049) 0:02:10.926 ********* 2020-07-18T03:22:47.085 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:47.128 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:47 +0000 (0:00:00.054) 0:02:10.981 ********* 2020-07-18T03:22:47.311 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-07-18T03:22:47.313 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "gid": 1224, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-07-18T03:22:47.356 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:47 +0000 (0:00:00.227) 0:02:11.208 ********* 2020-07-18T03:22:47.377 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "gid": 1224, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-07-18T03:22:47.444 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:47 +0000 (0:00:00.087) 0:02:11.296 ********* 2020-07-18T03:22:47.943 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-07-18T03:22:47.945 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1224, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-07-18T03:22:47.990 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:47 +0000 (0:00:00.545) 0:02:11.842 ********* 2020-07-18T03:22:48.038 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1224, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-07-18T03:22:48.081 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:48 +0000 (0:00:00.091) 0:02:11.933 ********* 2020-07-18T03:22:49.471 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-07-18T03:22:49.473 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python 2020-07-18T03:22:49.485 INFO:teuthology.task.ansible.out:3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-07-18T03:22:49.508 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:01.426) 0:02:13.360 ********* 2020-07-18T03:22:49.550 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:00.042) 0:02:13.402 ********* 2020-07-18T03:22:49.564 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-07-18T03:22:49.565 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:49.603 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:00.053) 0:02:13.455 ********* 2020-07-18T03:22:49.614 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:49.654 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:00.050) 0:02:13.506 ********* 2020-07-18T03:22:49.666 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-07-18T03:22:49.717 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:00.063) 0:02:13.569 ********* 2020-07-18T03:22:49.729 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-07-18T03:22:49.770 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:00.052) 0:02:13.622 ********* 2020-07-18T03:22:49.939 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi018.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-07-18T03:22:49.990 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:49 +0000 (0:00:00.219) 0:02:13.841 ********* 2020-07-18T03:22:50.013 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-07-18T03:22:50.051 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.061) 0:02:13.903 ********* 2020-07-18T03:22:50.062 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-07-18T03:22:50.063 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:50.109 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.058) 0:02:13.961 ********* 2020-07-18T03:22:50.121 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:50.159 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.050) 0:02:14.012 ********* 2020-07-18T03:22:50.171 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-07-18T03:22:50.173 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-07-18T03:22:50.218 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.055) 0:02:14.067 ********* 2020-07-18T03:22:50.227 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-07-18T03:22:50.268 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.053) 0:02:14.120 ********* 2020-07-18T03:22:50.285 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:50.328 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.060) 0:02:14.180 ********* 2020-07-18T03:22:50.341 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:50.389 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.060) 0:02:14.241 ********* 2020-07-18T03:22:50.556 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi018.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-07-18T03:22:50.619 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.230) 0:02:14.471 ********* 2020-07-18T03:22:50.645 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-07-18T03:22:50.687 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.067) 0:02:14.539 ********* 2020-07-18T03:22:50.849 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-07-18T03:22:50.851 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-07-18T03:22:50.887 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.199) 0:02:14.739 ********* 2020-07-18T03:22:50.928 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-07-18T03:22:50.929 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:50.980 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:50 +0000 (0:00:00.088) 0:02:14.828 ********* 2020-07-18T03:22:50.987 INFO:teuthology.task.ansible.out: 2020-07-18T03:22:50.991 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi047.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-07-18T03:22:51.031 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.054) 0:02:14.883 ********* 2020-07-18T03:22:51.048 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-07-18T03:22:51.050 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.052 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.088 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.056) 0:02:14.940 ********* 2020-07-18T03:22:51.098 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-07-18T03:22:51.100 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.148 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.060) 0:02:15.000 ********* 2020-07-18T03:22:51.159 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-07-18T03:22:51.161 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi018.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.162 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.174 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.221 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.072) 0:02:15.073 ********* 2020-07-18T03:22:51.230 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-07-18T03:22:51.231 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.233 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi047.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.233 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.276 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.052) 0:02:15.125 ********* 2020-07-18T03:22:51.285 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.329 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.055) 0:02:15.181 ********* 2020-07-18T03:22:51.350 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-07-18T03:22:51.352 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.353 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.353 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.354 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.354 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result 2020-07-18T03:22:51.356 INFO:teuthology.task.ansible.out: was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.394 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.065) 0:02:15.246 ********* 2020-07-18T03:22:51.405 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-07-18T03:22:51.406 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.447 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.052) 0:02:15.299 ********* 2020-07-18T03:22:51.457 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-07-18T03:22:51.458 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.506 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.055) 0:02:15.354 ********* 2020-07-18T03:22:51.612 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Saturday 18 July 2020 03:22:51 +0000 (0:00:00.063) 0:02:15.417 ********* TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.631 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.065) 0:02:15.483 ********* 2020-07-18T03:22:51.641 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-07-18T03:22:51.643 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.684 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.053) 0:02:15.536 ********* 2020-07-18T03:22:51.696 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-07-18T03:22:51.698 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.762 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.077) 0:02:15.614 ********* 2020-07-18T03:22:51.774 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-07-18T03:22:51.776 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.833 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.071) 0:02:15.685 ********* 2020-07-18T03:22:51.844 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-07-18T03:22:51.846 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.888 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.054) 0:02:15.739 ********* 2020-07-18T03:22:51.900 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-07-18T03:22:51.902 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:51.975 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:51 +0000 (0:00:00.086) 0:02:15.827 ********* 2020-07-18T03:22:51.986 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-07-18T03:22:51.988 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:52.044 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:52 +0000 (0:00:00.069) 0:02:15.896 ********* 2020-07-18T03:22:52.057 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-07-18T03:22:52.058 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:52.103 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:52 +0000 (0:00:00.058) 0:02:15.955 ********* 2020-07-18T03:22:52.116 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:52.163 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:52 +0000 (0:00:00.059) 0:02:16.015 ********* 2020-07-18T03:22:52.173 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-07-18T03:22:52.175 INFO:teuthology.task.ansible.out:skipping: [smithi018.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:52.213 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:52 +0000 (0:00:00.050) 0:02:16.065 ********* 2020-07-18T03:22:52.228 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-07-18T03:22:52.230 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:52.276 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:52 +0000 (0:00:00.062) 0:02:16.127 ********* 2020-07-18T03:22:52.319 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-07-18T03:22:52.321 INFO:teuthology.task.ansible.out:skipping: [smithi047.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-07-18T03:22:52.371 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:52 +0000 (0:00:00.093) 0:02:16.221 ********* 2020-07-18T03:22:52.647 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-07-18T03:22:52.649 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.005295", "end": "2020-07-18 03:22:52.598026", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-07-18 03:22:52.592731", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:52.860 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003234", "end": "2020-07-18 03:22:52.821191", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-07-18 03:22:52.817957", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:53.083 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002920", "end": "2020-07-18 03:22:53.034476", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-07-18 03:22:53.031556", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:53.317 INFO:teuthology.task.ansible.out:ok: [smithi047.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.024201", "end": "2020-07-18 03:22:53.287470", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-07-18 03:22:53.263269", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:53.362 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:53 +0000 (0:00:00.992) 0:02:17.214 ********* 2020-07-18T03:22:53.402 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-07-18T03:22:53.440 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:53 +0000 (0:00:00.078) 0:02:17.292 ********* 2020-07-18T03:22:53.483 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-07-18T03:22:53.485 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.006522", "end": "2020-07-18 03:22:53.435047", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-07-18 03:22:52.428525", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:53.668 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-07-18T03:22:53.674 INFO:teuthology.task.ansible.out:changed: [smithi047.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-07-18T03:22:54.705 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-07-18T03:22:54.707 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:01.004297", "end": "2020-07-18 03:22:54.657506", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-07-18 03:22:53.653209", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:54.946 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.003066", "end": "2020-07-18 03:22:54.898081", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-07-18 03:22:54.895015", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:56.163 INFO:teuthology.task.ansible.out:ok: [smithi018.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:01.020494", "end": "2020-07-18 03:22:56.131103", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-07-18 03:22:55.110609", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-07-18T03:22:56.226 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:56 +0000 (0:00:02.785) 0:02:20.078 ********* 2020-07-18T03:22:56.275 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-07-18T03:22:56.308 INFO:teuthology.task.ansible.out:Saturday 18 July 2020 03:22:56 +0000 (0:00:00.082) 0:02:20.160 ********* 2020-07-18T03:22:56.557 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-07-18T03:22:56.559 INFO:teuthology.task.ansible.out:changed: [smithi018.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-07-18T03:22:56.676 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-07-18T03:22:56.679 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-07-18T03:22:56.683 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-07-18T03:22:56.698 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi018.front.sepia.ceph.com : ok=111 changed=30 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 smithi047.front.sepia.ceph.com : ok=111 changed=30 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 Saturday 18 July 2020 03:22:56 +0000 (0:00:00.375) 0:02:20.536 ********* =============================================================================== common : nrpe - Remove SELinux policy package -------------------------- 17.31s common : nrpe - Load SELinux policy package ---------------------------- 16.96s testnode : Zap all non-root disks --------------------------------------- 8.03s testnode : Install packages --------------------------------------------- 3.17s testnode : Ensure ceph packages are not present. ------------------------ 2.38s testnode : Create logical volume(s) ------------------------------------- 2.06s testnode : Ensure ceph dependency packages are not present. ------------- 2.04s testnode : Rebuild rpmdb ------------------------------------------------ 2.01s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.73s Gathering Facts --------------------------------------------------------- 1.63s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.63s testnode : Configure local mirrorlists ---------------------------------- 1.62s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.59s common : nrpe - Set SELinux boolean nagios_run_sudo true ---------------- 1.56s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.53s testnode : Install packages via pip ------------------------------------- 1.43s common : Install Kerberos Packages (RedHat) ----------------------------- 1.36s common : Configure local epel mirrorlists ------------------------------- 1.18s common : Config 2020-07-18T03:22:56.700 INFO:teuthology.task.ansible.out:ure epel repos in /etc/yum.repos.d/ ---------------------- 1.14s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.12s 2020-07-18T03:22:57.120 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi018.front.sepia.ceph.com')] 2020-07-18T03:22:57.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-07-18T03:22:57.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:22:57.220 INFO:teuthology.orchestra.run.smithi047:> true 2020-07-18T03:22:57.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-07-18T03:22:57.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-07-18T03:22:57.391 INFO:teuthology.orchestra.run.smithi018:> true 2020-07-18T03:22:57.481 INFO:teuthology.run_tasks:Running task clock... 2020-07-18T03:22:58.708 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-07-18T03:22:58.709 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-07-18T03:22:58.710 INFO:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-07-18T03:22:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-07-18T03:22:58.780 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-07-18T03:22:58.833 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2020-07-18T03:22:58.851 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2020-07-18T03:22:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-07-18T03:22:58.894 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-07-18T03:22:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2020-07-18T03:22:58.965 INFO:teuthology.orchestra.run.smithi047.stdout:210 Number of sources = 4 2020-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2020-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:58.967 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-07-18T03:22:59.015 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-07-18T03:22:59.037 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-07-18T03:22:59.086 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2020-07-18T03:22:59.104 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2020-07-18T03:22:59.126 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-07-18T03:22:59.147 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-07-18T03:22:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:bash: ntpq: command not found 2020-07-18T03:22:59.212 INFO:teuthology.orchestra.run.smithi018.stdout:210 Number of sources = 4 2020-07-18T03:22:59.212 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-07-18T03:22:59.213 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2020-07-18T03:22:59.213 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:59.213 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:59.213 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:59.213 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-07-18T03:22:59.214 INFO:teuthology.run_tasks:Running task install... 2020-07-18T03:22:59.244 DEBUG:teuthology.task.install:project ceph 2020-07-18T03:22:59.247 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d'}} 2020-07-18T03:22:59.247 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d'} 2020-07-18T03:22:59.248 INFO:teuthology.task.install:Using flavor: basic 2020-07-18T03:22:59.265 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-07-18T03:22:59.266 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-07-18T03:22:59.267 INFO:teuthology.orchestra.run.smithi047:> sudo lsb_release -is 2020-07-18T03:22:59.313 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS 2020-07-18T03:22:59.315 DEBUG:teuthology.misc:System to be installed: CentOS 2020-07-18T03:22:59.316 INFO:teuthology.orchestra.run.smithi018:> sudo lsb_release -is 2020-07-18T03:22:59.316 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d', 'debuginfo': None, 'flavor': 'basic', 'downgrade_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-07-18T03:22:59.317 INFO:teuthology.orchestra.run.smithi047:> sudo lsb_release -is 2020-07-18T03:22:59.370 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS 2020-07-18T03:22:59.371 DEBUG:teuthology.misc:System to be installed: CentOS 2020-07-18T03:22:59.372 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d', 'debuginfo': None, 'flavor': 'basic', 'downgrade_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-07-18T03:22:59.372 INFO:teuthology.orchestra.run.smithi018:> sudo lsb_release -is 2020-07-18T03:22:59.395 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS 2020-07-18T03:22:59.396 DEBUG:teuthology.misc:System to be installed: CentOS 2020-07-18T03:22:59.397 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0baf19aeafe8fa5b354a28b0fb65984f752d8b9d 2020-07-18T03:22:59.450 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS 2020-07-18T03:22:59.451 DEBUG:teuthology.misc:System to be installed: CentOS 2020-07-18T03:22:59.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0baf19aeafe8fa5b354a28b0fb65984f752d8b9d 2020-07-18T03:22:59.587 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/ 2020-07-18T03:22:59.588 INFO:teuthology.task.install.rpm:Package version is 16.0.0-3461.g0baf19a 2020-07-18T03:22:59.638 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/ 2020-07-18T03:22:59.639 INFO:teuthology.task.install.rpm:Package version is 16.0.0-3461.g0baf19a 2020-07-18T03:22:59.727 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-07-18T03:22:59.728 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-07-18T03:22:59.775 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mds-inode-smartptr/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-07-18T03:22:59.776 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-07-18T03:22:59.779 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-07-18T03:22:59.779 INFO:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-07-18T03:22:59.825 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-07-18T03:22:59.826 INFO:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0baf19aeafe8fa5b354a28b0fb65984f752d8b9d/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-07-18T03:22:59.852 INFO:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-07-18T03:22:59.899 INFO:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-07-18T03:22:59.941 INFO:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-07-18T03:22:59.974 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2020-07-18T03:22:59.975 INFO:teuthology.orchestra.run.smithi047:> sudo yum clean all 2020-07-18T03:22:59.988 INFO:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-07-18T03:23:00.021 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2020-07-18T03:23:00.022 INFO:teuthology.orchestra.run.smithi018:> sudo yum clean all 2020-07-18T03:23:00.270 INFO:teuthology.orchestra.run.smithi047.stdout:41 files removed 2020-07-18T03:23:00.300 INFO:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-07-18T03:23:00.318 INFO:teuthology.orchestra.run.smithi018.stdout:41 files removed 2020-07-18T03:23:00.348 INFO:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-07-18T03:23:00.919 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 287 kB/s | 71 kB 00:00 2020-07-18T03:23:00.965 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 290 kB/s | 71 kB 00:00 2020-07-18T03:23:01.156 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 72 kB/s | 15 kB 00:00 2020-07-18T03:23:01.202 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 72 kB/s | 15 kB 00:00 2020-07-18T03:23:01.384 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-07-18T03:23:01.430 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-07-18T03:23:01.553 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-8 - AppStream 47 MB/s | 7.0 MB 00:00 2020-07-18T03:23:01.615 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - AppStream 43 MB/s | 7.0 MB 00:00 2020-07-18T03:23:02.810 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-07-18T03:23:02.892 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-07-18T03:23:03.273 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-8 - Extras 87 kB/s | 5.5 kB 00:00 2020-07-18T03:23:03.344 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Extras 82 kB/s | 5.5 kB 00:00 2020-07-18T03:23:03.702 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 20 MB/s | 7.4 MB 00:00 2020-07-18T03:23:03.708 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 24 MB/s | 7.4 MB 00:00 2020-07-18T03:23:05.312 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 3.9 MB/s | 20 kB 00:00 2020-07-18T03:23:05.317 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 4.0 MB/s | 20 kB 00:00 2020-07-18T03:23:06.256 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-07-18T03:23:06.256 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-07-18T03:23:06.257 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-07-18T03:23:06.257 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-07-18T03:23:06.267 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.20.0-3.el8.x86_64 is already installed. 2020-07-18T03:23:06.268 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-4.18.0-15.el8.x86_64 is already installed. 2020-07-18T03:23:06.268 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.32.1-17.el8.x86_64 is already installed. 2020-07-18T03:23:06.268 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-07-18T03:23:06.268 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-07-18T03:23:06.268 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-07-18T03:23:06.269 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-07-18T03:23:06.269 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-1:1.20.0-3.el8.x86_64 is already installed. 2020-07-18T03:23:06.269 INFO:teuthology.orchestra.run.smithi018.stdout:Package iproute-4.18.0-15.el8.x86_64 is already installed. 2020-07-18T03:23:06.269 INFO:teuthology.orchestra.run.smithi018.stdout:Package util-linux-2.32.1-17.el8.x86_64 is already installed. 2020-07-18T03:23:06.269 INFO:teuthology.orchestra.run.smithi018.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-07-18T03:23:06.270 INFO:teuthology.orchestra.run.smithi018.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-07-18T03:23:06.270 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-07-18T03:23:06.270 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-07-18T03:23:06.398 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2020-07-18T03:23:06.399 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2020-07-18T03:23:06.417 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2020-07-18T03:23:06.418 INFO:teuthology.orchestra.run.smithi018.stdout: Package ArchVersion Repository Size 2020-07-18T03:23:06.418 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2020-07-18T03:23:06.418 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2020-07-18T03:23:06.418 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_642:16.0.0-3461.g0baf19a.el8 ceph 6.4 k 2020-07-18T03:23:06.419 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_642:16.0.0-3461.g0baf19a.el8 ceph 6.0 M 2020-07-18T03:23:06.419 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_642:16.0.0-3461.g0baf19a.el8 ceph 604 k 2020-07-18T03:23:06.419 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_642:16.0.0-3461.g0baf19a.el8 ceph 202 k 2020-07-18T03:23:06.419 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_642:16.0.0-3461.g0baf19a.el8 ceph 1.5 M 2020-07-18T03:23:06.419 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_642:16.0.0-3461.g0baf19a.el8 ceph 10 M 2020-07-18T03:23:06.420 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_642:16.0.0-3461.g0baf19a.el8 ceph 45 M 2020-07-18T03:23:06.420 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm x86_642:16.0.0-3461.g0baf19a.el8 ceph 48 k 2020-07-18T03:23:06.420 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_642:16.0.0-3461.g0baf19a.el8 ceph 23 k 2020-07-18T03:23:06.420 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_642:16.0.0-3461.g0baf19a.el8 ceph 536 k 2020-07-18T03:23:06.420 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_642:16.0.0-3461.g0baf19a.el8 ceph 131 k 2020-07-18T03:23:06.421 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_642:16.0.0-3461.g0baf19a.el8 ceph 204 k 2020-07-18T03:23:06.421 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_642:16.0.0-3461.g0baf19a.el8 ceph 364 k 2020-07-18T03:23:06.421 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_642:16.0.0-3461.g0baf19a.el8 ceph 334 k 2020-07-18T03:23:06.421 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_642:16.0.0-3461.g0baf19a.el8 ceph 113 k 2020-07-18T03:23:06.422 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_642:16.0.0-3461.g0baf19a.el8 ceph 87 k 2020-07-18T03:23:06.422 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_642:16.0.0-3461.g0baf19a.el8 ceph 2.7 M 2020-07-18T03:23:06.422 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_642:16.0.0-3461.g0baf19a.el8 ceph 161 k 2020-07-18T03:23:06.422 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 54 k 2020-07-18T03:23:06.423 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 3.5 M 2020-07-18T03:23:06.423 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 8.6 M 2020-07-18T03:23:06.423 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 27 k 2020-07-18T03:23:06.423 INFO:teuthology.orchestra.run.smithi018.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-07-18T03:23:06.423 INFO:teuthology.orchestra.run.smithi018.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-07-18T03:23:06.424 INFO:teuthology.orchestra.run.smithi018.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-07-18T03:23:06.424 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-07-18T03:23:06.424 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-07-18T03:23:06.424 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-07-18T03:23:06.424 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-07-18T03:23:06.425 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-07-18T03:23:06.425 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2020-07-18T03:23:06.425 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_642:16.0.0-3461.g0baf19a.el8 ceph 3.7 M 2020-07-18T03:23:06.425 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_642:16.0.0-3461.g0baf19a.el8 ceph 2.0 M 2020-07-18T03:23:06.425 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager x86_641:1.20.0-5.el8_1 CentOS-Base 2.1 M 2020-07-18T03:23:06.426 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm x86_641:1.20.0-5.el8_1 CentOS-Base 1.7 M 2020-07-18T03:23:06.426 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team x86_641:1.20.0-5.el8_1 CentOS-Base 133 k 2020-07-18T03:23:06.426 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui x86_641:1.20.0-5.el8_1 CentOS-Base 308 k 2020-07-18T03:23:06.426 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2020-07-18T03:23:06.426 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_642:16.0.0-3461.g0baf19a.el8 ceph 22 M 2020-07-18T03:23:06.427 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_642:16.0.0-3461.g0baf19a.el8 ceph 2.2 M 2020-07-18T03:23:06.427 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_642:16.0.0-3461.g0baf19a.el8 ceph 4.9 M 2020-07-18T03:23:06.427 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_642:16.0.0-3461.g0baf19a.el8 ceph 18 M 2020-07-18T03:23:06.427 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_642:16.0.0-3461.g0baf19a.el8 ceph 25 k 2020-07-18T03:23:06.428 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_642:16.0.0-3461.g0baf19a.el8 ceph 483 k 2020-07-18T03:23:06.428 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_642:16.0.0-3461.g0baf19a.el8 ceph 3.4 M 2020-07-18T03:23:06.428 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_642:16.0.0-3461.g0baf19a.el8 ceph 40 k 2020-07-18T03:23:06.428 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_642:16.0.0-3461.g0baf19a.el8 ceph 67 k 2020-07-18T03:23:06.428 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 24 k 2020-07-18T03:23:06.429 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 174 k 2020-07-18T03:23:06.429 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 8.7 k 2020-07-18T03:23:06.429 INFO:teuthology.orchestra.run.smithi018.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-07-18T03:23:06.429 INFO:teuthology.orchestra.run.smithi018.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-07-18T03:23:06.429 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-07-18T03:23:06.430 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-07-18T03:23:06.430 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-07-18T03:23:06.430 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-07-18T03:23:06.430 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-07-18T03:23:06.430 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-07-18T03:23:06.431 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-07-18T03:23:06.431 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-07-18T03:23:06.431 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-07-18T03:23:06.431 INFO:teuthology.orchestra.run.smithi018.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-07-18T03:23:06.431 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-07-18T03:23:06.432 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-07-18T03:23:06.432 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-07-18T03:23:06.432 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-07-18T03:23:06.432 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-07-18T03:23:06.432 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-07-18T03:23:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-07-18T03:23:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-07-18T03:23:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-07-18T03:23:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-07-18T03:23:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-07-18T03:23:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-07-18T03:23:06.434 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-07-18T03:23:06.434 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-07-18T03:23:06.434 INFO:teuthology.orchestra.run.smithi018.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-07-18T03:23:06.434 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-07-18T03:23:06.434 INFO:teuthology.orchestra.run.smithi018.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-07-18T03:23:06.435 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-07-18T03:23:06.435 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-07-18T03:23:06.435 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-07-18T03:23:06.435 INFO:teuthology.orchestra.run.smithi018.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-07-18T03:23:06.435 INFO:teuthology.orchestra.run.smithi018.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-07-18T03:23:06.436 INFO:teuthology.orchestra.run.smithi018.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-07-18T03:23:06.436 INFO:teuthology.orchestra.run.smithi018.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-07-18T03:23:06.436 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-07-18T03:23:06.436 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-07-18T03:23:06.436 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-07-18T03:23:06.437 INFO:teuthology.orchestra.run.smithi018.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-07-18T03:23:06.437 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-07-18T03:23:06.437 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-07-18T03:23:06.437 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-07-18T03:23:06.437 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-07-18T03:23:06.438 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-07-18T03:23:06.438 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-07-18T03:23:06.438 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-07-18T03:23:06.438 INFO:teuthology.orchestra.run.smithi018.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-07-18T03:23:06.438 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-07-18T03:23:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-07-18T03:23:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-07-18T03:23:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-07-18T03:23:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-07-18T03:23:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-07-18T03:23:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-07-18T03:23:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-07-18T03:23:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-07-18T03:23:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-07-18T03:23:06.441 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-07-18T03:23:06.441 INFO:teuthology.orchestra.run.smithi018.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-07-18T03:23:06.441 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-07-18T03:23:06.441 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-07-18T03:23:06.441 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-07-18T03:23:06.442 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-07-18T03:23:06.442 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-07-18T03:23:06.442 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-07-18T03:23:06.443 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-07-18T03:23:06.443 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-07-18T03:23:06.443 INFO:teuthology.orchestra.run.smithi018.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-07-18T03:23:06.443 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-07-18T03:23:06.444 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-07-18T03:23:06.444 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch3.4.2-12.el8 epel 73 k 2020-07-18T03:23:06.444 INFO:teuthology.orchestra.run.smithi018.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-07-18T03:23:06.444 INFO:teuthology.orchestra.run.smithi018.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-07-18T03:23:06.444 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-07-18T03:23:06.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-trustme noarch0.5.2-4.el8 epel 26 k 2020-07-18T03:23:06.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-07-18T03:23:06.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-07-18T03:23:06.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-07-18T03:23:06.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-07-18T03:23:06.446 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-07-18T03:23:06.452 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-07-18T03:23:06.453 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2020-07-18T03:23:06.453 INFO:teuthology.orchestra.run.smithi018.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-07-18T03:23:06.453 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-07-18T03:23:06.453 INFO:teuthology.orchestra.run.smithi018.stdout:Downgrading: 2020-07-18T03:23:06.453 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-07-18T03:23:06.454 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-07-18T03:23:06.454 INFO:teuthology.orchestra.run.smithi018.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-07-18T03:23:06.454 INFO:teuthology.orchestra.run.smithi018.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-07-18T03:23:06.454 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-07-18T03:23:06.454 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:23:06.455 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2020-07-18T03:23:06.455 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2020-07-18T03:23:06.455 INFO:teuthology.orchestra.run.smithi018.stdout:Install 126 Packages 2020-07-18T03:23:06.455 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 6 Packages 2020-07-18T03:23:06.455 INFO:teuthology.orchestra.run.smithi018.stdout:Downgrade 5 Packages 2020-07-18T03:23:06.455 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:23:06.457 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2020-07-18T03:23:06.457 INFO:teuthology.orchestra.run.smithi047.stdout: Package ArchVersion Repository Size 2020-07-18T03:23:06.457 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2020-07-18T03:23:06.457 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2020-07-18T03:23:06.458 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_642:16.0.0-3461.g0baf19a.el8 ceph 6.4 k 2020-07-18T03:23:06.458 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_642:16.0.0-3461.g0baf19a.el8 ceph 6.0 M 2020-07-18T03:23:06.458 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_642:16.0.0-3461.g0baf19a.el8 ceph 604 k 2020-07-18T03:23:06.458 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_642:16.0.0-3461.g0baf19a.el8 ceph 202 k 2020-07-18T03:23:06.458 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_642:16.0.0-3461.g0baf19a.el8 ceph 1.5 M 2020-07-18T03:23:06.459 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_642:16.0.0-3461.g0baf19a.el8 ceph 10 M 2020-07-18T03:23:06.459 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_642:16.0.0-3461.g0baf19a.el8 ceph 45 M 2020-07-18T03:23:06.459 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm x86_642:16.0.0-3461.g0baf19a.el8 ceph 48 k 2020-07-18T03:23:06.459 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_642:16.0.0-3461.g0baf19a.el8 ceph 23 k 2020-07-18T03:23:06.459 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_642:16.0.0-3461.g0baf19a.el8 ceph 536 k 2020-07-18T03:23:06.460 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_642:16.0.0-3461.g0baf19a.el8 ceph 131 k 2020-07-18T03:23:06.460 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_642:16.0.0-3461.g0baf19a.el8 ceph 204 k 2020-07-18T03:23:06.460 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_642:16.0.0-3461.g0baf19a.el8 ceph 364 k 2020-07-18T03:23:06.460 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_642:16.0.0-3461.g0baf19a.el8 ceph 334 k 2020-07-18T03:23:06.460 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_642:16.0.0-3461.g0baf19a.el8 ceph 113 k 2020-07-18T03:23:06.461 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_642:16.0.0-3461.g0baf19a.el8 ceph 87 k 2020-07-18T03:23:06.461 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_642:16.0.0-3461.g0baf19a.el8 ceph 2.7 M 2020-07-18T03:23:06.461 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_642:16.0.0-3461.g0baf19a.el8 ceph 161 k 2020-07-18T03:23:06.461 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 54 k 2020-07-18T03:23:06.461 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 3.5 M 2020-07-18T03:23:06.461 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 8.6 M 2020-07-18T03:23:06.462 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 27 k 2020-07-18T03:23:06.462 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-07-18T03:23:06.462 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-07-18T03:23:06.462 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-07-18T03:23:06.462 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-07-18T03:23:06.463 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-07-18T03:23:06.463 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-07-18T03:23:06.463 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-07-18T03:23:06.463 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-07-18T03:23:06.463 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2020-07-18T03:23:06.464 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_642:16.0.0-3461.g0baf19a.el8 ceph 3.7 M 2020-07-18T03:23:06.464 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_642:16.0.0-3461.g0baf19a.el8 ceph 2.0 M 2020-07-18T03:23:06.464 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_641:1.20.0-5.el8_1 CentOS-Base 2.1 M 2020-07-18T03:23:06.464 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_641:1.20.0-5.el8_1 CentOS-Base 1.7 M 2020-07-18T03:23:06.464 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_641:1.20.0-5.el8_1 CentOS-Base 133 k 2020-07-18T03:23:06.465 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_641:1.20.0-5.el8_1 CentOS-Base 308 k 2020-07-18T03:23:06.465 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2020-07-18T03:23:06.465 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_642:16.0.0-3461.g0baf19a.el8 ceph 22 M 2020-07-18T03:23:06.465 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_642:16.0.0-3461.g0baf19a.el8 ceph 2.2 M 2020-07-18T03:23:06.465 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_642:16.0.0-3461.g0baf19a.el8 ceph 4.9 M 2020-07-18T03:23:06.466 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_642:16.0.0-3461.g0baf19a.el8 ceph 18 M 2020-07-18T03:23:06.466 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_642:16.0.0-3461.g0baf19a.el8 ceph 25 k 2020-07-18T03:23:06.466 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_642:16.0.0-3461.g0baf19a.el8 ceph 483 k 2020-07-18T03:23:06.466 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_642:16.0.0-3461.g0baf19a.el8 ceph 3.4 M 2020-07-18T03:23:06.466 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_642:16.0.0-3461.g0baf19a.el8 ceph 40 k 2020-07-18T03:23:06.466 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_642:16.0.0-3461.g0baf19a.el8 ceph 67 k 2020-07-18T03:23:06.467 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 24 k 2020-07-18T03:23:06.467 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 174 k 2020-07-18T03:23:06.467 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch2:16.0.0-3461.g0baf19a.el8 ceph-noarch 8.7 k 2020-07-18T03:23:06.467 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-07-18T03:23:06.467 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-07-18T03:23:06.468 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-07-18T03:23:06.468 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-07-18T03:23:06.468 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-07-18T03:23:06.468 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-07-18T03:23:06.468 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-07-18T03:23:06.469 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-07-18T03:23:06.469 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-07-18T03:23:06.469 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-07-18T03:23:06.469 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-07-18T03:23:06.469 INFO:teuthology.orchestra.run.smithi047.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-07-18T03:23:06.469 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-07-18T03:23:06.470 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-07-18T03:23:06.470 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-07-18T03:23:06.470 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-07-18T03:23:06.470 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-07-18T03:23:06.470 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-07-18T03:23:06.471 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-07-18T03:23:06.471 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-07-18T03:23:06.471 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-07-18T03:23:06.471 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-07-18T03:23:06.471 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-07-18T03:23:06.472 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-07-18T03:23:06.472 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-07-18T03:23:06.472 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-07-18T03:23:06.472 INFO:teuthology.orchestra.run.smithi047.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-07-18T03:23:06.472 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-07-18T03:23:06.473 INFO:teuthology.orchestra.run.smithi047.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-07-18T03:23:06.473 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-07-18T03:23:06.473 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-07-18T03:23:06.473 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-07-18T03:23:06.473 INFO:teuthology.orchestra.run.smithi047.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-07-18T03:23:06.474 INFO:teuthology.orchestra.run.smithi047.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-07-18T03:23:06.474 INFO:teuthology.orchestra.run.smithi047.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-07-18T03:23:06.474 INFO:teuthology.orchestra.run.smithi047.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-07-18T03:23:06.474 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-07-18T03:23:06.474 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-07-18T03:23:06.474 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-07-18T03:23:06.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-07-18T03:23:06.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-07-18T03:23:06.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-07-18T03:23:06.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-07-18T03:23:06.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-07-18T03:23:06.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-07-18T03:23:06.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-07-18T03:23:06.476 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-07-18T03:23:06.476 INFO:teuthology.orchestra.run.smithi047.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-07-18T03:23:06.476 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-07-18T03:23:06.477 INFO:teuthology.orchestra.run.smithi047.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-07-18T03:23:06.477 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-07-18T03:23:06.477 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-07-18T03:23:06.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-07-18T03:23:06.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-07-18T03:23:06.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-07-18T03:23:06.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-07-18T03:23:06.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-07-18T03:23:06.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-07-18T03:23:06.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-07-18T03:23:06.479 INFO:teuthology.orchestra.run.smithi047.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-07-18T03:23:06.479 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-07-18T03:23:06.479 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-07-18T03:23:06.479 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-07-18T03:23:06.479 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-07-18T03:23:06.479 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-07-18T03:23:06.480 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-07-18T03:23:06.480 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-07-18T03:23:06.480 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-07-18T03:23:06.480 INFO:teuthology.orchestra.run.smithi047.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-07-18T03:23:06.480 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-07-18T03:23:06.481 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-07-18T03:23:06.481 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch3.4.2-12.el8 epel 73 k 2020-07-18T03:23:06.481 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-07-18T03:23:06.481 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-07-18T03:23:06.481 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-07-18T03:23:06.482 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme noarch0.5.2-4.el8 epel 26 k 2020-07-18T03:23:06.482 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-07-18T03:23:06.482 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-07-18T03:23:06.482 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-07-18T03:23:06.482 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-07-18T03:23:06.483 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-07-18T03:23:06.483 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-07-18T03:23:06.483 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2020-07-18T03:23:06.483 INFO:teuthology.orchestra.run.smithi047.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-07-18T03:23:06.483 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-07-18T03:23:06.483 INFO:teuthology.orchestra.run.smithi047.stdout:Downgrading: 2020-07-18T03:23:06.484 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-07-18T03:23:06.484 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-07-18T03:23:06.484 INFO:teuthology.orchestra.run.smithi047.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-07-18T03:23:06.484 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-07-18T03:23:06.484 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-07-18T03:23:06.485 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:23:06.485 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2020-07-18T03:23:06.485 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2020-07-18T03:23:06.485 INFO:teuthology.orchestra.run.smithi047.stdout:Install 126 Packages 2020-07-18T03:23:06.485 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 6 Packages 2020-07-18T03:23:06.486 INFO:teuthology.orchestra.run.smithi047.stdout:Downgrade 5 Packages 2020-07-18T03:23:06.486 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:23:06.486 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 195 M 2020-07-18T03:23:06.487 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2020-07-18T03:23:06.487 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 195 M 2020-07-18T03:23:06.487 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2020-07-18T03:23:06.497 INFO:teuthology.orchestra.run.smithi047.stdout:(1/137): libkadm5-1.17-9.el8.x86_64.rpm 3.1 MB/s | 184 kB 00:00 2020-07-18T03:23:06.501 INFO:teuthology.orchestra.run.smithi018.stdout:(1/137): krb5-libs-1.17-9.el8.x86_64.rpm 13 MB/s | 851 kB 00:00 2020-07-18T03:23:06.506 INFO:teuthology.orchestra.run.smithi018.stdout:(2/137): libkadm5-1.17-9.el8.x86_64.rpm 2.6 MB/s | 184 kB 00:00 2020-07-18T03:23:06.531 INFO:teuthology.orchestra.run.smithi018.stdout:(3/137): krb5-workstation-1.17-9.el8.x86_64.rpm 9.9 MB/s | 940 kB 00:00 2020-07-18T03:23:06.533 INFO:teuthology.orchestra.run.smithi047.stdout:(2/137): krb5-workstation-1.17-9.el8.x86_64.rpm 9.7 MB/s | 940 kB 00:00 2020-07-18T03:23:06.533 INFO:teuthology.orchestra.run.smithi018.stdout:(4/137): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 21 MB/s | 686 kB 00:00 2020-07-18T03:23:06.537 INFO:teuthology.orchestra.run.smithi047.stdout:(3/137): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 17 MB/s | 686 kB 00:00 2020-07-18T03:23:06.540 INFO:teuthology.orchestra.run.smithi047.stdout:(4/137): krb5-libs-1.17-9.el8.x86_64.rpm 8.2 MB/s | 851 kB 00:00 2020-07-18T03:23:06.556 INFO:teuthology.orchestra.run.smithi018.stdout:(5/137): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 29 MB/s | 1.5 MB 00:00 2020-07-18T03:23:06.562 INFO:teuthology.orchestra.run.smithi047.stdout:(5/137): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 50 MB/s | 1.5 MB 00:00 2020-07-18T03:23:06.633 INFO:teuthology.orchestra.run.smithi018.stdout:(6/137): ceph-16.0.0-3461.g0baf19a.el8.x86_64.r 63 kB/s | 6.4 kB 00:00 2020-07-18T03:23:06.638 INFO:teuthology.orchestra.run.smithi047.stdout:(6/137): ceph-16.0.0-3461.g0baf19a.el8.x86_64.r 64 kB/s | 6.4 kB 00:00 2020-07-18T03:23:06.794 INFO:teuthology.orchestra.run.smithi018.stdout:(7/137): ceph-fuse-16.0.0-3461.g0baf19a.el8.x86 3.7 MB/s | 604 kB 00:00 2020-07-18T03:23:06.809 INFO:teuthology.orchestra.run.smithi047.stdout:(7/137): ceph-fuse-16.0.0-3461.g0baf19a.el8.x86 3.5 MB/s | 604 kB 00:00 2020-07-18T03:23:06.843 INFO:teuthology.orchestra.run.smithi018.stdout:(8/137): ceph-immutable-object-cache-16.0.0-346 4.1 MB/s | 202 kB 00:00 2020-07-18T03:23:06.858 INFO:teuthology.orchestra.run.smithi047.stdout:(8/137): ceph-immutable-object-cache-16.0.0-346 4.1 MB/s | 202 kB 00:00 2020-07-18T03:23:07.327 INFO:teuthology.orchestra.run.smithi018.stdout:(9/137): ceph-mds-16.0.0-3461.g0baf19a.el8.x86_ 4.5 MB/s | 2.2 MB 00:00 2020-07-18T03:23:07.346 INFO:teuthology.orchestra.run.smithi047.stdout:(9/137): ceph-mds-16.0.0-3461.g0baf19a.el8.x86_ 4.4 MB/s | 2.2 MB 00:00 2020-07-18T03:23:07.730 INFO:teuthology.orchestra.run.smithi047.stdout:(10/137): ceph-mgr-16.0.0-3461.g0baf19a.el8.x86 4.0 MB/s | 1.5 MB 00:00 2020-07-18T03:23:07.759 INFO:teuthology.orchestra.run.smithi018.stdout:(10/137): ceph-mgr-16.0.0-3461.g0baf19a.el8.x86 3.5 MB/s | 1.5 MB 00:00 2020-07-18T03:23:07.782 INFO:teuthology.orchestra.run.smithi018.stdout:(11/137): ceph-base-16.0.0-3461.g0baf19a.el8.x8 4.8 MB/s | 6.0 MB 00:01 2020-07-18T03:23:07.842 INFO:teuthology.orchestra.run.smithi047.stdout:(11/137): ceph-base-16.0.0-3461.g0baf19a.el8.x8 4.6 MB/s | 6.0 MB 00:01 2020-07-18T03:23:08.816 INFO:teuthology.orchestra.run.smithi047.stdout:(12/137): ceph-mon-16.0.0-3461.g0baf19a.el8.x86 4.5 MB/s | 4.9 MB 00:01 2020-07-18T03:23:08.850 INFO:teuthology.orchestra.run.smithi018.stdout:(12/137): ceph-mon-16.0.0-3461.g0baf19a.el8.x86 4.5 MB/s | 4.9 MB 00:01 2020-07-18T03:23:11.098 INFO:teuthology.orchestra.run.smithi018.stdout:(13/137): ceph-radosgw-16.0.0-3461.g0baf19a.el8 4.6 MB/s | 10 MB 00:02 2020-07-18T03:23:11.188 INFO:teuthology.orchestra.run.smithi047.stdout:(13/137): ceph-common-16.0.0-3461.g0baf19a.el8. 4.8 MB/s | 22 MB 00:04 2020-07-18T03:23:11.264 INFO:teuthology.orchestra.run.smithi018.stdout:(14/137): ceph-common-16.0.0-3461.g0baf19a.el8. 4.7 MB/s | 22 MB 00:04 2020-07-18T03:23:11.310 INFO:teuthology.orchestra.run.smithi018.stdout:(15/137): ceph-selinux-16.0.0-3461.g0baf19a.el8 117 kB/s | 25 kB 00:00 2020-07-18T03:23:11.318 INFO:teuthology.orchestra.run.smithi047.stdout:(14/137): ceph-radosgw-16.0.0-3461.g0baf19a.el8 4.2 MB/s | 10 MB 00:02 2020-07-18T03:23:11.319 INFO:teuthology.orchestra.run.smithi047.stdout:(15/137): ceph-selinux-16.0.0-3461.g0baf19a.el8 189 kB/s | 25 kB 00:00 2020-07-18T03:23:11.356 INFO:teuthology.orchestra.run.smithi047.stdout:(16/137): cephadm-16.0.0-3461.g0baf19a.el8.x86_ 1.3 MB/s | 48 kB 00:00 2020-07-18T03:23:11.380 INFO:teuthology.orchestra.run.smithi047.stdout:(17/137): libcephfs-devel-16.0.0-3461.g0baf19a. 1.0 MB/s | 23 kB 00:00 2020-07-18T03:23:11.474 INFO:teuthology.orchestra.run.smithi018.stdout:(16/137): ceph-osd-16.0.0-3461.g0baf19a.el8.x86 4.8 MB/s | 18 MB 00:03 2020-07-18T03:23:11.475 INFO:teuthology.orchestra.run.smithi018.stdout:(17/137): cephadm-16.0.0-3461.g0baf19a.el8.x86_ 292 kB/s | 48 kB 00:00 2020-07-18T03:23:11.500 INFO:teuthology.orchestra.run.smithi047.stdout:(18/137): ceph-osd-16.0.0-3461.g0baf19a.el8.x86 4.9 MB/s | 18 MB 00:03 2020-07-18T03:23:11.501 INFO:teuthology.orchestra.run.smithi018.stdout:(18/137): libcephfs-devel-16.0.0-3461.g0baf19a. 854 kB/s | 23 kB 00:00 2020-07-18T03:23:11.505 INFO:teuthology.orchestra.run.smithi047.stdout:(19/137): libcephfs2-16.0.0-3461.g0baf19a.el8.x 4.2 MB/s | 536 kB 00:00 2020-07-18T03:23:11.547 INFO:teuthology.orchestra.run.smithi047.stdout:(20/137): librados-devel-16.0.0-3461.g0baf19a.e 2.7 MB/s | 131 kB 00:00 2020-07-18T03:23:11.552 INFO:teuthology.orchestra.run.smithi018.stdout:(19/137): librados-devel-16.0.0-3461.g0baf19a.e 2.5 MB/s | 131 kB 00:00 2020-07-18T03:23:11.575 INFO:teuthology.orchestra.run.smithi018.stdout:(20/137): libcephfs2-16.0.0-3461.g0baf19a.el8.x 5.7 MB/s | 536 kB 00:00 2020-07-18T03:23:11.621 INFO:teuthology.orchestra.run.smithi047.stdout:(21/137): libradosstriper1-16.0.0-3461.g0baf19a 4.1 MB/s | 483 kB 00:00 2020-07-18T03:23:11.677 INFO:teuthology.orchestra.run.smithi018.stdout:(21/137): libradosstriper1-16.0.0-3461.g0baf19a 3.8 MB/s | 483 kB 00:00 2020-07-18T03:23:11.679 INFO:teuthology.orchestra.run.smithi047.stdout:(22/137): python3-ceph-argparse-16.0.0-3461.g0b 681 kB/s | 40 kB 00:00 2020-07-18T03:23:11.732 INFO:teuthology.orchestra.run.smithi018.stdout:(22/137): python3-ceph-argparse-16.0.0-3461.g0b 734 kB/s | 40 kB 00:00 2020-07-18T03:23:11.734 INFO:teuthology.orchestra.run.smithi047.stdout:(23/137): python3-ceph-common-16.0.0-3461.g0baf 1.2 MB/s | 67 kB 00:00 2020-07-18T03:23:11.788 INFO:teuthology.orchestra.run.smithi018.stdout:(23/137): python3-ceph-common-16.0.0-3461.g0baf 1.2 MB/s | 67 kB 00:00 2020-07-18T03:23:11.796 INFO:teuthology.orchestra.run.smithi047.stdout:(24/137): python3-cephfs-16.0.0-3461.g0baf19a.e 3.2 MB/s | 204 kB 00:00 2020-07-18T03:23:11.854 INFO:teuthology.orchestra.run.smithi018.stdout:(24/137): python3-cephfs-16.0.0-3461.g0baf19a.e 3.0 MB/s | 204 kB 00:00 2020-07-18T03:23:11.930 INFO:teuthology.orchestra.run.smithi047.stdout:(25/137): python3-rados-16.0.0-3461.g0baf19a.el 2.7 MB/s | 364 kB 00:00 2020-07-18T03:23:11.985 INFO:teuthology.orchestra.run.smithi018.stdout:(25/137): python3-rados-16.0.0-3461.g0baf19a.el 2.8 MB/s | 364 kB 00:00 2020-07-18T03:23:12.058 INFO:teuthology.orchestra.run.smithi047.stdout:(26/137): python3-rbd-16.0.0-3461.g0baf19a.el8. 2.6 MB/s | 334 kB 00:00 2020-07-18T03:23:12.110 INFO:teuthology.orchestra.run.smithi018.stdout:(26/137): python3-rbd-16.0.0-3461.g0baf19a.el8. 2.6 MB/s | 334 kB 00:00 2020-07-18T03:23:12.115 INFO:teuthology.orchestra.run.smithi047.stdout:(27/137): python3-rgw-16.0.0-3461.g0baf19a.el8. 1.9 MB/s | 113 kB 00:00 2020-07-18T03:23:12.127 INFO:teuthology.orchestra.run.smithi018.stdout:(27/137): librgw2-16.0.0-3461.g0baf19a.el8.x86_ 6.0 MB/s | 3.4 MB 00:00 2020-07-18T03:23:12.160 INFO:teuthology.orchestra.run.smithi018.stdout:(28/137): python3-rgw-16.0.0-3461.g0baf19a.el8. 2.3 MB/s | 113 kB 00:00 2020-07-18T03:23:12.163 INFO:teuthology.orchestra.run.smithi047.stdout:(28/137): rbd-fuse-16.0.0-3461.g0baf19a.el8.x86 1.8 MB/s | 87 kB 00:00 2020-07-18T03:23:12.176 INFO:teuthology.orchestra.run.smithi018.stdout:(29/137): rbd-fuse-16.0.0-3461.g0baf19a.el8.x86 1.8 MB/s | 87 kB 00:00 2020-07-18T03:23:12.210 INFO:teuthology.orchestra.run.smithi047.stdout:(29/137): librgw2-16.0.0-3461.g0baf19a.el8.x86_ 5.1 MB/s | 3.4 MB 00:00 2020-07-18T03:23:12.240 INFO:teuthology.orchestra.run.smithi018.stdout:(30/137): rbd-nbd-16.0.0-3461.g0baf19a.el8.x86_ 2.5 MB/s | 161 kB 00:00 2020-07-18T03:23:12.278 INFO:teuthology.orchestra.run.smithi047.stdout:(30/137): rbd-nbd-16.0.0-3461.g0baf19a.el8.x86_ 2.6 MB/s | 161 kB 00:00 2020-07-18T03:23:12.296 INFO:teuthology.orchestra.run.smithi018.stdout:(31/137): ceph-grafana-dashboards-16.0.0-3461.g 425 kB/s | 24 kB 00:00 2020-07-18T03:23:12.323 INFO:teuthology.orchestra.run.smithi047.stdout:(31/137): ceph-grafana-dashboards-16.0.0-3461.g 469 kB/s | 24 kB 00:00 2020-07-18T03:23:12.349 INFO:teuthology.orchestra.run.smithi018.stdout:(32/137): ceph-mgr-cephadm-16.0.0-3461.g0baf19a 1.0 MB/s | 54 kB 00:00 2020-07-18T03:23:12.377 INFO:teuthology.orchestra.run.smithi047.stdout:(32/137): ceph-mgr-cephadm-16.0.0-3461.g0baf19a 1.0 MB/s | 54 kB 00:00 2020-07-18T03:23:12.734 INFO:teuthology.orchestra.run.smithi018.stdout:(33/137): rbd-mirror-16.0.0-3461.g0baf19a.el8.x 4.7 MB/s | 2.7 MB 00:00 2020-07-18T03:23:12.759 INFO:teuthology.orchestra.run.smithi047.stdout:(33/137): rbd-mirror-16.0.0-3461.g0baf19a.el8.x 4.6 MB/s | 2.7 MB 00:00 2020-07-18T03:23:13.014 INFO:teuthology.orchestra.run.smithi018.stdout:(34/137): ceph-mgr-dashboard-16.0.0-3461.g0baf1 5.3 MB/s | 3.5 MB 00:00 2020-07-18T03:23:13.076 INFO:teuthology.orchestra.run.smithi018.stdout:(35/137): ceph-mgr-modules-core-16.0.0-3461.g0b 2.5 MB/s | 174 kB 00:00 2020-07-18T03:23:13.139 INFO:teuthology.orchestra.run.smithi018.stdout:(36/137): ceph-mgr-rook-16.0.0-3461.g0baf19a.el 431 kB/s | 27 kB 00:00 2020-07-18T03:23:13.170 INFO:teuthology.orchestra.run.smithi047.stdout:(34/137): ceph-mgr-dashboard-16.0.0-3461.g0baf1 4.4 MB/s | 3.5 MB 00:00 2020-07-18T03:23:13.197 INFO:teuthology.orchestra.run.smithi018.stdout:(37/137): ceph-prometheus-alerts-16.0.0-3461.g0 151 kB/s | 8.7 kB 00:00 2020-07-18T03:23:13.227 INFO:teuthology.orchestra.run.smithi047.stdout:(35/137): ceph-mgr-modules-core-16.0.0-3461.g0b 3.0 MB/s | 174 kB 00:00 2020-07-18T03:23:13.257 INFO:teuthology.orchestra.run.smithi018.stdout:(38/137): automake-1.16.1-6.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2020-07-18T03:23:13.267 INFO:teuthology.orchestra.run.smithi018.stdout:(39/137): bison-3.0.4-10.el8.x86_64.rpm 74 MB/s | 688 kB 00:00 2020-07-18T03:23:13.271 INFO:teuthology.orchestra.run.smithi018.stdout:(40/137): flex-2.6.1-9.el8.x86_64.rpm 70 MB/s | 320 kB 00:00 2020-07-18T03:23:13.274 INFO:teuthology.orchestra.run.smithi018.stdout:(41/137): jq-1.5-12.el8.x86_64.rpm 62 MB/s | 161 kB 00:00 2020-07-18T03:23:13.279 INFO:teuthology.orchestra.run.smithi047.stdout:(36/137): ceph-mgr-rook-16.0.0-3461.g0baf19a.el 525 kB/s | 27 kB 00:00 2020-07-18T03:23:13.280 INFO:teuthology.orchestra.run.smithi018.stdout:(42/137): librdkafka-0.11.4-1.el8.x86_64.rpm 72 MB/s | 353 kB 00:00 2020-07-18T03:23:13.288 INFO:teuthology.orchestra.run.smithi018.stdout:(43/137): libtool-2.4.6-25.el8.x86_64.rpm 77 MB/s | 709 kB 00:00 2020-07-18T03:23:13.292 INFO:teuthology.orchestra.run.smithi018.stdout:(44/137): lttng-ust-2.8.1-11.el8.x86_64.rpm 67 MB/s | 259 kB 00:00 2020-07-18T03:23:13.295 INFO:teuthology.orchestra.run.smithi018.stdout:(45/137): oniguruma-6.8.2-1.el8.x86_64.rpm 63 MB/s | 188 kB 00:00 2020-07-18T03:23:13.334 INFO:teuthology.orchestra.run.smithi047.stdout:(37/137): ceph-prometheus-alerts-16.0.0-3461.g0 171 kB/s | 8.7 kB 00:00 2020-07-18T03:23:13.347 INFO:teuthology.orchestra.run.smithi018.stdout:(46/137): openblas-0.3.3-2.el8.x86_64.rpm 83 MB/s | 4.3 MB 00:00 2020-07-18T03:23:13.397 INFO:teuthology.orchestra.run.smithi047.stdout:(38/137): automake-1.16.1-6.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2020-07-18T03:23:13.402 INFO:teuthology.orchestra.run.smithi018.stdout:(47/137): openblas-threads-0.3.3-2.el8.x86_64.r 80 MB/s | 4.4 MB 00:00 2020-07-18T03:23:13.405 INFO:teuthology.orchestra.run.smithi047.stdout:(39/137): bison-3.0.4-10.el8.x86_64.rpm 77 MB/s | 688 kB 00:00 2020-07-18T03:23:13.425 INFO:teuthology.orchestra.run.smithi047.stdout:(40/137): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2020-07-18T03:23:13.436 INFO:teuthology.orchestra.run.smithi047.stdout:(41/137): jq-1.5-12.el8.x86_64.rpm 14 MB/s | 161 kB 00:00 2020-07-18T03:23:13.456 INFO:teuthology.orchestra.run.smithi047.stdout:(42/137): librdkafka-0.11.4-1.el8.x86_64.rpm 17 MB/s | 353 kB 00:00 2020-07-18T03:23:13.465 INFO:teuthology.orchestra.run.smithi047.stdout:(43/137): libtool-2.4.6-25.el8.x86_64.rpm 77 MB/s | 709 kB 00:00 2020-07-18T03:23:13.466 INFO:teuthology.orchestra.run.smithi018.stdout:(48/137): python3-babel-2.5.1-5.el8.noarch.rpm 75 MB/s | 4.8 MB 00:00 2020-07-18T03:23:13.469 INFO:teuthology.orchestra.run.smithi047.stdout:(44/137): lttng-ust-2.8.1-11.el8.x86_64.rpm 60 MB/s | 259 kB 00:00 2020-07-18T03:23:13.471 INFO:teuthology.orchestra.run.smithi018.stdout:(49/137): python3-html5lib-0.999999999-6.el8.no 50 MB/s | 214 kB 00:00 2020-07-18T03:23:13.472 INFO:teuthology.orchestra.run.smithi047.stdout:(45/137): oniguruma-6.8.2-1.el8.x86_64.rpm 64 MB/s | 188 kB 00:00 2020-07-18T03:23:13.479 INFO:teuthology.orchestra.run.smithi018.stdout:(50/137): python3-jinja2-2.10.1-2.el8_0.noarch. 62 MB/s | 538 kB 00:00 2020-07-18T03:23:13.481 INFO:teuthology.orchestra.run.smithi018.stdout:(51/137): python3-jsonpatch-1.21-2.el8.noarch.r 14 MB/s | 27 kB 00:00 2020-07-18T03:23:13.484 INFO:teuthology.orchestra.run.smithi018.stdout:(52/137): python3-jsonpointer-1.10-11.el8.noarc 8.9 MB/s | 20 kB 00:00 2020-07-18T03:23:13.507 INFO:teuthology.orchestra.run.smithi018.stdout:(53/137): python3-lxml-4.2.3-1.el8.x86_64.rpm 65 MB/s | 1.5 MB 00:00 2020-07-18T03:23:13.512 INFO:teuthology.orchestra.run.smithi018.stdout:(54/137): python3-mako-1.0.6-13.el8.noarch.rpm 32 MB/s | 157 kB 00:00 2020-07-18T03:23:13.515 INFO:teuthology.orchestra.run.smithi018.stdout:(55/137): python3-markupsafe-0.23-19.el8.x86_64 14 MB/s | 39 kB 00:00 2020-07-18T03:23:13.574 INFO:teuthology.orchestra.run.smithi018.stdout:(56/137): python3-numpy-1.14.3-9.el8.x86_64.rpm 64 MB/s | 3.7 MB 00:00 2020-07-18T03:23:13.578 INFO:teuthology.orchestra.run.smithi047.stdout:(46/137): openblas-0.3.3-2.el8.x86_64.rpm 41 MB/s | 4.3 MB 00:00 2020-07-18T03:23:13.578 INFO:teuthology.orchestra.run.smithi018.stdout:(57/137): python3-numpy-f2py-1.14.3-9.el8.x86_6 60 MB/s | 225 kB 00:00 2020-07-18T03:23:13.580 INFO:teuthology.orchestra.run.smithi018.stdout:(58/137): python3-prettytable-0.7.2-14.el8.noar 25 MB/s | 44 kB 00:00 2020-07-18T03:23:13.584 INFO:teuthology.orchestra.run.smithi018.stdout:(59/137): python3-pyasn1-0.3.7-6.el8.noarch.rpm 26 MB/s | 126 kB 00:00 2020-07-18T03:23:13.589 INFO:teuthology.orchestra.run.smithi018.stdout:(60/137): python3-pyasn1-modules-0.3.7-6.el8.no 23 MB/s | 110 kB 00:00 2020-07-18T03:23:13.594 INFO:teuthology.orchestra.run.smithi018.stdout:(61/137): python3-pytz-2017.2-9.el8.noarch.rpm 13 MB/s | 54 kB 00:00 2020-07-18T03:23:13.689 INFO:teuthology.orchestra.run.smithi047.stdout:(47/137): openblas-threads-0.3.3-2.el8.x86_64.r 39 MB/s | 4.4 MB 00:00 2020-07-18T03:23:13.814 INFO:teuthology.orchestra.run.smithi047.stdout:(48/137): python3-babel-2.5.1-5.el8.noarch.rpm 38 MB/s | 4.8 MB 00:00 2020-07-18T03:23:13.828 INFO:teuthology.orchestra.run.smithi047.stdout:(49/137): python3-html5lib-0.999999999-6.el8.no 58 MB/s | 214 kB 00:00 2020-07-18T03:23:13.829 INFO:teuthology.orchestra.run.smithi047.stdout:(50/137): python3-jinja2-2.10.1-2.el8_0.noarch. 74 MB/s | 538 kB 00:00 2020-07-18T03:23:13.829 INFO:teuthology.orchestra.run.smithi047.stdout:(51/137): python3-jsonpatch-1.21-2.el8.noarch.r 21 MB/s | 27 kB 00:00 2020-07-18T03:23:13.830 INFO:teuthology.orchestra.run.smithi047.stdout:(52/137): python3-jsonpointer-1.10-11.el8.noarc 19 MB/s | 20 kB 00:00 2020-07-18T03:23:13.847 INFO:teuthology.orchestra.run.smithi047.stdout:(53/137): python3-lxml-4.2.3-1.el8.x86_64.rpm 81 MB/s | 1.5 MB 00:00 2020-07-18T03:23:13.849 INFO:teuthology.orchestra.run.smithi047.stdout:(54/137): python3-mako-1.0.6-13.el8.noarch.rpm 60 MB/s | 157 kB 00:00 2020-07-18T03:23:13.850 INFO:teuthology.orchestra.run.smithi047.stdout:(55/137): python3-markupsafe-0.23-19.el8.x86_64 33 MB/s | 39 kB 00:00 2020-07-18T03:23:13.885 INFO:teuthology.orchestra.run.smithi018.stdout:(62/137): python3-scipy-1.0.0-20.module_el8.1.0 50 MB/s | 15 MB 00:00 2020-07-18T03:23:13.888 INFO:teuthology.orchestra.run.smithi018.stdout:(63/137): python3-webencodings-0.5.1-6.el8.noar 8.5 MB/s | 27 kB 00:00 2020-07-18T03:23:13.894 INFO:teuthology.orchestra.run.smithi018.stdout:(64/137): python3-werkzeug-0.12.2-4.el8.noarch. 75 MB/s | 457 kB 00:00 2020-07-18T03:23:13.897 INFO:teuthology.orchestra.run.smithi047.stdout:(56/137): python3-numpy-1.14.3-9.el8.x86_64.rpm 81 MB/s | 3.7 MB 00:00 2020-07-18T03:23:13.898 INFO:teuthology.orchestra.run.smithi018.stdout:(65/137): socat-1.7.3.2-6.el8.x86_64.rpm 71 MB/s | 298 kB 00:00 2020-07-18T03:23:13.900 INFO:teuthology.orchestra.run.smithi018.stdout:(66/137): elfutils-libelf-devel-0.176-5.el8.x86 27 MB/s | 54 kB 00:00 2020-07-18T03:23:13.901 INFO:teuthology.orchestra.run.smithi047.stdout:(57/137): python3-numpy-f2py-1.14.3-9.el8.x86_6 63 MB/s | 225 kB 00:00 2020-07-18T03:23:13.902 INFO:teuthology.orchestra.run.smithi018.stdout:(67/137): fuse-2.9.7-12.el8.x86_64.rpm 45 MB/s | 82 kB 00:00 2020-07-18T03:23:13.903 INFO:teuthology.orchestra.run.smithi047.stdout:(58/137): python3-prettytable-0.7.2-14.el8.noar 21 MB/s | 44 kB 00:00 2020-07-18T03:23:13.903 INFO:teuthology.orchestra.run.smithi018.stdout:(68/137): fuse-common-3.2.1-12.el8.x86_64.rpm 21 MB/s | 21 kB 00:00 2020-07-18T03:23:13.905 INFO:teuthology.orchestra.run.smithi047.stdout:(59/137): python3-pyasn1-0.3.7-6.el8.noarch.rpm 46 MB/s | 126 kB 00:00 2020-07-18T03:23:13.906 INFO:teuthology.orchestra.run.smithi018.stdout:(69/137): keyutils-libs-devel-1.5.10-6.el8.x86_ 24 MB/s | 48 kB 00:00 2020-07-18T03:23:13.908 INFO:teuthology.orchestra.run.smithi047.stdout:(60/137): python3-pyasn1-modules-0.3.7-6.el8.no 44 MB/s | 110 kB 00:00 2020-07-18T03:23:13.912 INFO:teuthology.orchestra.run.smithi047.stdout:(61/137): python3-pytz-2017.2-9.el8.noarch.rpm 14 MB/s | 54 kB 00:00 2020-07-18T03:23:13.914 INFO:teuthology.orchestra.run.smithi018.stdout:(70/137): krb5-devel-1.17-9.el8.x86_64.rpm 65 MB/s | 548 kB 00:00 2020-07-18T03:23:13.918 INFO:teuthology.orchestra.run.smithi018.stdout:(71/137): libaio-devel-0.3.112-1.el8.x86_64.rpm 4.3 MB/s | 19 kB 00:00 2020-07-18T03:23:13.926 INFO:teuthology.orchestra.run.smithi018.stdout:(72/137): libcom_err-devel-1.44.6-3.el8.x86_64. 4.7 MB/s | 38 kB 00:00 2020-07-18T03:23:13.944 INFO:teuthology.orchestra.run.smithi018.stdout:(73/137): libgfortran-8.3.1-4.5.el8.x86_64.rpm 36 MB/s | 638 kB 00:00 2020-07-18T03:23:13.955 INFO:teuthology.orchestra.run.smithi018.stdout:(74/137): libquadmath-8.3.1-4.5.el8.x86_64.rpm 15 MB/s | 169 kB 00:00 2020-07-18T03:23:13.964 INFO:teuthology.orchestra.run.smithi018.stdout:(75/137): librabbitmq-0.9.0-1.el8.x86_64.rpm 5.3 MB/s | 47 kB 00:00 2020-07-18T03:23:13.976 INFO:teuthology.orchestra.run.smithi018.stdout:(76/137): libselinux-devel-2.9-2.1.el8.x86_64.r 17 MB/s | 199 kB 00:00 2020-07-18T03:23:13.987 INFO:teuthology.orchestra.run.smithi018.stdout:(77/137): libsepol-devel-2.9-1.el8.x86_64.rpm 7.7 MB/s | 86 kB 00:00 2020-07-18T03:23:13.997 INFO:teuthology.orchestra.run.smithi018.stdout:(78/137): libuuid-devel-2.32.1-17.el8.x86_64.rp 8.9 MB/s | 95 kB 00:00 2020-07-18T03:23:14.007 INFO:teuthology.orchestra.run.smithi018.stdout:(79/137): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.9 MB/s | 18 kB 00:00 2020-07-18T03:23:14.021 INFO:teuthology.orchestra.run.smithi018.stdout:(80/137): libxslt-1.1.32-3.el8.x86_64.rpm 17 MB/s | 249 kB 00:00 2020-07-18T03:23:14.075 INFO:teuthology.orchestra.run.smithi018.stdout:(81/137): openssl-devel-1.1.1c-2.el8_1.1.x86_64 43 MB/s | 2.3 MB 00:00 2020-07-18T03:23:14.092 INFO:teuthology.orchestra.run.smithi018.stdout:(82/137): pcre2-devel-10.32-1.el8.x86_64.rpm 35 MB/s | 605 kB 00:00 2020-07-18T03:23:14.095 INFO:teuthology.orchestra.run.smithi018.stdout:(83/137): pcre2-utf16-10.32-1.el8.x86_64.rpm 67 MB/s | 228 kB 00:00 2020-07-18T03:23:14.098 INFO:teuthology.orchestra.run.smithi018.stdout:(84/137): pcre2-utf32-10.32-1.el8.x86_64.rpm 68 MB/s | 220 kB 00:00 2020-07-18T03:23:14.102 INFO:teuthology.orchestra.run.smithi018.stdout:(85/137): python3-chardet-3.0.4-7.el8.noarch.rp 65 MB/s | 195 kB 00:00 2020-07-18T03:23:14.103 INFO:teuthology.orchestra.run.smithi018.stdout:(86/137): python3-jwt-1.6.1-2.el8.noarch.rpm 36 MB/s | 43 kB 00:00 2020-07-18T03:23:14.107 INFO:teuthology.orchestra.run.smithi018.stdout:(87/137): python3-oauthlib-2.1.0-1.el8.noarch.r 62 MB/s | 155 kB 00:00 2020-07-18T03:23:14.107 INFO:teuthology.orchestra.run.smithi018.stdout:(88/137): python3-pysocks-1.6.8-3.el8.noarch.rp 31 MB/s | 34 kB 00:00 2020-07-18T03:23:14.110 INFO:teuthology.orchestra.run.smithi018.stdout:(89/137): python3-requests-2.20.0-2.1.el8_1.noa 48 MB/s | 123 kB 00:00 2020-07-18T03:23:14.111 INFO:teuthology.orchestra.run.smithi018.stdout:(90/137): python3-requests-oauthlib-1.0.0-1.el8 37 MB/s | 43 kB 00:00 2020-07-18T03:23:14.114 INFO:teuthology.orchestra.run.smithi018.stdout:(91/137): python3-urllib3-1.24.2-2.el8.noarch.r 64 MB/s | 176 kB 00:00 2020-07-18T03:23:14.116 INFO:teuthology.orchestra.run.smithi018.stdout:(92/137): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 42 MB/s | 69 kB 00:00 2020-07-18T03:23:14.117 INFO:teuthology.orchestra.run.smithi018.stdout:(93/137): zlib-devel-1.2.11-10.el8.x86_64.rpm 38 MB/s | 56 kB 00:00 2020-07-18T03:23:14.160 INFO:teuthology.orchestra.run.smithi018.stdout:(94/137): ceph-mgr-diskprediction-local-16.0.0- 6.0 MB/s | 8.6 MB 00:01 2020-07-18T03:23:14.195 INFO:teuthology.orchestra.run.smithi018.stdout:(95/137): fmt-6.2.1-1.el8.x86_64.rpm 1.5 MB/s | 116 kB 00:00 2020-07-18T03:23:14.197 INFO:teuthology.orchestra.run.smithi047.stdout:(62/137): python3-scipy-1.0.0-20.module_el8.1.0 51 MB/s | 15 MB 00:00 2020-07-18T03:23:14.199 INFO:teuthology.orchestra.run.smithi047.stdout:(63/137): python3-webencodings-0.5.1-6.el8.noar 20 MB/s | 27 kB 00:00 2020-07-18T03:23:14.226 INFO:teuthology.orchestra.run.smithi018.stdout:(96/137): leveldb-1.22-1.el8.x86_64.rpm 5.7 MB/s | 181 kB 00:00 2020-07-18T03:23:14.227 INFO:teuthology.orchestra.run.smithi047.stdout:(64/137): ceph-mgr-diskprediction-local-16.0.0- 5.9 MB/s | 8.6 MB 00:01 2020-07-18T03:23:14.229 INFO:teuthology.orchestra.run.smithi047.stdout:(65/137): python3-werkzeug-0.12.2-4.el8.noarch. 15 MB/s | 457 kB 00:00 2020-07-18T03:23:14.235 INFO:teuthology.orchestra.run.smithi047.stdout:(66/137): elfutils-libelf-devel-0.176-5.el8.x86 34 MB/s | 54 kB 00:00 2020-07-18T03:23:14.238 INFO:teuthology.orchestra.run.smithi047.stdout:(67/137): fuse-2.9.7-12.el8.x86_64.rpm 42 MB/s | 82 kB 00:00 2020-07-18T03:23:14.239 INFO:teuthology.orchestra.run.smithi047.stdout:(68/137): fuse-common-3.2.1-12.el8.x86_64.rpm 21 MB/s | 21 kB 00:00 2020-07-18T03:23:14.240 INFO:teuthology.orchestra.run.smithi047.stdout:(69/137): keyutils-libs-devel-1.5.10-6.el8.x86_ 30 MB/s | 48 kB 00:00 2020-07-18T03:23:14.242 INFO:teuthology.orchestra.run.smithi018.stdout:(97/137): liboath-2.6.2-3.el8.x86_64.rpm 3.7 MB/s | 59 kB 00:00 2020-07-18T03:23:14.248 INFO:teuthology.orchestra.run.smithi047.stdout:(70/137): krb5-devel-1.17-9.el8.x86_64.rpm 74 MB/s | 548 kB 00:00 2020-07-18T03:23:14.251 INFO:teuthology.orchestra.run.smithi047.stdout:(71/137): libaio-devel-0.3.112-1.el8.x86_64.rpm 7.0 MB/s | 19 kB 00:00 2020-07-18T03:23:14.258 INFO:teuthology.orchestra.run.smithi018.stdout:(98/137): libunwind-1.3.1-3.el8.x86_64.rpm 4.6 MB/s | 75 kB 00:00 2020-07-18T03:23:14.273 INFO:teuthology.orchestra.run.smithi018.stdout:(99/137): gperftools-libs-2.7-6.el8.x86_64.rpm 2.7 MB/s | 315 kB 00:00 2020-07-18T03:23:14.274 INFO:teuthology.orchestra.run.smithi018.stdout:(100/137): python3-apipkg-1.5-6.el8.noarch.rpm 1.3 MB/s | 21 kB 00:00 2020-07-18T03:23:14.278 INFO:teuthology.orchestra.run.smithi047.stdout:(72/137): libcom_err-devel-1.44.6-3.el8.x86_64. 1.4 MB/s | 38 kB 00:00 2020-07-18T03:23:14.287 INFO:teuthology.orchestra.run.smithi047.stdout:(73/137): libgfortran-8.3.1-4.5.el8.x86_64.rpm 73 MB/s | 638 kB 00:00 2020-07-18T03:23:14.290 INFO:teuthology.orchestra.run.smithi047.stdout:(74/137): libquadmath-8.3.1-4.5.el8.x86_64.rpm 53 MB/s | 169 kB 00:00 2020-07-18T03:23:14.291 INFO:teuthology.orchestra.run.smithi018.stdout:(101/137): python3-beautifulsoup4-4.6.3-2.el8.1 11 MB/s | 185 kB 00:00 2020-07-18T03:23:14.292 INFO:teuthology.orchestra.run.smithi047.stdout:(75/137): librabbitmq-0.9.0-1.el8.x86_64.rpm 34 MB/s | 47 kB 00:00 2020-07-18T03:23:14.292 INFO:teuthology.orchestra.run.smithi018.stdout:(102/137): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.2 MB/s | 44 kB 00:00 2020-07-18T03:23:14.293 INFO:teuthology.orchestra.run.smithi047.stdout:(76/137): socat-1.7.3.2-6.el8.x86_64.rpm 4.4 MB/s | 298 kB 00:00 2020-07-18T03:23:14.295 INFO:teuthology.orchestra.run.smithi047.stdout:(77/137): libselinux-devel-2.9-2.1.el8.x86_64.r 63 MB/s | 199 kB 00:00 2020-07-18T03:23:14.296 INFO:teuthology.orchestra.run.smithi047.stdout:(78/137): libsepol-devel-2.9-1.el8.x86_64.rpm 30 MB/s | 86 kB 00:00 2020-07-18T03:23:14.297 INFO:teuthology.orchestra.run.smithi047.stdout:(79/137): libuuid-devel-2.32.1-17.el8.x86_64.rp 50 MB/s | 95 kB 00:00 2020-07-18T03:23:14.298 INFO:teuthology.orchestra.run.smithi047.stdout:(80/137): libverto-devel-0.3.0-5.el8.x86_64.rpm 12 MB/s | 18 kB 00:00 2020-07-18T03:23:14.301 INFO:teuthology.orchestra.run.smithi047.stdout:(81/137): libxslt-1.1.32-3.el8.x86_64.rpm 61 MB/s | 249 kB 00:00 2020-07-18T03:23:14.306 INFO:teuthology.orchestra.run.smithi018.stdout:(103/137): python3-cachetools-3.1.1-4.el8.noarc 2.1 MB/s | 33 kB 00:00 2020-07-18T03:23:14.311 INFO:teuthology.orchestra.run.smithi018.stdout:(104/137): python3-certifi-2018.10.15-7.el8.noa 868 kB/s | 16 kB 00:00 2020-07-18T03:23:14.311 INFO:teuthology.orchestra.run.smithi047.stdout:(82/137): pcre2-devel-10.32-1.el8.x86_64.rpm 60 MB/s | 605 kB 00:00 2020-07-18T03:23:14.315 INFO:teuthology.orchestra.run.smithi047.stdout:(83/137): pcre2-utf16-10.32-1.el8.x86_64.rpm 56 MB/s | 228 kB 00:00 2020-07-18T03:23:14.320 INFO:teuthology.orchestra.run.smithi047.stdout:(84/137): pcre2-utf32-10.32-1.el8.x86_64.rpm 52 MB/s | 220 kB 00:00 2020-07-18T03:23:14.323 INFO:teuthology.orchestra.run.smithi018.stdout:(105/137): python3-cheroot-8.2.1-1.el8.noarch.r 8.6 MB/s | 145 kB 00:00 2020-07-18T03:23:14.324 INFO:teuthology.orchestra.run.smithi047.stdout:(85/137): python3-chardet-3.0.4-7.el8.noarch.rp 50 MB/s | 195 kB 00:00 2020-07-18T03:23:14.326 INFO:teuthology.orchestra.run.smithi047.stdout:(86/137): python3-jwt-1.6.1-2.el8.noarch.rpm 22 MB/s | 43 kB 00:00 2020-07-18T03:23:14.330 INFO:teuthology.orchestra.run.smithi047.stdout:(87/137): python3-oauthlib-2.1.0-1.el8.noarch.r 45 MB/s | 155 kB 00:00 2020-07-18T03:23:14.332 INFO:teuthology.orchestra.run.smithi047.stdout:(88/137): python3-pysocks-1.6.8-3.el8.noarch.rp 21 MB/s | 34 kB 00:00 2020-07-18T03:23:14.335 INFO:teuthology.orchestra.run.smithi047.stdout:(89/137): python3-requests-2.20.0-2.1.el8_1.noa 40 MB/s | 123 kB 00:00 2020-07-18T03:23:14.337 INFO:teuthology.orchestra.run.smithi047.stdout:(90/137): python3-requests-oauthlib-1.0.0-1.el8 24 MB/s | 43 kB 00:00 2020-07-18T03:23:14.339 INFO:teuthology.orchestra.run.smithi018.stdout:(106/137): python3-cssselect-0.9.2-10.el8.noarc 2.5 MB/s | 40 kB 00:00 2020-07-18T03:23:14.340 INFO:teuthology.orchestra.run.smithi047.stdout:(91/137): python3-urllib3-1.24.2-2.el8.noarch.r 46 MB/s | 176 kB 00:00 2020-07-18T03:23:14.344 INFO:teuthology.orchestra.run.smithi047.stdout:(92/137): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 23 MB/s | 69 kB 00:00 2020-07-18T03:23:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:(107/137): python3-cherrypy-18.4.0-1.el8.noarch 9.8 MB/s | 384 kB 00:00 2020-07-18T03:23:14.350 INFO:teuthology.orchestra.run.smithi047.stdout:(93/137): openssl-devel-1.1.1c-2.el8_1.1.x86_64 44 MB/s | 2.3 MB 00:00 2020-07-18T03:23:14.351 INFO:teuthology.orchestra.run.smithi047.stdout:(94/137): zlib-devel-1.2.11-10.el8.x86_64.rpm 7.5 MB/s | 56 kB 00:00 2020-07-18T03:23:14.357 INFO:teuthology.orchestra.run.smithi018.stdout:(108/137): python3-execnet-1.7.1-1.el8.noarch.r 20 MB/s | 355 kB 00:00 2020-07-18T03:23:14.369 INFO:teuthology.orchestra.run.smithi018.stdout:(109/137): python3-google-auth-1.1.1-10.el8.noa 4.0 MB/s | 81 kB 00:00 2020-07-18T03:23:14.372 INFO:teuthology.orchestra.run.smithi018.stdout:(110/137): python3-jaraco-6.2-6.el8.noarch.rpm 710 kB/s | 11 kB 00:00 2020-07-18T03:23:14.388 INFO:teuthology.orchestra.run.smithi018.stdout:(111/137): python3-jaraco-functools-2.0-4.el8.n 930 kB/s | 18 kB 00:00 2020-07-18T03:23:14.408 INFO:teuthology.orchestra.run.smithi018.stdout:(112/137): python3-logutils-0.3.5-11.el8.noarch 2.5 MB/s | 49 kB 00:00 2020-07-18T03:23:14.427 INFO:teuthology.orchestra.run.smithi018.stdout:(113/137): python3-more-itertools-7.2.0-3.el8.n 3.0 MB/s | 59 kB 00:00 2020-07-18T03:23:14.444 INFO:teuthology.orchestra.run.smithi018.stdout:(114/137): python3-kubernetes-11.0.0-2.el8.noar 18 MB/s | 1.2 MB 00:00 2020-07-18T03:23:14.449 INFO:teuthology.orchestra.run.smithi018.stdout:(115/137): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2020-07-18T03:23:14.452 INFO:teuthology.orchestra.run.smithi018.stdout:(116/137): python3-portend-2.6-1.el8.noarch.rpm 997 kB/s | 16 kB 00:00 2020-07-18T03:23:14.466 INFO:teuthology.orchestra.run.smithi047.stdout:(95/137): fmt-6.2.1-1.el8.x86_64.rpm 1.0 MB/s | 116 kB 00:00 2020-07-18T03:23:14.468 INFO:teuthology.orchestra.run.smithi018.stdout:(117/137): python3-repoze-lru-0.7-6.el8.noarch. 2.1 MB/s | 34 kB 00:00 2020-07-18T03:23:14.469 INFO:teuthology.orchestra.run.smithi018.stdout:(118/137): python3-remoto-1.1.4-4.el8.noarch.rp 2.4 MB/s | 48 kB 00:00 2020-07-18T03:23:14.485 INFO:teuthology.orchestra.run.smithi018.stdout:(119/137): python3-routes-2.4.1-12.el8.noarch.r 11 MB/s | 196 kB 00:00 2020-07-18T03:23:14.488 INFO:teuthology.orchestra.run.smithi018.stdout:(120/137): python3-rsa-3.4.2-12.el8.noarch.rpm 3.7 MB/s | 73 kB 00:00 2020-07-18T03:23:14.498 INFO:teuthology.orchestra.run.smithi047.stdout:(96/137): leveldb-1.22-1.el8.x86_64.rpm 5.7 MB/s | 181 kB 00:00 2020-07-18T03:23:14.501 INFO:teuthology.orchestra.run.smithi018.stdout:(121/137): python3-simplegeneric-0.8.1-17.el8.n 1.2 MB/s | 19 kB 00:00 2020-07-18T03:23:14.507 INFO:teuthology.orchestra.run.smithi018.stdout:(122/137): python3-singledispatch-3.4.0.3-18.el 1.2 MB/s | 24 kB 00:00 2020-07-18T03:23:14.514 INFO:teuthology.orchestra.run.smithi047.stdout:(97/137): liboath-2.6.2-3.el8.x86_64.rpm 3.6 MB/s | 59 kB 00:00 2020-07-18T03:23:14.517 INFO:teuthology.orchestra.run.smithi018.stdout:(123/137): python3-tempora-1.14.1-5.el8.noarch. 1.8 MB/s | 29 kB 00:00 2020-07-18T03:23:14.526 INFO:teuthology.orchestra.run.smithi018.stdout:(124/137): python3-trustme-0.5.2-4.el8.noarch.r 1.4 MB/s | 26 kB 00:00 2020-07-18T03:23:14.530 INFO:teuthology.orchestra.run.smithi047.stdout:(98/137): libunwind-1.3.1-3.el8.x86_64.rpm 4.6 MB/s | 75 kB 00:00 2020-07-18T03:23:14.534 INFO:teuthology.orchestra.run.smithi018.stdout:(125/137): python3-waitress-1.2.1-2.el8.1.noarc 14 MB/s | 241 kB 00:00 2020-07-18T03:23:14.546 INFO:teuthology.orchestra.run.smithi047.stdout:(99/137): python3-apipkg-1.5-6.el8.noarch.rpm 1.3 MB/s | 21 kB 00:00 2020-07-18T03:23:14.547 INFO:teuthology.orchestra.run.smithi018.stdout:(126/137): python3-webob-1.8.5-1.el8.1.noarch.r 12 MB/s | 251 kB 00:00 2020-07-18T03:23:14.550 INFO:teuthology.orchestra.run.smithi018.stdout:(127/137): python3-websocket-client-0.56.0-5.el 3.8 MB/s | 61 kB 00:00 2020-07-18T03:23:14.568 INFO:teuthology.orchestra.run.smithi018.stdout:(128/137): python3-zc-lockfile-2.0-2.el8.noarch 1.4 MB/s | 23 kB 00:00 2020-07-18T03:23:14.568 INFO:teuthology.orchestra.run.smithi018.stdout:(129/137): python3-webtest-2.0.33-1.el8.noarch. 4.3 MB/s | 85 kB 00:00 2020-07-18T03:23:14.568 INFO:teuthology.orchestra.run.smithi047.stdout:(100/137): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.7 MB/s | 44 kB 00:00 2020-07-18T03:23:14.578 INFO:teuthology.orchestra.run.smithi047.stdout:(101/137): python3-beautifulsoup4-4.6.3-2.el8.1 11 MB/s | 185 kB 00:00 2020-07-18T03:23:14.582 INFO:teuthology.orchestra.run.smithi018.stdout:(130/137): xmlstarlet-1.6.1-11.el8.x86_64.rpm 4.2 MB/s | 69 kB 00:00 2020-07-18T03:23:14.594 INFO:teuthology.orchestra.run.smithi047.stdout:(102/137): python3-cachetools-3.1.1-4.el8.noarc 2.1 MB/s | 33 kB 00:00 2020-07-18T03:23:14.610 INFO:teuthology.orchestra.run.smithi047.stdout:(103/137): python3-certifi-2018.10.15-7.el8.noa 1.0 MB/s | 16 kB 00:00 2020-07-18T03:23:14.627 INFO:teuthology.orchestra.run.smithi047.stdout:(104/137): python3-cheroot-8.2.1-1.el8.noarch.r 8.6 MB/s | 145 kB 00:00 2020-07-18T03:23:14.659 INFO:teuthology.orchestra.run.smithi047.stdout:(105/137): python3-cherrypy-18.4.0-1.el8.noarch 12 MB/s | 384 kB 00:00 2020-07-18T03:23:14.675 INFO:teuthology.orchestra.run.smithi047.stdout:(106/137): python3-cssselect-0.9.2-10.el8.noarc 2.5 MB/s | 40 kB 00:00 2020-07-18T03:23:14.693 INFO:teuthology.orchestra.run.smithi047.stdout:(107/137): python3-execnet-1.7.1-1.el8.noarch.r 20 MB/s | 355 kB 00:00 2020-07-18T03:23:14.709 INFO:teuthology.orchestra.run.smithi047.stdout:(108/137): python3-google-auth-1.1.1-10.el8.noa 4.9 MB/s | 81 kB 00:00 2020-07-18T03:23:14.725 INFO:teuthology.orchestra.run.smithi047.stdout:(109/137): python3-jaraco-6.2-6.el8.noarch.rpm 705 kB/s | 11 kB 00:00 2020-07-18T03:23:14.735 INFO:teuthology.orchestra.run.smithi047.stdout:(110/137): gperftools-libs-2.7-6.el8.x86_64.rpm 822 kB/s | 315 kB 00:00 2020-07-18T03:23:14.740 INFO:teuthology.orchestra.run.smithi047.stdout:(111/137): python3-jaraco-functools-2.0-4.el8.n 1.1 MB/s | 18 kB 00:00 2020-07-18T03:23:14.758 INFO:teuthology.orchestra.run.smithi047.stdout:(112/137): python3-logutils-0.3.5-11.el8.noarch 3.0 MB/s | 49 kB 00:00 2020-07-18T03:23:14.773 INFO:teuthology.orchestra.run.smithi047.stdout:(113/137): python3-more-itertools-7.2.0-3.el8.n 3.6 MB/s | 59 kB 00:00 2020-07-18T03:23:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:(114/137): python3-pecan-1.3.2-9.el8.noarch.rpm 16 MB/s | 283 kB 00:00 2020-07-18T03:23:14.807 INFO:teuthology.orchestra.run.smithi047.stdout:(115/137): python3-portend-2.6-1.el8.noarch.rpm 987 kB/s | 16 kB 00:00 2020-07-18T03:23:14.812 INFO:teuthology.orchestra.run.smithi047.stdout:(116/137): python3-kubernetes-11.0.0-2.el8.noar 15 MB/s | 1.2 MB 00:00 2020-07-18T03:23:14.822 INFO:teuthology.orchestra.run.smithi047.stdout:(117/137): python3-remoto-1.1.4-4.el8.noarch.rp 3.0 MB/s | 48 kB 00:00 2020-07-18T03:23:14.831 INFO:teuthology.orchestra.run.smithi047.stdout:(118/137): python3-repoze-lru-0.7-6.el8.noarch. 1.7 MB/s | 34 kB 00:00 2020-07-18T03:23:14.840 INFO:teuthology.orchestra.run.smithi047.stdout:(119/137): python3-routes-2.4.1-12.el8.noarch.r 11 MB/s | 196 kB 00:00 2020-07-18T03:23:14.851 INFO:teuthology.orchestra.run.smithi047.stdout:(120/137): python3-rsa-3.4.2-12.el8.noarch.rpm 3.7 MB/s | 73 kB 00:00 2020-07-18T03:23:14.855 INFO:teuthology.orchestra.run.smithi047.stdout:(121/137): python3-simplegeneric-0.8.1-17.el8.n 1.2 MB/s | 19 kB 00:00 2020-07-18T03:23:14.858 INFO:teuthology.orchestra.run.smithi018.stdout:(131/137): librbd1-16.0.0-3461.g0baf19a.el8.x86 7.3 MB/s | 2.0 MB 00:00 2020-07-18T03:23:14.870 INFO:teuthology.orchestra.run.smithi047.stdout:(122/137): python3-singledispatch-3.4.0.3-18.el 1.2 MB/s | 24 kB 00:00 2020-07-18T03:23:14.871 INFO:teuthology.orchestra.run.smithi047.stdout:(123/137): python3-tempora-1.14.1-5.el8.noarch. 1.8 MB/s | 29 kB 00:00 2020-07-18T03:23:14.884 INFO:teuthology.orchestra.run.smithi018.stdout:(132/137): NetworkManager-1.20.0-5.el8_1.x86_64 82 MB/s | 2.1 MB 00:00 2020-07-18T03:23:14.889 INFO:teuthology.orchestra.run.smithi047.stdout:(124/137): python3-waitress-1.2.1-2.el8.1.noarc 13 MB/s | 241 kB 00:00 2020-07-18T03:23:14.890 INFO:teuthology.orchestra.run.smithi047.stdout:(125/137): python3-trustme-0.5.2-4.el8.noarch.r 1.3 MB/s | 26 kB 00:00 2020-07-18T03:23:14.906 INFO:teuthology.orchestra.run.smithi047.stdout:(126/137): python3-webob-1.8.5-1.el8.1.noarch.r 14 MB/s | 251 kB 00:00 2020-07-18T03:23:14.907 INFO:teuthology.orchestra.run.smithi018.stdout:(133/137): NetworkManager-libnm-1.20.0-5.el8_1. 75 MB/s | 1.7 MB 00:00 2020-07-18T03:23:14.909 INFO:teuthology.orchestra.run.smithi047.stdout:(127/137): python3-websocket-client-0.56.0-5.el 3.1 MB/s | 61 kB 00:00 2020-07-18T03:23:14.910 INFO:teuthology.orchestra.run.smithi018.stdout:(134/137): NetworkManager-team-1.20.0-5.el8_1.x 54 MB/s | 133 kB 00:00 2020-07-18T03:23:14.914 INFO:teuthology.orchestra.run.smithi018.stdout:(135/137): NetworkManager-tui-1.20.0-5.el8_1.x8 71 MB/s | 308 kB 00:00 2020-07-18T03:23:14.922 INFO:teuthology.orchestra.run.smithi047.stdout:(128/137): python3-webtest-2.0.33-1.el8.noarch. 5.2 MB/s | 85 kB 00:00 2020-07-18T03:23:14.928 INFO:teuthology.orchestra.run.smithi047.stdout:(129/137): python3-zc-lockfile-2.0-2.el8.noarch 1.2 MB/s | 23 kB 00:00 2020-07-18T03:23:14.938 INFO:teuthology.orchestra.run.smithi047.stdout:(130/137): xmlstarlet-1.6.1-11.el8.x86_64.rpm 4.2 MB/s | 69 kB 00:00 2020-07-18T03:23:15.182 INFO:teuthology.orchestra.run.smithi018.stdout:(136/137): librados2-16.0.0-3461.g0baf19a.el8.x 6.0 MB/s | 3.7 MB 00:00 2020-07-18T03:23:15.403 INFO:teuthology.orchestra.run.smithi047.stdout:(131/137): librbd1-16.0.0-3461.g0baf19a.el8.x86 4.3 MB/s | 2.0 MB 00:00 2020-07-18T03:23:15.430 INFO:teuthology.orchestra.run.smithi047.stdout:(132/137): NetworkManager-1.20.0-5.el8_1.x86_64 81 MB/s | 2.1 MB 00:00 2020-07-18T03:23:15.451 INFO:teuthology.orchestra.run.smithi047.stdout:(133/137): NetworkManager-libnm-1.20.0-5.el8_1. 81 MB/s | 1.7 MB 00:00 2020-07-18T03:23:15.453 INFO:teuthology.orchestra.run.smithi047.stdout:(134/137): NetworkManager-team-1.20.0-5.el8_1.x 49 MB/s | 133 kB 00:00 2020-07-18T03:23:15.466 INFO:teuthology.orchestra.run.smithi047.stdout:(135/137): NetworkManager-tui-1.20.0-5.el8_1.x8 68 MB/s | 308 kB 00:00 2020-07-18T03:23:15.619 INFO:teuthology.orchestra.run.smithi047.stdout:(136/137): librados2-16.0.0-3461.g0baf19a.el8.x 5.4 MB/s | 3.7 MB 00:00 2020-07-18T03:23:16.625 INFO:teuthology.orchestra.run.smithi047.stdout:(137/137): ceph-test-16.0.0-3461.g0baf19a.el8.x 8.4 MB/s | 45 MB 00:05 2020-07-18T03:23:16.630 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2020-07-18T03:23:16.630 INFO:teuthology.orchestra.run.smithi047.stdout:Total 19 MB/s | 195 MB 00:10 2020-07-18T03:23:16.675 INFO:teuthology.orchestra.run.smithi018.stdout:(137/137): ceph-test-16.0.0-3461.g0baf19a.el8.x 8.3 MB/s | 45 MB 00:05 2020-07-18T03:23:16.679 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2020-07-18T03:23:16.679 INFO:teuthology.orchestra.run.smithi018.stdout:Total 19 MB/s | 195 MB 00:10 2020-07-18T03:23:16.706 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2020-07-18T03:23:16.755 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2020-07-18T03:23:16.908 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2020-07-18T03:23:16.909 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2020-07-18T03:23:17.071 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2020-07-18T03:23:17.071 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2020-07-18T03:23:18.470 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2020-07-18T03:23:18.496 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2020-07-18T03:23:18.700 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2020-07-18T03:23:18.701 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2020-07-18T03:23:23.686 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2020-07-18T03:23:23.917 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/1 2020-07-18T03:23:24.105 INFO:teuthology.orchestra.run.smithi047.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-07-18T03:23:24.177 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-07-18T03:23:24.264 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2020-07-18T03:23:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/148 2020-07-18T03:23:24.464 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/1 2020-07-18T03:23:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/148 2020-07-18T03:23:24.693 INFO:teuthology.orchestra.run.smithi018.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-07-18T03:23:24.743 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/148 2020-07-18T03:23:24.786 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-07-18T03:23:24.891 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/148 2020-07-18T03:23:24.934 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/148 2020-07-18T03:23:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/148 2020-07-18T03:23:25.096 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/148 2020-07-18T03:23:25.188 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/148 2020-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/148 2020-07-18T03:23:25.358 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-07-18T03:23:25.500 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/148 2020-07-18T03:23:25.572 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/148 2020-07-18T03:23:25.789 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-07-18T03:23:25.803 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/148 2020-07-18T03:23:25.983 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 8/148 2020-07-18T03:23:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-07-18T03:23:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 8/148 2020-07-18T03:23:26.444 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-07-18T03:23:26.491 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 9/148 2020-07-18T03:23:26.612 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 9/148 2020-07-18T03:23:26.619 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 8/148 2020-07-18T03:23:26.816 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 10/148 2020-07-18T03:23:26.903 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 8/148 2020-07-18T03:23:26.918 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 10/148 2020-07-18T03:23:27.152 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 9/148 2020-07-18T03:23:27.245 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 9/148 2020-07-18T03:23:27.419 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 10/148 2020-07-18T03:23:27.425 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:16.0.0-3461.g0baf19a.el8.x86_64 11/148 2020-07-18T03:23:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 10/148 2020-07-18T03:23:27.591 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 12/148 2020-07-18T03:23:27.765 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 12/148 2020-07-18T03:23:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:16.0.0-3461.g0baf19a.el8.x86_64 11/148 2020-07-18T03:23:28.004 INFO:teuthology.orchestra.run.smithi047.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/148 2020-07-18T03:23:28.168 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 12/148 2020-07-18T03:23:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/148 2020-07-18T03:23:28.371 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 12/148 2020-07-18T03:23:28.392 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/148 2020-07-18T03:23:28.587 INFO:teuthology.orchestra.run.smithi047.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/148 2020-07-18T03:23:28.640 INFO:teuthology.orchestra.run.smithi018.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/148 2020-07-18T03:23:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_ 17/148 2020-07-18T03:23:28.860 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/148 2020-07-18T03:23:29.078 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/148 2020-07-18T03:23:29.086 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_ 17/148 2020-07-18T03:23:29.306 INFO:teuthology.orchestra.run.smithi018.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/148 2020-07-18T03:23:29.374 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-07-18T03:23:29.436 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-07-18T03:23:29.443 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_ 17/148 2020-07-18T03:23:29.848 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_ 17/148 2020-07-18T03:23:29.903 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-07-18T03:23:30.152 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-07-18T03:23:30.245 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-07-18T03:23:30.258 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-07-18T03:23:30.383 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-07-18T03:23:30.597 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/148 2020-07-18T03:23:30.651 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-07-18T03:23:30.753 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 21/148 2020-07-18T03:23:30.894 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-07-18T03:23:30.949 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 22/148 2020-07-18T03:23:31.021 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-07-18T03:23:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 23/148 2020-07-18T03:23:31.251 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/148 2020-07-18T03:23:31.324 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-07-18T03:23:31.425 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 21/148 2020-07-18T03:23:31.472 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-07-18T03:23:31.602 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 22/148 2020-07-18T03:23:31.607 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-07-18T03:23:31.824 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 23/148 2020-07-18T03:23:32.001 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-07-18T03:23:32.131 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-07-18T03:23:32.175 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-07-18T03:23:32.315 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.3-2.el8.x86_64 26/148 2020-07-18T03:23:32.317 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-07-18T03:23:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 26/148 2020-07-18T03:23:32.911 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-07-18T03:23:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-07-18T03:23:33.085 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.3-2.el8.x86_64 26/148 2020-07-18T03:23:33.658 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 26/148 2020-07-18T03:23:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-07-18T03:23:33.818 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-07-18T03:23:34.098 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 28/148 2020-07-18T03:23:34.339 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/148 2020-07-18T03:23:34.506 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 30/148 2020-07-18T03:23:34.617 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-07-18T03:23:34.695 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/148 2020-07-18T03:23:34.829 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:16.0.0-3461.g0baf19a.el8 32/148 2020-07-18T03:23:35.048 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:16.0.0-3461.g0baf19a.el8.x86_64 33/148 2020-07-18T03:23:35.497 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 28/148 2020-07-18T03:23:35.734 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/148 2020-07-18T03:23:35.932 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 30/148 2020-07-18T03:23:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/148 2020-07-18T03:23:36.123 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/148 2020-07-18T03:23:36.257 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:16.0.0-3461.g0baf19a.el8 32/148 2020-07-18T03:23:36.389 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 35/148 2020-07-18T03:23:36.497 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:16.0.0-3461.g0baf19a.el8.x86_64 33/148 2020-07-18T03:23:36.683 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 36/148 2020-07-18T03:23:36.873 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 37/148 2020-07-18T03:23:37.503 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/148 2020-07-18T03:23:37.799 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 35/148 2020-07-18T03:23:38.202 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 36/148 2020-07-18T03:23:38.417 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 37/148 2020-07-18T03:23:38.965 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 37/148 2020-07-18T03:23:39.248 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 38/148 2020-07-18T03:23:39.490 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 39/148 2020-07-18T03:23:39.712 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 40/148 2020-07-18T03:23:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/148 2020-07-18T03:23:40.147 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-3.4.2-12.el8.noarch 42/148 2020-07-18T03:23:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:16.0.0-3461.g0baf19a.el8.x86_64 43/148 2020-07-18T03:23:40.499 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 37/148 2020-07-18T03:23:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:16.0.0-3461.g0baf19a.el8.x86_64 44/148 2020-07-18T03:23:40.750 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 45/148 2020-07-18T03:23:40.879 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 38/148 2020-07-18T03:23:40.898 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 46/148 2020-07-18T03:23:41.136 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 39/148 2020-07-18T03:23:41.144 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 47/148 2020-07-18T03:23:41.383 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 48/148 2020-07-18T03:23:41.398 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 40/148 2020-07-18T03:23:41.656 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 49/148 2020-07-18T03:23:41.669 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/148 2020-07-18T03:23:41.796 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 50/148 2020-07-18T03:23:41.885 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-3.4.2-12.el8.noarch 42/148 2020-07-18T03:23:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 51/148 2020-07-18T03:23:42.100 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 52/148 2020-07-18T03:23:42.122 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:16.0.0-3461.g0baf19a.el8.x86_64 43/148 2020-07-18T03:23:42.305 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:16.0.0-3461.g0baf19a.el8.x86_64 44/148 2020-07-18T03:23:42.318 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 53/148 2020-07-18T03:23:42.514 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 45/148 2020-07-18T03:23:42.519 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 54/148 2020-07-18T03:23:42.667 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 46/148 2020-07-18T03:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 55/148 2020-07-18T03:23:42.829 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 56/148 2020-07-18T03:23:42.897 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 47/148 2020-07-18T03:23:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 57/148 2020-07-18T03:23:43.174 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 48/148 2020-07-18T03:23:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 58/148 2020-07-18T03:23:43.370 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 59/148 2020-07-18T03:23:43.551 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 49/148 2020-07-18T03:23:43.635 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-2.6-1.el8.noarch 60/148 2020-07-18T03:23:43.733 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 50/148 2020-07-18T03:23:43.876 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 61/148 2020-07-18T03:23:43.924 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 51/148 2020-07-18T03:23:44.030 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 62/148 2020-07-18T03:23:44.070 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 52/148 2020-07-18T03:23:44.190 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/148 2020-07-18T03:23:44.296 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 53/148 2020-07-18T03:23:44.418 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/148 2020-07-18T03:23:44.586 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 54/148 2020-07-18T03:23:44.600 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/148 2020-07-18T03:23:44.761 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 55/148 2020-07-18T03:23:44.805 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 66/148 2020-07-18T03:23:44.931 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 56/148 2020-07-18T03:23:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 67/148 2020-07-18T03:23:45.119 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 57/148 2020-07-18T03:23:45.289 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 68/148 2020-07-18T03:23:45.320 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 58/148 2020-07-18T03:23:45.497 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 59/148 2020-07-18T03:23:45.510 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 69/148 2020-07-18T03:23:45.679 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 70/148 2020-07-18T03:23:45.735 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-2.6-1.el8.noarch 60/148 2020-07-18T03:23:45.880 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 71/148 2020-07-18T03:23:45.982 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 61/148 2020-07-18T03:23:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 72/148 2020-07-18T03:23:46.132 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 62/148 2020-07-18T03:23:46.334 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/148 2020-07-18T03:23:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 73/148 2020-07-18T03:23:46.570 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/148 2020-07-18T03:23:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 74/148 2020-07-18T03:23:46.777 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/148 2020-07-18T03:23:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 75/148 2020-07-18T03:23:46.990 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 66/148 2020-07-18T03:23:47.230 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 67/148 2020-07-18T03:23:47.458 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 68/148 2020-07-18T03:23:47.684 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 69/148 2020-07-18T03:23:47.892 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 70/148 2020-07-18T03:23:47.905 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/148 2020-07-18T03:23:48.074 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 71/148 2020-07-18T03:23:48.160 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 77/148 2020-07-18T03:23:48.300 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 72/148 2020-07-18T03:23:48.370 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 78/148 2020-07-18T03:23:48.582 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 73/148 2020-07-18T03:23:48.668 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 79/148 2020-07-18T03:23:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 74/148 2020-07-18T03:23:48.865 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 80/148 2020-07-18T03:23:48.987 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 81/148 2020-07-18T03:23:48.990 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 75/148 2020-07-18T03:23:49.170 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 82/148 2020-07-18T03:23:49.445 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 83/148 2020-07-18T03:23:49.684 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 84/148 2020-07-18T03:23:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 85/148 2020-07-18T03:23:50.079 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 86/148 2020-07-18T03:23:50.169 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/148 2020-07-18T03:23:50.257 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 87/148 2020-07-18T03:23:50.258 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 88/148 2020-07-18T03:23:50.258 INFO:teuthology.orchestra.run.smithi047.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-07-18T03:23:50.258 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:23:50.461 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 77/148 2020-07-18T03:23:50.651 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.7-12.el8.x86_64 89/148 2020-07-18T03:23:50.699 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 78/148 2020-07-18T03:23:50.901 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 90/148 2020-07-18T03:23:50.994 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 79/148 2020-07-18T03:23:51.113 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 91/148 2020-07-18T03:23:51.192 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 80/148 2020-07-18T03:23:51.314 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 81/148 2020-07-18T03:23:51.514 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 82/148 2020-07-18T03:23:51.517 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 92/148 2020-07-18T03:23:51.770 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 93/148 2020-07-18T03:23:51.800 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 83/148 2020-07-18T03:23:52.108 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 94/148 2020-07-18T03:23:52.205 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 84/148 2020-07-18T03:23:52.397 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 95/148 2020-07-18T03:23:52.414 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 85/148 2020-07-18T03:23:52.537 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/148 2020-07-18T03:23:52.638 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 86/148 2020-07-18T03:23:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/148 2020-07-18T03:23:52.893 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 87/148 2020-07-18T03:23:52.893 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 88/148 2020-07-18T03:23:52.893 INFO:teuthology.orchestra.run.smithi018.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-07-18T03:23:52.894 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:23:52.898 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/148 2020-07-18T03:23:53.055 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 99/148 2020-07-18T03:23:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 99/148 2020-07-18T03:23:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.7-12.el8.x86_64 89/148 2020-07-18T03:23:53.443 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.5-12.el8.x86_64 100/148 2020-07-18T03:23:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 90/148 2020-07-18T03:23:53.731 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 91/148 2020-07-18T03:23:53.739 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.1-6.el8.noarch 101/148 2020-07-18T03:23:53.862 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-3461.g0baf19a.el 102/148 2020-07-18T03:23:54.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-3461.g0baf19a.e 103/148 2020-07-18T03:23:54.101 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 92/148 2020-07-18T03:23:54.322 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:16.0.0-3461.g0baf19a.el8.x 104/148 2020-07-18T03:23:54.390 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 93/148 2020-07-18T03:23:54.725 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 94/148 2020-07-18T03:23:55.007 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 95/148 2020-07-18T03:23:55.321 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/148 2020-07-18T03:23:55.531 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/148 2020-07-18T03:23:55.711 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/148 2020-07-18T03:23:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 99/148 2020-07-18T03:23:56.070 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 99/148 2020-07-18T03:23:56.384 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : jq-1.5-12.el8.x86_64 100/148 2020-07-18T03:23:56.666 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : automake-1.16.1-6.el8.noarch 101/148 2020-07-18T03:23:56.826 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-3461.g0baf19a.el 102/148 2020-07-18T03:23:57.041 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-3461.g0baf19a.e 103/148 2020-07-18T03:23:57.306 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:16.0.0-3461.g0baf19a.el8.x 104/148 2020-07-18T03:23:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 105/148 2020-07-18T03:23:58.596 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 105/148 2020-07-18T03:23:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 105/148 2020-07-18T03:23:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 106/148 2020-07-18T03:23:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 106/148 2020-07-18T03:23:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 107/148 2020-07-18T03:24:01.520 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 105/148 2020-07-18T03:24:01.789 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 105/148 2020-07-18T03:24:02.553 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 105/148 2020-07-18T03:24:02.896 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 106/148 2020-07-18T03:24:02.990 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 106/148 2020-07-18T03:24:03.146 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 107/148 2020-07-18T03:24:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 107/148 2020-07-18T03:24:17.424 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2020-07-18T03:24:17.425 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2020-07-18T03:24:17.425 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2020-07-18T03:24:17.425 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /run 2020-07-18T03:24:17.425 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2020-07-18T03:24:17.425 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2020-07-18T03:24:17.426 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2020-07-18T03:24:17.426 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2020-07-18T03:24:17.426 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2020-07-18T03:24:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:17.810 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 108/148 2020-07-18T03:24:18.272 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 108/148 2020-07-18T03:24:19.035 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 109/148 2020-07-18T03:24:20.624 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 107/148 2020-07-18T03:24:20.624 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2020-07-18T03:24:20.624 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2020-07-18T03:24:20.625 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /dev 2020-07-18T03:24:20.625 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /run 2020-07-18T03:24:20.625 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2020-07-18T03:24:20.625 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2020-07-18T03:24:20.626 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2020-07-18T03:24:20.626 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2020-07-18T03:24:20.626 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /dev 2020-07-18T03:24:20.626 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:20.645 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 109/148 2020-07-18T03:24:20.852 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 110/148 2020-07-18T03:24:20.876 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 110/148 2020-07-18T03:24:21.018 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 108/148 2020-07-18T03:24:21.483 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 108/148 2020-07-18T03:24:21.718 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 109/148 2020-07-18T03:24:22.314 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 111/148 2020-07-18T03:24:22.480 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 111/148 2020-07-18T03:24:22.684 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noar 112/148 2020-07-18T03:24:23.319 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 109/148 2020-07-18T03:24:23.606 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noar 112/148 2020-07-18T03:24:23.751 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 110/148 2020-07-18T03:24:23.778 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 110/148 2020-07-18T03:24:23.872 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.no 113/148 2020-07-18T03:24:24.980 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.no 113/148 2020-07-18T03:24:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-3461.g0ba 114/148 2020-07-18T03:24:25.251 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-3461.g0ba 114/148 2020-07-18T03:24:25.572 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-3461.g0baf19a.el8 115/148 2020-07-18T03:24:25.724 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 111/148 2020-07-18T03:24:25.755 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 116/148 2020-07-18T03:24:25.784 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 116/148 2020-07-18T03:24:25.917 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 111/148 2020-07-18T03:24:25.975 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 117/148 2020-07-18T03:24:25.989 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 117/148 2020-07-18T03:24:26.109 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noar 112/148 2020-07-18T03:24:27.041 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noar 112/148 2020-07-18T03:24:27.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:16.0.0-3461.g0baf19a.el8.x86_64 118/148 2020-07-18T03:24:27.308 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.no 113/148 2020-07-18T03:24:27.316 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 119/148 2020-07-18T03:24:27.675 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 119/148 2020-07-18T03:24:27.908 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 120/148 2020-07-18T03:24:28.417 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.no 113/148 2020-07-18T03:24:28.567 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-3461.g0ba 114/148 2020-07-18T03:24:28.640 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-3461.g0ba 114/148 2020-07-18T03:24:28.949 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-3461.g0baf19a.el8 115/148 2020-07-18T03:24:29.166 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 116/148 2020-07-18T03:24:29.194 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 116/148 2020-07-18T03:24:29.383 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 117/148 2020-07-18T03:24:29.397 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 117/148 2020-07-18T03:24:30.501 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:16.0.0-3461.g0baf19a.el8.x86_64 118/148 2020-07-18T03:24:30.808 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 119/148 2020-07-18T03:24:31.165 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 119/148 2020-07-18T03:24:31.400 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 120/148 2020-07-18T03:24:32.278 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 120/148 2020-07-18T03:24:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:16.0.0-3461.g0baf19a.el8.x86_64 121/148 2020-07-18T03:24:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/148 2020-07-18T03:24:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/148 2020-07-18T03:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 123/148 2020-07-18T03:24:34.042 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 124/148 2020-07-18T03:24:34.227 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 125/148 2020-07-18T03:24:34.436 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:16.0.0-3461.g0baf19a.el8.x86_6 126/148 2020-07-18T03:24:34.586 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 127/148 2020-07-18T03:24:34.782 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 128/148 2020-07-18T03:24:35.232 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 129/148 2020-07-18T03:24:35.666 INFO:teuthology.orchestra.run.smithi047.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 130/148 2020-07-18T03:24:35.889 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 120/148 2020-07-18T03:24:36.340 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 131/148 2020-07-18T03:24:36.507 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:16.0.0-3461.g0baf19a.el8.x86_64 121/148 2020-07-18T03:24:36.594 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:16.0.0-3461.g0baf19a.el8.x86_64 132/148 2020-07-18T03:24:36.666 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/148 2020-07-18T03:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/148 2020-07-18T03:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-3461.g0baf1 133/148 2020-07-18T03:24:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-3461.g0baf1 133/148 2020-07-18T03:24:37.150 INFO:teuthology.orchestra.run.smithi047.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 134/148 2020-07-18T03:24:37.348 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 135/148 2020-07-18T03:24:37.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.1-9.el8.x86_64 136/148 2020-07-18T03:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 136/148 2020-07-18T03:24:37.659 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 123/148 2020-07-18T03:24:37.823 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.0.4-10.el8.x86_64 137/148 2020-07-18T03:24:37.843 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 137/148 2020-07-18T03:24:37.931 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 124/148 2020-07-18T03:24:38.017 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 138/148 2020-07-18T03:24:38.127 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 125/148 2020-07-18T03:24:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.20.0-3.el8.x86_64 139/148 2020-07-18T03:24:38.295 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 140/148 2020-07-18T03:24:38.318 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:16.0.0-3461.g0baf19a.el8.x86_6 126/148 2020-07-18T03:24:38.471 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 141/148 2020-07-18T03:24:38.477 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 127/148 2020-07-18T03:24:38.700 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 142/148 2020-07-18T03:24:38.701 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-07-18T03:24:38.720 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 128/148 2020-07-18T03:24:38.960 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-07-18T03:24:39.076 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.20.0-3.el8.x86_64 144/148 2020-07-18T03:24:39.095 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-07-18T03:24:39.096 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-07-18T03:24:39.186 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 129/148 2020-07-18T03:24:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-07-18T03:24:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-07-18T03:24:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 130/148 2020-07-18T03:24:39.602 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-07-18T03:24:39.602 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 147/148 2020-07-18T03:24:39.781 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 131/148 2020-07-18T03:24:39.860 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 147/148 2020-07-18T03:24:39.861 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-07-18T03:24:40.036 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:16.0.0-3461.g0baf19a.el8.x86_64 132/148 2020-07-18T03:24:40.249 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-3461.g0baf1 133/148 2020-07-18T03:24:40.366 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-3461.g0baf1 133/148 2020-07-18T03:24:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 134/148 2020-07-18T03:24:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-07-18T03:24:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/148 2020-07-18T03:24:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/148 2020-07-18T03:24:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/148 2020-07-18T03:24:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/148 2020-07-18T03:24:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/148 2020-07-18T03:24:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/148 2020-07-18T03:24:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/148 2020-07-18T03:24:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/148 2020-07-18T03:24:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/148 2020-07-18T03:24:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/148 2020-07-18T03:24:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:16.0.0-3461.g0baf19a.el8.x86_64 11/148 2020-07-18T03:24:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 12/148 2020-07-18T03:24:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 13/148 2020-07-18T03:24:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 14/148 2020-07-18T03:24:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-3461.g0baf1 15/148 2020-07-18T03:24:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 16/148 2020-07-18T03:24:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 17/148 2020-07-18T03:24:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 18/148 2020-07-18T03:24:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 19/148 2020-07-18T03:24:40.800 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 20/148 2020-07-18T03:24:40.800 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 21/148 2020-07-18T03:24:40.800 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:16.0.0-3461.g0baf19a.el8.x86_64 22/148 2020-07-18T03:24:40.800 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 23/148 2020-07-18T03:24:40.801 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:16.0.0-3461.g0baf19a.el8.x86_6 24/148 2020-07-18T03:24:40.801 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 25/148 2020-07-18T03:24:40.801 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 26/148 2020-07-18T03:24:40.801 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_ 27/148 2020-07-18T03:24:40.801 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 28/148 2020-07-18T03:24:40.802 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:16.0.0-3461.g0baf19a.el8 29/148 2020-07-18T03:24:40.802 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:16.0.0-3461.g0baf19a.el8.x 30/148 2020-07-18T03:24:40.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:16.0.0-3461.g0baf19a.el8.x86_64 31/148 2020-07-18T03:24:40.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:16.0.0-3461.g0baf19a.el8.x86_64 32/148 2020-07-18T03:24:40.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:16.0.0-3461.g0baf19a.el8.x86_64 33/148 2020-07-18T03:24:40.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:16.0.0-3461.g0baf19a.el8.x86_64 34/148 2020-07-18T03:24:40.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 35/148 2020-07-18T03:24:40.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 36/148 2020-07-18T03:24:40.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:16.0.0-3461.g0baf19a.el8.x86_64 37/148 2020-07-18T03:24:40.821 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-3461.g0baf19a.e 38/148 2020-07-18T03:24:40.821 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noar 39/148 2020-07-18T03:24:40.821 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.no 40/148 2020-07-18T03:24:40.821 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-3461.g0ba 41/148 2020-07-18T03:24:40.821 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-3461.g0baf19a.el8 42/148 2020-07-18T03:24:40.826 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 43/148 2020-07-18T03:24:40.826 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-3461.g0baf19a.el 44/148 2020-07-18T03:24:40.826 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/148 2020-07-18T03:24:40.826 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/148 2020-07-18T03:24:40.827 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/148 2020-07-18T03:24:40.827 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.5-12.el8.x86_64 48/148 2020-07-18T03:24:40.827 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/148 2020-07-18T03:24:40.827 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/148 2020-07-18T03:24:40.827 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/148 2020-07-18T03:24:40.828 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/148 2020-07-18T03:24:40.828 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/148 2020-07-18T03:24:40.828 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/148 2020-07-18T03:24:40.828 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/148 2020-07-18T03:24:40.829 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/148 2020-07-18T03:24:40.829 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/148 2020-07-18T03:24:40.829 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/148 2020-07-18T03:24:40.829 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/148 2020-07-18T03:24:40.830 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/148 2020-07-18T03:24:40.831 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/148 2020-07-18T03:24:40.831 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/148 2020-07-18T03:24:40.831 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/148 2020-07-18T03:24:40.837 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/148 2020-07-18T03:24:40.837 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/148 2020-07-18T03:24:40.837 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/148 2020-07-18T03:24:40.838 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/148 2020-07-18T03:24:40.838 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/148 2020-07-18T03:24:40.838 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/148 2020-07-18T03:24:40.838 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/148 2020-07-18T03:24:40.839 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/148 2020-07-18T03:24:40.839 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/148 2020-07-18T03:24:40.839 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/148 2020-07-18T03:24:40.839 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/148 2020-07-18T03:24:40.840 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/148 2020-07-18T03:24:40.840 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/148 2020-07-18T03:24:40.840 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/148 2020-07-18T03:24:40.840 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/148 2020-07-18T03:24:40.840 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/148 2020-07-18T03:24:40.841 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/148 2020-07-18T03:24:40.841 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/148 2020-07-18T03:24:40.841 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/148 2020-07-18T03:24:40.841 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/148 2020-07-18T03:24:40.841 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/148 2020-07-18T03:24:40.842 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/148 2020-07-18T03:24:40.842 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/148 2020-07-18T03:24:40.842 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/148 2020-07-18T03:24:40.842 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/148 2020-07-18T03:24:40.843 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/148 2020-07-18T03:24:40.843 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/148 2020-07-18T03:24:40.843 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/148 2020-07-18T03:24:40.843 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/148 2020-07-18T03:24:40.843 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/148 2020-07-18T03:24:40.843 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/148 2020-07-18T03:24:40.844 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/148 2020-07-18T03:24:40.844 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/148 2020-07-18T03:24:40.844 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/148 2020-07-18T03:24:40.844 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/148 2020-07-18T03:24:40.844 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/148 2020-07-18T03:24:40.845 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/148 2020-07-18T03:24:40.845 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/148 2020-07-18T03:24:40.845 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/148 2020-07-18T03:24:40.845 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/148 2020-07-18T03:24:40.845 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/148 2020-07-18T03:24:40.846 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/148 2020-07-18T03:24:40.846 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/148 2020-07-18T03:24:40.846 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/148 2020-07-18T03:24:40.846 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/148 2020-07-18T03:24:40.846 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/148 2020-07-18T03:24:40.847 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/148 2020-07-18T03:24:40.847 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/148 2020-07-18T03:24:40.847 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/148 2020-07-18T03:24:40.847 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/148 2020-07-18T03:24:40.847 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/148 2020-07-18T03:24:40.847 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/148 2020-07-18T03:24:40.848 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/148 2020-07-18T03:24:40.848 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/148 2020-07-18T03:24:40.848 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/148 2020-07-18T03:24:40.848 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/148 2020-07-18T03:24:40.848 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/148 2020-07-18T03:24:40.849 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/148 2020-07-18T03:24:40.850 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/148 2020-07-18T03:24:40.850 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/148 2020-07-18T03:24:40.850 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/148 2020-07-18T03:24:40.851 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/148 2020-07-18T03:24:40.851 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-3.4.2-12.el8.noarch 126/148 2020-07-18T03:24:40.851 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/148 2020-07-18T03:24:40.851 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/148 2020-07-18T03:24:40.852 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/148 2020-07-18T03:24:40.852 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 130/148 2020-07-18T03:24:40.852 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/148 2020-07-18T03:24:40.852 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/148 2020-07-18T03:24:40.853 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/148 2020-07-18T03:24:40.853 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/148 2020-07-18T03:24:40.853 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/148 2020-07-18T03:24:40.854 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/148 2020-07-18T03:24:40.854 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 137/148 2020-07-18T03:24:40.854 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/148 2020-07-18T03:24:40.854 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 139/148 2020-07-18T03:24:40.854 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/148 2020-07-18T03:24:40.855 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.20.0-5.el8_1.x86_64 141/148 2020-07-18T03:24:40.855 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.20.0-3.el8.x86_64 142/148 2020-07-18T03:24:40.855 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 143/148 2020-07-18T03:24:40.855 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 144/148 2020-07-18T03:24:40.856 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 145/148 2020-07-18T03:24:40.856 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.20.0-3.el8.x86_64 146/148 2020-07-18T03:24:40.856 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 147/148 2020-07-18T03:24:40.876 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 135/148 2020-07-18T03:24:41.542 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flex-2.6.1-9.el8.x86_64 136/148 2020-07-18T03:24:41.628 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 136/148 2020-07-18T03:24:41.876 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : bison-3.0.4-10.el8.x86_64 137/148 2020-07-18T03:24:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 137/148 2020-07-18T03:24:42.116 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 138/148 2020-07-18T03:24:42.258 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-tui-1:1.20.0-3.el8.x86_64 139/148 2020-07-18T03:24:42.445 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 140/148 2020-07-18T03:24:42.625 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.20.0-3.el8.x86_64 148/148 2020-07-18T03:24:42.625 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:42.625 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2020-07-18T03:24:42.626 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.626 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.626 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:42.626 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:42.626 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:42.627 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:42.627 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:42.627 INFO:teuthology.orchestra.run.smithi047.stdout:Downgraded: 2020-07-18T03:24:42.627 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-07-18T03:24:42.627 INFO:teuthology.orchestra.run.smithi047.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-07-18T03:24:42.628 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-07-18T03:24:42.628 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:42.628 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2020-07-18T03:24:42.628 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.628 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.629 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.629 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.629 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.629 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.629 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.630 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.630 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.630 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.630 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.630 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.631 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.631 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.631 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.631 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.631 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.632 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.633 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.634 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.634 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.634 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.634 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.0.4-10.el8.x86_64 2020-07-18T03:24:42.634 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.1-9.el8.x86_64 2020-07-18T03:24:42.635 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-25.el8.x86_64 2020-07-18T03:24:42.635 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-07-18T03:24:42.635 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-07-18T03:24:42.635 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-07-18T03:24:42.636 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-07-18T03:24:42.636 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-07-18T03:24:42.636 INFO:teuthology.orchestra.run.smithi047.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-07-18T03:24:42.636 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-07-18T03:24:42.636 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.637 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.638 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:42.638 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.638 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.639 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:42.639 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.1-6.el8.noarch 2020-07-18T03:24:42.639 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.5-12.el8.x86_64 2020-07-18T03:24:42.639 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-07-18T03:24:42.639 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-07-18T03:24:42.640 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-07-18T03:24:42.640 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.3-2.el8.x86_64 2020-07-18T03:24:42.640 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-07-18T03:24:42.640 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.5.1-5.el8.noarch 2020-07-18T03:24:42.640 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-07-18T03:24:42.641 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-07-18T03:24:42.641 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-07-18T03:24:42.641 INFO:teuthology.orchestra.run.smithi047.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-07-18T03:24:42.645 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.0.6-13.el8.noarch 2020-07-18T03:24:42.645 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-07-18T03:24:42.645 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-07-18T03:24:42.645 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-07-18T03:24:42.646 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-07-18T03:24:42.646 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-07-18T03:24:42.646 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-07-18T03:24:42.646 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2017.2-9.el8.noarch 2020-07-18T03:24:42.646 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-07-18T03:24:42.647 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-07-18T03:24:42.647 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-07-18T03:24:42.647 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.3.2-6.el8.x86_64 2020-07-18T03:24:42.647 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.7-12.el8.x86_64 2020-07-18T03:24:42.648 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-07-18T03:24:42.648 INFO:teuthology.orchestra.run.smithi047.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-07-18T03:24:42.648 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-devel-1.17-9.el8.x86_64 2020-07-18T03:24:42.648 INFO:teuthology.orchestra.run.smithi047.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-07-18T03:24:42.648 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-07-18T03:24:42.649 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-07-18T03:24:42.649 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-07-18T03:24:42.649 INFO:teuthology.orchestra.run.smithi047.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-07-18T03:24:42.649 INFO:teuthology.orchestra.run.smithi047.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-07-18T03:24:42.649 INFO:teuthology.orchestra.run.smithi047.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-07-18T03:24:42.650 INFO:teuthology.orchestra.run.smithi047.stdout: libxslt-1.1.32-3.el8.x86_64 2020-07-18T03:24:42.650 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-07-18T03:24:42.650 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-07-18T03:24:42.650 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-07-18T03:24:42.651 INFO:teuthology.orchestra.run.smithi047.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-07-18T03:24:42.651 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-07-18T03:24:42.651 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-07-18T03:24:42.651 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-07-18T03:24:42.652 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-07-18T03:24:42.652 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-07-18T03:24:42.652 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-07-18T03:24:42.652 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-07-18T03:24:42.653 INFO:teuthology.orchestra.run.smithi047.stdout: fmt-6.2.1-1.el8.x86_64 2020-07-18T03:24:42.653 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-07-18T03:24:42.653 INFO:teuthology.orchestra.run.smithi047.stdout: leveldb-1.22-1.el8.x86_64 2020-07-18T03:24:42.654 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.2-3.el8.x86_64 2020-07-18T03:24:42.654 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.3.1-3.el8.x86_64 2020-07-18T03:24:42.654 INFO:teuthology.orchestra.run.smithi047.stdout: python3-apipkg-1.5-6.el8.noarch 2020-07-18T03:24:42.655 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-07-18T03:24:42.655 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-07-18T03:24:42.655 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-07-18T03:24:42.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-07-18T03:24:42.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-07-18T03:24:42.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-07-18T03:24:42.657 INFO:teuthology.orchestra.run.smithi047.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-07-18T03:24:42.657 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-07-18T03:24:42.657 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-6.2-6.el8.noarch 2020-07-18T03:24:42.657 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-07-18T03:24:42.658 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-07-18T03:24:42.658 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-07-18T03:24:42.658 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-07-18T03:24:42.658 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-07-18T03:24:42.659 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-2.6-1.el8.noarch 2020-07-18T03:24:42.659 INFO:teuthology.orchestra.run.smithi047.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-07-18T03:24:42.659 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-07-18T03:24:42.660 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.4.1-12.el8.noarch 2020-07-18T03:24:42.660 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-3.4.2-12.el8.noarch 2020-07-18T03:24:42.660 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-07-18T03:24:42.661 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-07-18T03:24:42.661 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-07-18T03:24:42.661 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-07-18T03:24:42.662 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-07-18T03:24:42.662 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-07-18T03:24:42.662 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-07-18T03:24:42.663 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-07-18T03:24:42.663 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-07-18T03:24:42.663 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-07-18T03:24:42.664 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:42.665 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 141/148 2020-07-18T03:24:42.665 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2020-07-18T03:24:42.773 DEBUG:teuthology.parallel:result is None 2020-07-18T03:24:42.891 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 142/148 2020-07-18T03:24:42.892 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-07-18T03:24:43.134 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-07-18T03:24:43.258 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-team-1:1.20.0-3.el8.x86_64 144/148 2020-07-18T03:24:43.272 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-07-18T03:24:43.272 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-07-18T03:24:43.568 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-07-18T03:24:43.569 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-07-18T03:24:43.818 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-07-18T03:24:43.818 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 147/148 2020-07-18T03:24:44.093 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 147/148 2020-07-18T03:24:44.093 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-07-18T03:24:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-07-18T03:24:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/148 2020-07-18T03:24:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/148 2020-07-18T03:24:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/148 2020-07-18T03:24:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/148 2020-07-18T03:24:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/148 2020-07-18T03:24:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/148 2020-07-18T03:24:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/148 2020-07-18T03:24:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/148 2020-07-18T03:24:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/148 2020-07-18T03:24:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/148 2020-07-18T03:24:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:16.0.0-3461.g0baf19a.el8.x86_64 11/148 2020-07-18T03:24:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 12/148 2020-07-18T03:24:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 13/148 2020-07-18T03:24:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 14/148 2020-07-18T03:24:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-3461.g0baf1 15/148 2020-07-18T03:24:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 16/148 2020-07-18T03:24:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 17/148 2020-07-18T03:24:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 18/148 2020-07-18T03:24:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 19/148 2020-07-18T03:24:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 20/148 2020-07-18T03:24:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 21/148 2020-07-18T03:24:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:16.0.0-3461.g0baf19a.el8.x86_64 22/148 2020-07-18T03:24:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 23/148 2020-07-18T03:24:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:16.0.0-3461.g0baf19a.el8.x86_6 24/148 2020-07-18T03:24:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 25/148 2020-07-18T03:24:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 26/148 2020-07-18T03:24:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_ 27/148 2020-07-18T03:24:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 28/148 2020-07-18T03:24:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:16.0.0-3461.g0baf19a.el8 29/148 2020-07-18T03:24:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:16.0.0-3461.g0baf19a.el8.x 30/148 2020-07-18T03:24:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:16.0.0-3461.g0baf19a.el8.x86_64 31/148 2020-07-18T03:24:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:16.0.0-3461.g0baf19a.el8.x86_64 32/148 2020-07-18T03:24:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:16.0.0-3461.g0baf19a.el8.x86_64 33/148 2020-07-18T03:24:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:16.0.0-3461.g0baf19a.el8.x86_64 34/148 2020-07-18T03:24:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 35/148 2020-07-18T03:24:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 36/148 2020-07-18T03:24:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:16.0.0-3461.g0baf19a.el8.x86_64 37/148 2020-07-18T03:24:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-3461.g0baf19a.e 38/148 2020-07-18T03:24:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noar 39/148 2020-07-18T03:24:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.no 40/148 2020-07-18T03:24:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-3461.g0ba 41/148 2020-07-18T03:24:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-3461.g0baf19a.el8 42/148 2020-07-18T03:24:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 43/148 2020-07-18T03:24:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-3461.g0baf19a.el 44/148 2020-07-18T03:24:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/148 2020-07-18T03:24:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/148 2020-07-18T03:24:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/148 2020-07-18T03:24:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : jq-1.5-12.el8.x86_64 48/148 2020-07-18T03:24:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/148 2020-07-18T03:24:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/148 2020-07-18T03:24:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/148 2020-07-18T03:24:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/148 2020-07-18T03:24:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/148 2020-07-18T03:24:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/148 2020-07-18T03:24:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/148 2020-07-18T03:24:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/148 2020-07-18T03:24:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/148 2020-07-18T03:24:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/148 2020-07-18T03:24:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/148 2020-07-18T03:24:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/148 2020-07-18T03:24:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/148 2020-07-18T03:24:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/148 2020-07-18T03:24:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/148 2020-07-18T03:24:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/148 2020-07-18T03:24:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/148 2020-07-18T03:24:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/148 2020-07-18T03:24:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/148 2020-07-18T03:24:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/148 2020-07-18T03:24:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/148 2020-07-18T03:24:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/148 2020-07-18T03:24:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/148 2020-07-18T03:24:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/148 2020-07-18T03:24:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/148 2020-07-18T03:24:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/148 2020-07-18T03:24:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/148 2020-07-18T03:24:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/148 2020-07-18T03:24:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/148 2020-07-18T03:24:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/148 2020-07-18T03:24:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/148 2020-07-18T03:24:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/148 2020-07-18T03:24:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/148 2020-07-18T03:24:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/148 2020-07-18T03:24:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/148 2020-07-18T03:24:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/148 2020-07-18T03:24:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/148 2020-07-18T03:24:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/148 2020-07-18T03:24:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/148 2020-07-18T03:24:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/148 2020-07-18T03:24:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/148 2020-07-18T03:24:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/148 2020-07-18T03:24:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/148 2020-07-18T03:24:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/148 2020-07-18T03:24:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/148 2020-07-18T03:24:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/148 2020-07-18T03:24:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/148 2020-07-18T03:24:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/148 2020-07-18T03:24:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/148 2020-07-18T03:24:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/148 2020-07-18T03:24:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/148 2020-07-18T03:24:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/148 2020-07-18T03:24:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/148 2020-07-18T03:24:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/148 2020-07-18T03:24:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/148 2020-07-18T03:24:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/148 2020-07-18T03:24:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/148 2020-07-18T03:24:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/148 2020-07-18T03:24:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/148 2020-07-18T03:24:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/148 2020-07-18T03:24:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/148 2020-07-18T03:24:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/148 2020-07-18T03:24:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/148 2020-07-18T03:24:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/148 2020-07-18T03:24:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/148 2020-07-18T03:24:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/148 2020-07-18T03:24:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/148 2020-07-18T03:24:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/148 2020-07-18T03:24:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/148 2020-07-18T03:24:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/148 2020-07-18T03:24:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/148 2020-07-18T03:24:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/148 2020-07-18T03:24:45.036 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/148 2020-07-18T03:24:45.036 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/148 2020-07-18T03:24:45.036 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/148 2020-07-18T03:24:45.036 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/148 2020-07-18T03:24:45.036 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/148 2020-07-18T03:24:45.037 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-3.4.2-12.el8.noarch 126/148 2020-07-18T03:24:45.037 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/148 2020-07-18T03:24:45.037 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/148 2020-07-18T03:24:45.037 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/148 2020-07-18T03:24:45.037 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 130/148 2020-07-18T03:24:45.038 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/148 2020-07-18T03:24:45.038 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/148 2020-07-18T03:24:45.038 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/148 2020-07-18T03:24:45.038 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/148 2020-07-18T03:24:45.038 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/148 2020-07-18T03:24:45.039 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/148 2020-07-18T03:24:45.039 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 137/148 2020-07-18T03:24:45.039 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/148 2020-07-18T03:24:45.039 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 139/148 2020-07-18T03:24:45.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/148 2020-07-18T03:24:45.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.20.0-5.el8_1.x86_64 141/148 2020-07-18T03:24:45.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.20.0-3.el8.x86_64 142/148 2020-07-18T03:24:45.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 143/148 2020-07-18T03:24:45.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 144/148 2020-07-18T03:24:45.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 145/148 2020-07-18T03:24:45.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.20.0-3.el8.x86_64 146/148 2020-07-18T03:24:45.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 147/148 2020-07-18T03:24:46.948 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.20.0-3.el8.x86_64 148/148 2020-07-18T03:24:46.949 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:46.949 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2020-07-18T03:24:46.949 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.949 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.950 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:46.950 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:46.950 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:46.950 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 2020-07-18T03:24:46.950 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:46.951 INFO:teuthology.orchestra.run.smithi018.stdout:Downgraded: 2020-07-18T03:24:46.951 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-07-18T03:24:46.951 INFO:teuthology.orchestra.run.smithi018.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-07-18T03:24:46.951 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-07-18T03:24:46.951 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:46.952 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2020-07-18T03:24:46.952 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.952 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.952 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.952 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.953 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.953 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.953 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.953 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.953 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.954 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.954 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.954 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.955 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.955 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.955 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.956 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.956 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.956 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.956 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.957 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.957 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.957 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.958 INFO:teuthology.orchestra.run.smithi018.stdout: bison-3.0.4-10.el8.x86_64 2020-07-18T03:24:46.958 INFO:teuthology.orchestra.run.smithi018.stdout: flex-2.6.1-9.el8.x86_64 2020-07-18T03:24:46.958 INFO:teuthology.orchestra.run.smithi018.stdout: libtool-2.4.6-25.el8.x86_64 2020-07-18T03:24:46.959 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-07-18T03:24:46.959 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-07-18T03:24:46.959 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-07-18T03:24:46.960 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-07-18T03:24:46.960 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-07-18T03:24:46.960 INFO:teuthology.orchestra.run.smithi018.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-07-18T03:24:46.961 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-07-18T03:24:46.961 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.961 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.962 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.962 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.962 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.963 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.963 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.963 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.963 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:16.0.0-3461.g0baf19a.el8.x86_64 2020-07-18T03:24:46.964 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.964 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.964 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:16.0.0-3461.g0baf19a.el8.noarch 2020-07-18T03:24:46.964 INFO:teuthology.orchestra.run.smithi018.stdout: automake-1.16.1-6.el8.noarch 2020-07-18T03:24:46.964 INFO:teuthology.orchestra.run.smithi018.stdout: jq-1.5-12.el8.x86_64 2020-07-18T03:24:46.965 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-07-18T03:24:46.965 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-07-18T03:24:46.965 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-07-18T03:24:46.965 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.3-2.el8.x86_64 2020-07-18T03:24:46.965 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-07-18T03:24:46.965 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.5.1-5.el8.noarch 2020-07-18T03:24:46.966 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-07-18T03:24:46.966 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-07-18T03:24:46.966 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-07-18T03:24:46.966 INFO:teuthology.orchestra.run.smithi018.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-07-18T03:24:46.966 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.0.6-13.el8.noarch 2020-07-18T03:24:46.967 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-07-18T03:24:46.967 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-07-18T03:24:46.967 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-07-18T03:24:46.967 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-07-18T03:24:46.967 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-07-18T03:24:46.968 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-07-18T03:24:46.968 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2017.2-9.el8.noarch 2020-07-18T03:24:46.971 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-07-18T03:24:46.971 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-07-18T03:24:46.971 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-07-18T03:24:46.971 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.3.2-6.el8.x86_64 2020-07-18T03:24:46.972 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-2.9.7-12.el8.x86_64 2020-07-18T03:24:46.972 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-07-18T03:24:46.972 INFO:teuthology.orchestra.run.smithi018.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-07-18T03:24:46.972 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-devel-1.17-9.el8.x86_64 2020-07-18T03:24:46.972 INFO:teuthology.orchestra.run.smithi018.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-07-18T03:24:46.973 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-07-18T03:24:46.973 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-07-18T03:24:46.973 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-07-18T03:24:46.973 INFO:teuthology.orchestra.run.smithi018.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-07-18T03:24:46.974 INFO:teuthology.orchestra.run.smithi018.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-07-18T03:24:46.974 INFO:teuthology.orchestra.run.smithi018.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-07-18T03:24:46.974 INFO:teuthology.orchestra.run.smithi018.stdout: libxslt-1.1.32-3.el8.x86_64 2020-07-18T03:24:46.974 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-07-18T03:24:46.974 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-07-18T03:24:46.974 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-07-18T03:24:46.975 INFO:teuthology.orchestra.run.smithi018.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-07-18T03:24:46.975 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-07-18T03:24:46.975 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-07-18T03:24:46.975 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-07-18T03:24:46.975 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-07-18T03:24:46.976 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-07-18T03:24:46.976 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-07-18T03:24:46.976 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-07-18T03:24:46.976 INFO:teuthology.orchestra.run.smithi018.stdout: fmt-6.2.1-1.el8.x86_64 2020-07-18T03:24:46.976 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-07-18T03:24:46.977 INFO:teuthology.orchestra.run.smithi018.stdout: leveldb-1.22-1.el8.x86_64 2020-07-18T03:24:46.977 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.2-3.el8.x86_64 2020-07-18T03:24:46.977 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.3.1-3.el8.x86_64 2020-07-18T03:24:46.977 INFO:teuthology.orchestra.run.smithi018.stdout: python3-apipkg-1.5-6.el8.noarch 2020-07-18T03:24:46.977 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-07-18T03:24:46.978 INFO:teuthology.orchestra.run.smithi018.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-07-18T03:24:46.978 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-07-18T03:24:46.978 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-07-18T03:24:46.978 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-07-18T03:24:46.978 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-07-18T03:24:46.979 INFO:teuthology.orchestra.run.smithi018.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-07-18T03:24:46.979 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-07-18T03:24:46.979 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-6.2-6.el8.noarch 2020-07-18T03:24:46.979 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-07-18T03:24:46.979 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-07-18T03:24:46.980 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-07-18T03:24:46.980 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-07-18T03:24:46.980 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-07-18T03:24:46.980 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-2.6-1.el8.noarch 2020-07-18T03:24:46.980 INFO:teuthology.orchestra.run.smithi018.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-07-18T03:24:46.981 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-07-18T03:24:46.981 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.4.1-12.el8.noarch 2020-07-18T03:24:46.981 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-3.4.2-12.el8.noarch 2020-07-18T03:24:46.981 INFO:teuthology.orchestra.run.smithi018.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-07-18T03:24:46.981 INFO:teuthology.orchestra.run.smithi018.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-07-18T03:24:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-07-18T03:24:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-07-18T03:24:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-07-18T03:24:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-07-18T03:24:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-07-18T03:24:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-07-18T03:24:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-07-18T03:24:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-07-18T03:24:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:46.984 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2020-07-18T03:24:47.134 DEBUG:teuthology.parallel:result is None 2020-07-18T03:24:47.135 INFO:teuthology.orchestra.run.smithi047:> sudo lsb_release -is 2020-07-18T03:24:47.196 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS 2020-07-18T03:24:47.197 DEBUG:teuthology.misc:System to be installed: CentOS 2020-07-18T03:24:47.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0baf19aeafe8fa5b354a28b0fb65984f752d8b9d 2020-07-18T03:24:47.395 INFO:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-07-18T03:24:47.419 INFO:teuthology.orchestra.run.smithi047.stdout:16.0.0-3461.g0baf19a.el8 2020-07-18T03:24:47.420 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-3461.g0baf19a.el8 2020-07-18T03:24:47.421 INFO:teuthology.task.install:The correct ceph version 16.0.0-3461.g0baf19a is installed. 2020-07-18T03:24:47.421 INFO:teuthology.orchestra.run.smithi018:> sudo lsb_release -is 2020-07-18T03:24:47.479 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS 2020-07-18T03:24:47.481 DEBUG:teuthology.misc:System to be installed: CentOS 2020-07-18T03:24:47.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0baf19aeafe8fa5b354a28b0fb65984f752d8b9d 2020-07-18T03:24:47.657 INFO:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-07-18T03:24:47.677 INFO:teuthology.orchestra.run.smithi018.stdout:16.0.0-3461.g0baf19a.el8 2020-07-18T03:24:47.678 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-3461.g0baf19a.el8 2020-07-18T03:24:47.678 INFO:teuthology.task.install:The correct ceph version 16.0.0-3461.g0baf19a is installed. 2020-07-18T03:24:47.679 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-07-18T03:24:47.680 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-07-18T03:24:47.720 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-07-18T03:24:47.752 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-07-18T03:24:47.753 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-07-18T03:24:47.796 INFO:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-07-18T03:24:47.870 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-07-18T03:24:47.904 INFO:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-07-18T03:24:47.977 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-07-18T03:24:47.977 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-07-18T03:24:48.023 INFO:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-07-18T03:24:48.097 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-07-18T03:24:48.143 INFO:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-07-18T03:24:48.217 INFO:teuthology.run_tasks:Running task ceph... 2020-07-18T03:24:48.324 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-07-18T03:24:48.325 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 777 /var/log/ceph 2020-07-18T03:24:48.328 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2020-07-18T03:24:48.364 INFO:tasks.ceph:Disabling ceph logrotate... 2020-07-18T03:24:48.365 INFO:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/logrotate.d/ceph 2020-07-18T03:24:48.400 INFO:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2020-07-18T03:24:48.438 INFO:tasks.ceph:Creating extra log directories... 2020-07-18T03:24:48.439 INFO:teuthology.orchestra.run.smithi018:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-07-18T03:24:48.472 INFO:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-07-18T03:24:48.513 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-07-18T03:24:48.514 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-07-18T03:24:48.514 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-07-18T03:24:48.515 INFO:teuthology.orchestra.run.smithi047:> cat > /home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-07-18T03:24:48.598 INFO:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-07-18T03:24:48.732 INFO:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-07-18T03:24:48.766 INFO:teuthology.orchestra.run.smithi018:> cat > /home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-07-18T03:24:48.810 INFO:teuthology.orchestra.run.smithi018:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-07-18T03:24:48.948 INFO:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-07-18T03:24:48.980 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-07-18T03:24:48.980 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': ''}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-07-18T03:24:48.981 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi/5235893', 'branch': 'wip-mds-inode-smartptr', 'description': 'fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_8} tasks/client-recovery}', 'email': None, 'first_in_suite': False, 'job_id': '5235893', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'zyan-2020-07-18_00:50:32-fs-wip-mds-inode-smartptr-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.1', 'overrides': {'admin_socket': {'branch': 'wip-mds-inode-smartptr'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': ''}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps'], 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mds-inode-smartptr', 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d'}}, 'owner': 'scheduled_zyan@teuthology', 'priority': 10, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3']], 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mds-inode-smartptr', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-mds-inode-smartptr/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '0baf19aeafe8fa5b354a28b0fb65984f752d8b9d'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_recovery']}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.8461', 'targets': {'smithi047.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlfLY62OC2NvIEUH5HHA/+xaCu1+D/xCdbiIctxbMCcyVzawjI+g16O57Opk7YYwfM9lWp6oS3eAybmdD3auMlq7UjUOlid4b0ROHRN6pquEyTfqy76YEGLgy2L31o8TS78w78B2DS1tbIL4QvmmGnD4V0985+UUTZRIG7254oMoEqwEqmeBFS0FhjR34+xwj9OENwkVQT/zZKjg4/h3fH38EF59pLwYWWs7ne5aLxs1dzMbr1KtfAtvNxKv75dfBdznh1P5B5HB8fvAeV0xeIUDrbJYCAc/AF9ikGO3znvLeQVhDkhgsTGV5wp+lkFi081c4hmuy4jD2pUkjB7ouU4C/XFIRw72HoUVTn+d+jOUEhcsJsibU0jr7kTxgGbm/vG/k2JQUM93gc0Fmpx3IqttyK/dYwkq0eDQa6Q7ZMPqbBG0YAObZsXTRVwobDL75fIEJEJcQGEKyn9RjrvI9NXSXLGx/6c+iq+2++O0TFJlnWKuDXhH2U1PP5/TbhBlc=', 'smithi018.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDERrVDQBxNSbMkwE3jucLqhwEvV0gUUUnYNWwPnW2daUJ0tuGhQy2hSrPYqs8OjmFSpwTe1ZUNBWdoh8yj6aRtC01hTIDJ0d6HSG7L5pwd43Wo0/7U0SL6bDHhzAp+yslLtn+fCqK73tZbotM/x39cwRIKbCoNgbnmsr+57E7KL13H9/AJ5Anxbrn1TaikhAaOoTyiQ4wty9AyjSEfg5gtCfshn3XthIkgIpr5NCzkiUIi69vw1FJ9jNV8uPs93JQM0sx7SAiLo6qF1DRPauigmOm7my4N2tNb6B0MeeiwsXm6W6euqsYA/Z4dqPFR55+TzmQ4R2pYdWfl6n1TAWdch1tlNPFl74hqkiqebo7LlhvWH3JmQVFqAebtzWXlFrpNUQtuHfN1YCOpOXCqaNOTwjGOEdLWt7s2jn7pnfgppuQ9BVDG/CDZ1Z6bs/3We1ASZaEjM1tK6s4ZEgjzZTGqapOwDoiUOx8RLi+BhhEMKjbavChfpbDXVgHwsbunaKk='}} 2020-07-18T03:24:48.982 INFO:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-07-18T03:24:49.022 INFO:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-07-18T03:24:49.039 INFO:teuthology.orchestra.run.smithi018:> sudo install -d -m0777 -- /var/run/ceph 2020-07-18T03:24:49.077 INFO:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2020-07-18T03:24:49.173 DEBUG:teuthology.orchestra.remote:smithi047:/scratch_devs is 72B 2020-07-18T03:24:49.221 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-07-18T03:24:49.222 INFO:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-07-18T03:24:49.238 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-07-18T03:24:49.238 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.238 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 58809 Links: 1 2020-07-18T03:24:49.239 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.239 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.239 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2020-07-18 03:24:40.382553647 +0000 2020-07-18T03:24:49.239 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2020-07-18 03:22:39.467195444 +0000 2020-07-18T03:24:49.239 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2020-07-18 03:22:39.467195444 +0000 2020-07-18T03:24:49.240 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2020-07-18T03:24:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2020-07-18T03:24:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2020-07-18T03:24:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00012065 s, 4.2 MB/s 2020-07-18T03:24:49.259 INFO:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-07-18T03:24:49.314 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-07-18T03:24:49.314 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.314 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 55206 Links: 1 2020-07-18T03:24:49.315 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.315 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.315 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2020-07-18 03:24:40.382553647 +0000 2020-07-18T03:24:49.315 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2020-07-18 03:22:39.816188399 +0000 2020-07-18T03:24:49.315 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2020-07-18 03:22:39.816188399 +0000 2020-07-18T03:24:49.316 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2020-07-18T03:24:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2020-07-18T03:24:49.332 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2020-07-18T03:24:49.332 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000108564 s, 4.7 MB/s 2020-07-18T03:24:49.334 INFO:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-07-18T03:24:49.393 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-07-18T03:24:49.393 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.393 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 61869 Links: 1 2020-07-18T03:24:49.393 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.394 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.394 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2020-07-18 03:24:40.383553626 +0000 2020-07-18T03:24:49.394 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2020-07-18 03:22:40.166181334 +0000 2020-07-18T03:24:49.394 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2020-07-18 03:22:40.166181334 +0000 2020-07-18T03:24:49.394 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2020-07-18T03:24:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2020-07-18T03:24:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2020-07-18T03:24:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000105029 s, 4.9 MB/s 2020-07-18T03:24:49.413 INFO:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-07-18T03:24:49.469 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-07-18T03:24:49.469 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.470 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 58921 Links: 1 2020-07-18T03:24:49.470 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.470 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.471 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2020-07-18 03:24:40.383553626 +0000 2020-07-18T03:24:49.471 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2020-07-18 03:22:40.544173703 +0000 2020-07-18T03:24:49.471 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2020-07-18 03:22:40.544173703 +0000 2020-07-18T03:24:49.471 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2020-07-18T03:24:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2020-07-18T03:24:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2020-07-18T03:24:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.053e-05 s, 5.7 MB/s 2020-07-18T03:24:49.488 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-07-18T03:24:49.507 DEBUG:teuthology.orchestra.remote:smithi018:/scratch_devs is 72B 2020-07-18T03:24:49.559 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-07-18T03:24:49.560 INFO:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-07-18T03:24:49.574 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-07-18T03:24:49.575 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.575 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 55138 Links: 1 2020-07-18T03:24:49.575 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.576 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.576 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2020-07-18 03:24:44.595714961 +0000 2020-07-18T03:24:49.576 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2020-07-18 03:22:39.463242989 +0000 2020-07-18T03:24:49.576 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2020-07-18 03:22:39.463242989 +0000 2020-07-18T03:24:49.577 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2020-07-18T03:24:49.591 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2020-07-18T03:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2020-07-18T03:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.9876e-05 s, 5.1 MB/s 2020-07-18T03:24:49.594 INFO:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-07-18T03:24:49.649 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-07-18T03:24:49.649 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.650 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 55205 Links: 1 2020-07-18T03:24:49.650 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.650 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.650 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2020-07-18 03:24:44.595714961 +0000 2020-07-18T03:24:49.650 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2020-07-18 03:22:39.824243663 +0000 2020-07-18T03:24:49.651 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2020-07-18 03:22:39.824243663 +0000 2020-07-18T03:24:49.651 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2020-07-18T03:24:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2020-07-18T03:24:49.666 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2020-07-18T03:24:49.666 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.6645e-05 s, 5.3 MB/s 2020-07-18T03:24:49.668 INFO:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-07-18T03:24:49.723 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-07-18T03:24:49.724 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.724 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 57891 Links: 1 2020-07-18T03:24:49.724 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.724 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.725 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2020-07-18 03:24:44.595714961 +0000 2020-07-18T03:24:49.725 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2020-07-18 03:22:40.204244372 +0000 2020-07-18T03:24:49.725 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2020-07-18 03:22:40.204244372 +0000 2020-07-18T03:24:49.725 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2020-07-18T03:24:49.740 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2020-07-18T03:24:49.741 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2020-07-18T03:24:49.741 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000101171 s, 5.1 MB/s 2020-07-18T03:24:49.743 INFO:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-07-18T03:24:49.797 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-07-18T03:24:49.797 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-07-18T03:24:49.798 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 57952 Links: 1 2020-07-18T03:24:49.798 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-07-18T03:24:49.798 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2020-07-18T03:24:49.798 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2020-07-18 03:24:44.596714949 +0000 2020-07-18T03:24:49.798 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2020-07-18 03:22:40.615245139 +0000 2020-07-18T03:24:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2020-07-18 03:22:40.615245139 +0000 2020-07-18T03:24:49.799 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2020-07-18T03:24:49.814 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2020-07-18T03:24:49.814 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2020-07-18T03:24:49.814 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 8.3985e-05 s, 6.1 MB/s 2020-07-18T03:24:49.818 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-07-18T03:24:49.818 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi018.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-07-18T03:24:49.818 INFO:tasks.ceph:Generating config... 2020-07-18T03:24:49.820 INFO:tasks.ceph:[client] client mount timeout = 600 2020-07-18T03:24:49.820 INFO:tasks.ceph:[client] debug client = 20 2020-07-18T03:24:49.821 INFO:tasks.ceph:[client] debug ms = 1 2020-07-18T03:24:49.821 INFO:tasks.ceph:[client] pid file = 2020-07-18T03:24:49.821 INFO:tasks.ceph:[mds] debug mds = 20 2020-07-18T03:24:49.821 INFO:tasks.ceph:[mds] debug ms = 1 2020-07-18T03:24:49.822 INFO:tasks.ceph:[mds] mds bal frag = True 2020-07-18T03:24:49.822 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-07-18T03:24:49.822 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-07-18T03:24:49.823 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-07-18T03:24:49.823 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-07-18T03:24:49.823 INFO:tasks.ceph:[mds] mds debug frag = True 2020-07-18T03:24:49.824 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-07-18T03:24:49.824 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-07-18T03:24:49.824 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-07-18T03:24:49.824 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-07-18T03:24:49.825 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-07-18T03:24:49.825 INFO:tasks.ceph:[mgr] debug ms = 1 2020-07-18T03:24:49.825 INFO:tasks.ceph:[mon] debug mon = 20 2020-07-18T03:24:49.826 INFO:tasks.ceph:[mon] debug ms = 1 2020-07-18T03:24:49.826 INFO:tasks.ceph:[mon] debug paxos = 20 2020-07-18T03:24:49.827 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-07-18T03:24:49.827 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-07-18T03:24:49.827 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-07-18T03:24:49.828 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-07-18T03:24:49.828 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-07-18T03:24:49.828 INFO:tasks.ceph:[osd] debug filestore = 20 2020-07-18T03:24:49.828 INFO:tasks.ceph:[osd] debug journal = 20 2020-07-18T03:24:49.829 INFO:tasks.ceph:[osd] debug ms = 20 2020-07-18T03:24:49.829 INFO:tasks.ceph:[osd] debug osd = 25 2020-07-18T03:24:49.829 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-07-18T03:24:49.830 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-07-18T03:24:49.830 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-07-18T03:24:49.830 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-07-18T03:24:49.830 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-07-18T03:24:49.831 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-07-18T03:24:49.831 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-07-18T03:24:49.831 INFO:tasks.ceph:Setting up mon.a... 2020-07-18T03:24:49.832 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-07-18T03:24:49.905 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2020-07-18T03:24:49.908 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-07-18T03:24:49.956 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-07-18T03:24:49.988 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.18'), ('mon.c', '[v2:172.21.15.18:3301,v1:172.21.15.18:6790]')] 2020-07-18T03:24:49.989 INFO:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.18 --addv c '[v2:172.21.15.18:3301,v1:172.21.15.18:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:50.067 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:50.068 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid e5c9cb72-d58e-4215-ba0e-aa8d854b8526 2020-07-18T03:24:50.068 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2020-07-18T03:24:50.068 INFO:teuthology.orchestra.run.smithi047.stdout:fsid e5c9cb72-d58e-4215-ba0e-aa8d854b8526 2020-07-18T03:24:50.068 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2020-07-18T03:24:50.067588+0000 2020-07-18T03:24:50.068 INFO:teuthology.orchestra.run.smithi047.stdout:created 2020-07-18T03:24:50.067588+0000 2020-07-18T03:24:50.069 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 0 (unknown) 2020-07-18T03:24:50.069 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2020-07-18T03:24:50.069 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.b 2020-07-18T03:24:50.069 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] mon.c 2020-07-18T03:24:50.069 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-07-18T03:24:50.071 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e5c9cb72-d58e-4215-ba0e-aa8d854b8526... 2020-07-18T03:24:50.072 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-07-18T03:24:50.075 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-07-18T03:24:50.137 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2020-07-18T03:24:50.137 INFO:teuthology.orchestra.run.smithi018.stdout: chdir = "" 2020-07-18T03:24:50.137 INFO:teuthology.orchestra.run.smithi018.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-07-18T03:24:50.138 INFO:teuthology.orchestra.run.smithi018.stdout: auth supported = cephx 2020-07-18T03:24:50.138 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.138 INFO:teuthology.orchestra.run.smithi018.stdout: filestore xattr use omap = true 2020-07-18T03:24:50.138 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.138 INFO:teuthology.orchestra.run.smithi018.stdout: mon clock drift allowed = 1.000 2020-07-18T03:24:50.139 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.139 INFO:teuthology.orchestra.run.smithi018.stdout: osd crush chooseleaf type = 0 2020-07-18T03:24:50.139 INFO:teuthology.orchestra.run.smithi018.stdout: auth debug = true 2020-07-18T03:24:50.139 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.140 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on old message = true 2020-07-18T03:24:50.140 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on bug = true 2020-07-18T03:24:50.140 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.140 INFO:teuthology.orchestra.run.smithi018.stdout: mon max pg per osd = 10000 # >= luminous 2020-07-18T03:24:50.140 INFO:teuthology.orchestra.run.smithi018.stdout: mon pg warn max object skew = 0 2020-07-18T03:24:50.141 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.141 INFO:teuthology.orchestra.run.smithi018.stdout: # disable pg_autoscaler by default for new pools 2020-07-18T03:24:50.141 INFO:teuthology.orchestra.run.smithi018.stdout: osd_pool_default_pg_autoscale_mode = off 2020-07-18T03:24:50.141 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.141 INFO:teuthology.orchestra.run.smithi018.stdout: osd pool default size = 2 2020-07-18T03:24:50.142 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.142 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd allow primary affinity = true 2020-07-18T03:24:50.142 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd allow pg remap = true 2020-07-18T03:24:50.142 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on legacy crush tunables = false 2020-07-18T03:24:50.142 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on crush straw calc version zero = false 2020-07-18T03:24:50.143 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on no sortbitwise = false 2020-07-18T03:24:50.143 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on osd down out interval zero = false 2020-07-18T03:24:50.143 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on too few osds = false 2020-07-18T03:24:50.143 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-07-18T03:24:50.143 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_pool_no_redundancy = false 2020-07-18T03:24:50.144 INFO:teuthology.orchestra.run.smithi018.stdout: mon_allow_pool_size_one = true 2020-07-18T03:24:50.144 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.144 INFO:teuthology.orchestra.run.smithi018.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-07-18T03:24:50.144 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.144 INFO:teuthology.orchestra.run.smithi018.stdout: osd default data pool replay window = 5 2020-07-18T03:24:50.145 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.145 INFO:teuthology.orchestra.run.smithi018.stdout: mon allow pool delete = true 2020-07-18T03:24:50.145 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.145 INFO:teuthology.orchestra.run.smithi018.stdout: mon cluster log file level = debug 2020-07-18T03:24:50.145 INFO:teuthology.orchestra.run.smithi018.stdout: debug asserts on shutdown = true 2020-07-18T03:24:50.146 INFO:teuthology.orchestra.run.smithi018.stdout: mon host = "172.21.15.47,172.21.15.18,[v2:172.21.15.18:3301,v1:172.21.15.18:6790]" 2020-07-18T03:24:50.146 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = e5c9cb72-d58e-4215-ba0e-aa8d854b8526 2020-07-18T03:24:50.146 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.146 INFO:teuthology.orchestra.run.smithi018.stdout:[osd] 2020-07-18T03:24:50.146 INFO:teuthology.orchestra.run.smithi018.stdout: osd journal size = 100 2020-07-18T03:24:50.146 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.147 INFO:teuthology.orchestra.run.smithi018.stdout: osd scrub load threshold = 5.0 2020-07-18T03:24:50.147 INFO:teuthology.orchestra.run.smithi018.stdout: osd scrub max interval = 600 2020-07-18T03:24:50.148 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.148 INFO:teuthology.orchestra.run.smithi018.stdout: osd recover clone overlap = true 2020-07-18T03:24:50.148 INFO:teuthology.orchestra.run.smithi018.stdout: osd recovery max chunk = 1048576 2020-07-18T03:24:50.148 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.149 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug shutdown = true 2020-07-18T03:24:50.149 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug op order = true 2020-07-18T03:24:50.149 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug verify stray on activate = true 2020-07-18T03:24:50.149 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.149 INFO:teuthology.orchestra.run.smithi018.stdout: osd open classes on start = true 2020-07-18T03:24:50.150 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug pg log writeout = true 2020-07-18T03:24:50.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.150 INFO:teuthology.orchestra.run.smithi018.stdout: osd deep scrub update digest min age = 30 2020-07-18T03:24:50.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.150 INFO:teuthology.orchestra.run.smithi018.stdout: osd map max advance = 10 2020-07-18T03:24:50.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.151 INFO:teuthology.orchestra.run.smithi018.stdout: journal zero on create = true 2020-07-18T03:24:50.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.151 INFO:teuthology.orchestra.run.smithi018.stdout: filestore ondisk finisher threads = 3 2020-07-18T03:24:50.151 INFO:teuthology.orchestra.run.smithi018.stdout: filestore apply finisher threads = 3 2020-07-18T03:24:50.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.152 INFO:teuthology.orchestra.run.smithi018.stdout: bdev debug aio = true 2020-07-18T03:24:50.152 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug misdirected ops = true 2020-07-18T03:24:50.152 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore block size = 96636764160 2020-07-18T03:24:50.152 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore fsck on mount = True 2020-07-18T03:24:50.152 INFO:teuthology.orchestra.run.smithi018.stdout: debug bluefs = 20 2020-07-18T03:24:50.153 INFO:teuthology.orchestra.run.smithi018.stdout: debug bluestore = 20 2020-07-18T03:24:50.153 INFO:teuthology.orchestra.run.smithi018.stdout: debug filestore = 20 2020-07-18T03:24:50.153 INFO:teuthology.orchestra.run.smithi018.stdout: debug journal = 20 2020-07-18T03:24:50.153 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 20 2020-07-18T03:24:50.154 INFO:teuthology.orchestra.run.smithi018.stdout: debug osd = 25 2020-07-18T03:24:50.154 INFO:teuthology.orchestra.run.smithi018.stdout: debug rocksdb = 10 2020-07-18T03:24:50.154 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd backfillfull_ratio = 0.85 2020-07-18T03:24:50.154 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd full ratio = 0.9 2020-07-18T03:24:50.154 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd nearfull ratio = 0.8 2020-07-18T03:24:50.154 INFO:teuthology.orchestra.run.smithi018.stdout: osd failsafe full ratio = 0.95 2020-07-18T03:24:50.155 INFO:teuthology.orchestra.run.smithi018.stdout: osd objectstore = bluestore 2020-07-18T03:24:50.155 INFO:teuthology.orchestra.run.smithi018.stdout: osd op complaint time = 180 2020-07-18T03:24:50.155 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.155 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr] 2020-07-18T03:24:50.155 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2020-07-18T03:24:50.156 INFO:teuthology.orchestra.run.smithi018.stdout: debug mgr = 20 2020-07-18T03:24:50.156 INFO:teuthology.orchestra.run.smithi018.stdout: debug mon = 20 2020-07-18T03:24:50.156 INFO:teuthology.orchestra.run.smithi018.stdout: debug auth = 20 2020-07-18T03:24:50.156 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min pgs per osd = 4 2020-07-18T03:24:50.156 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min bytes per osd = 10 2020-07-18T03:24:50.156 INFO:teuthology.orchestra.run.smithi018.stdout: mgr/telemetry/nag = false 2020-07-18T03:24:50.157 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.157 INFO:teuthology.orchestra.run.smithi018.stdout:[mon] 2020-07-18T03:24:50.157 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2020-07-18T03:24:50.157 INFO:teuthology.orchestra.run.smithi018.stdout: debug mon = 20 2020-07-18T03:24:50.157 INFO:teuthology.orchestra.run.smithi018.stdout: debug paxos = 20 2020-07-18T03:24:50.158 INFO:teuthology.orchestra.run.smithi018.stdout: debug auth = 20 2020-07-18T03:24:50.158 INFO:teuthology.orchestra.run.smithi018.stdout: mon data avail warn = 5 2020-07-18T03:24:50.158 INFO:teuthology.orchestra.run.smithi018.stdout: mon mgr mkfs grace = 240 2020-07-18T03:24:50.158 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min pgs per osd = 4 2020-07-18T03:24:50.158 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd reporter subtree level = osd 2020-07-18T03:24:50.159 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd prime pg temp = true 2020-07-18T03:24:50.159 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min bytes per osd = 10 2020-07-18T03:24:50.159 INFO:teuthology.orchestra.run.smithi018.stdout: mon op complaint time = 120 2020-07-18T03:24:50.159 INFO:teuthology.orchestra.run.smithi018.stdout: 2020-07-18T03:24:50.159 INFO:teuthology.orchestra.run.smithi018.stdout:[client] 2020-07-18T03:24:50.159 INFO:teuthology.orchestra.run.smithi018.stdout: rgw cache enabled = true 2020-07-18T03:24:50.160 INFO:teuthology.orchestra.run.smithi018.stdout: rgw enable ops log = true 2020-07-18T03:24:50.160 INFO:teuthology.orchestra.run.smithi018.stdout: rgw enable usage log = true 2020-07-18T03:24:50.160 INFO:teuthology.orchestra.run.smithi018.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-07-18T03:24:50.160 INFO:teuthology.orchestra.run.smithi018.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-07-18T03:24:50.160 INFO:teuthology.orchestra.run.smithi018.stdout: client mount timeout = 600 2020-07-18T03:24:50.161 INFO:teuthology.orchestra.run.smithi018.stdout: debug client = 20 2020-07-18T03:24:50.161 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2020-07-18T03:24:50.161 INFO:teuthology.orchestra.run.smithi018.stdout: pid file = "" 2020-07-18T03:24:50.161 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.a] 2020-07-18T03:24:50.161 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.b] 2020-07-18T03:24:50.162 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.c] 2020-07-18T03:24:50.162 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a] 2020-07-18T03:24:50.162 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b] 2020-07-18T03:24:50.162 INFO:teuthology.orchestra.run.smithi018.stdout:[mds] 2020-07-18T03:24:50.162 INFO:teuthology.orchestra.run.smithi018.stdout: debug mds = 20 2020-07-18T03:24:50.162 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2020-07-18T03:24:50.163 INFO:teuthology.orchestra.run.smithi018.stdout: mds bal frag = True 2020-07-18T03:24:50.163 INFO:teuthology.orchestra.run.smithi018.stdout: mds bal fragment size max = 10000 2020-07-18T03:24:50.163 INFO:teuthology.orchestra.run.smithi018.stdout: mds bal merge size = 5 2020-07-18T03:24:50.163 INFO:teuthology.orchestra.run.smithi018.stdout: mds bal split bits = 3 2020-07-18T03:24:50.163 INFO:teuthology.orchestra.run.smithi018.stdout: mds bal split size = 100 2020-07-18T03:24:50.164 INFO:teuthology.orchestra.run.smithi018.stdout: mds debug frag = True 2020-07-18T03:24:50.164 INFO:teuthology.orchestra.run.smithi018.stdout: mds debug scatterstat = True 2020-07-18T03:24:50.164 INFO:teuthology.orchestra.run.smithi018.stdout: mds op complaint time = 180 2020-07-18T03:24:50.164 INFO:teuthology.orchestra.run.smithi018.stdout: mds verify scatter = True 2020-07-18T03:24:50.164 INFO:teuthology.orchestra.run.smithi018.stdout: osd op complaint time = 180 2020-07-18T03:24:50.182 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2020-07-18T03:24:50.182 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2020-07-18T03:24:50.183 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-07-18T03:24:50.183 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2020-07-18T03:24:50.183 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.183 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2020-07-18T03:24:50.183 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.184 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2020-07-18T03:24:50.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.184 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2020-07-18T03:24:50.184 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2020-07-18T03:24:50.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.185 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2020-07-18T03:24:50.185 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2020-07-18T03:24:50.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2020-07-18T03:24:50.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2020-07-18T03:24:50.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.186 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2020-07-18T03:24:50.186 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2020-07-18T03:24:50.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.186 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2020-07-18T03:24:50.187 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2020-07-18T03:24:50.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2020-07-18T03:24:50.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2020-07-18T03:24:50.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2020-07-18T03:24:50.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2020-07-18T03:24:50.188 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2020-07-18T03:24:50.188 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2020-07-18T03:24:50.188 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-07-18T03:24:50.188 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2020-07-18T03:24:50.188 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2020-07-18T03:24:50.189 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.189 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-07-18T03:24:50.189 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.189 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2020-07-18T03:24:50.189 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.190 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2020-07-18T03:24:50.190 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.190 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2020-07-18T03:24:50.190 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2020-07-18T03:24:50.190 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.18,[v2:172.21.15.18:3301,v1:172.21.15.18:6790]" 2020-07-18T03:24:50.191 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = e5c9cb72-d58e-4215-ba0e-aa8d854b8526 2020-07-18T03:24:50.191 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.191 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2020-07-18T03:24:50.191 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2020-07-18T03:24:50.191 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.191 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2020-07-18T03:24:50.192 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2020-07-18T03:24:50.192 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.192 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2020-07-18T03:24:50.192 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2020-07-18T03:24:50.192 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.193 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2020-07-18T03:24:50.193 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2020-07-18T03:24:50.193 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2020-07-18T03:24:50.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.193 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2020-07-18T03:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2020-07-18T03:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2020-07-18T03:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2020-07-18T03:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.195 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2020-07-18T03:24:50.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.195 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2020-07-18T03:24:50.195 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2020-07-18T03:24:50.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.196 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2020-07-18T03:24:50.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2020-07-18T03:24:50.196 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2020-07-18T03:24:50.196 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2020-07-18T03:24:50.196 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2020-07-18T03:24:50.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2020-07-18T03:24:50.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug filestore = 20 2020-07-18T03:24:50.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug journal = 20 2020-07-18T03:24:50.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 20 2020-07-18T03:24:50.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 25 2020-07-18T03:24:50.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2020-07-18T03:24:50.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2020-07-18T03:24:50.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2020-07-18T03:24:50.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2020-07-18T03:24:50.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2020-07-18T03:24:50.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2020-07-18T03:24:50.199 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2020-07-18T03:24:50.199 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.199 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2020-07-18T03:24:50.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2020-07-18T03:24:50.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2020-07-18T03:24:50.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2020-07-18T03:24:50.200 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2020-07-18T03:24:50.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2020-07-18T03:24:50.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2020-07-18T03:24:50.200 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2020-07-18T03:24:50.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.201 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2020-07-18T03:24:50.201 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2020-07-18T03:24:50.201 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2020-07-18T03:24:50.201 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2020-07-18T03:24:50.201 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2020-07-18T03:24:50.202 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2020-07-18T03:24:50.202 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2020-07-18T03:24:50.202 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2020-07-18T03:24:50.202 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2020-07-18T03:24:50.202 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2020-07-18T03:24:50.202 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2020-07-18T03:24:50.203 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2020-07-18T03:24:50.203 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:24:50.203 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2020-07-18T03:24:50.203 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2020-07-18T03:24:50.203 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2020-07-18T03:24:50.204 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2020-07-18T03:24:50.204 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-07-18T03:24:50.204 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-07-18T03:24:50.204 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2020-07-18T03:24:50.204 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2020-07-18T03:24:50.205 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2020-07-18T03:24:50.205 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2020-07-18T03:24:50.205 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2020-07-18T03:24:50.205 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2020-07-18T03:24:50.205 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2020-07-18T03:24:50.205 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2020-07-18T03:24:50.206 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2020-07-18T03:24:50.206 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2020-07-18T03:24:50.206 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2020-07-18T03:24:50.206 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2020-07-18T03:24:50.206 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal frag = True 2020-07-18T03:24:50.207 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal fragment size max = 10000 2020-07-18T03:24:50.207 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal merge size = 5 2020-07-18T03:24:50.207 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split bits = 3 2020-07-18T03:24:50.207 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split size = 100 2020-07-18T03:24:50.207 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2020-07-18T03:24:50.207 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2020-07-18T03:24:50.208 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2020-07-18T03:24:50.208 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2020-07-18T03:24:50.208 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2020-07-18T03:24:50.209 INFO:tasks.ceph:Creating admin key on mon.a... 2020-07-18T03:24:50.210 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-07-18T03:24:50.267 INFO:tasks.ceph:Copying monmap to all nodes... 2020-07-18T03:24:50.287 DEBUG:teuthology.orchestra.remote:smithi047:/etc/ceph/ceph.keyring is 206B 2020-07-18T03:24:50.389 DEBUG:teuthology.orchestra.remote:smithi047:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-07-18T03:24:50.437 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:24:50.438 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-07-18T03:24:50.501 INFO:teuthology.orchestra.run.smithi047:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:50.593 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:24:50.594 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-07-18T03:24:50.659 INFO:teuthology.orchestra.run.smithi018:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:50.743 INFO:tasks.ceph:Setting up mon nodes... 2020-07-18T03:24:50.744 INFO:tasks.ceph:Setting up mgr nodes... 2020-07-18T03:24:50.744 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-07-18T03:24:50.808 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-07-18T03:24:50.810 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-07-18T03:24:50.894 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-07-18T03:24:50.897 INFO:tasks.ceph:Setting up mds nodes... 2020-07-18T03:24:50.897 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-07-18T03:24:50.961 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-07-18T03:24:50.962 INFO:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-07-18T03:24:50.993 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-07-18T03:24:51.056 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-07-18T03:24:51.059 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-07-18T03:24:51.091 INFO:tasks.ceph_client:Setting up client nodes... 2020-07-18T03:24:51.092 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-07-18T03:24:51.136 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-07-18T03:24:51.157 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-07-18T03:24:51.200 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-07-18T03:24:51.220 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-07-18T03:24:51.265 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.client.2.keyring 2020-07-18T03:24:51.284 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2020-07-18T03:24:51.328 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.client.3.keyring 2020-07-18T03:24:51.347 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-07-18T03:24:51.347 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi018.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-07-18T03:24:51.347 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-07-18T03:24:51.379 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-07-18T03:24:51.380 INFO:tasks.ceph:role: osd.0 2020-07-18T03:24:51.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:24:51.380 INFO:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-07-18T03:24:51.475 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:24:51.475 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:24:51.475 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:24:51.476 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2020-07-18T03:24:51.476 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:24:51.476 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:24:51.476 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:24:51.476 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:24:51.477 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:24:51.477 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:24:51.478 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2020-07-18T03:24:51.478 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-07-18T03:24:51.523 INFO:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-07-18T03:24:51.558 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-07-18T03:24:51.634 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-07-18T03:24:51.634 INFO:tasks.ceph:role: osd.1 2020-07-18T03:24:51.634 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:24:51.635 INFO:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-07-18T03:24:51.728 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:24:51.729 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:24:51.729 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:24:51.729 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2020-07-18T03:24:51.729 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:24:51.730 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:24:51.730 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:24:51.730 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:24:51.730 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:24:51.730 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:24:51.731 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2020-07-18T03:24:51.732 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-07-18T03:24:51.776 INFO:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-07-18T03:24:51.851 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-07-18T03:24:51.926 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-07-18T03:24:51.927 INFO:tasks.ceph:role: osd.2 2020-07-18T03:24:51.927 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:24:51.928 INFO:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-07-18T03:24:52.021 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:24:52.021 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:24:52.022 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:24:52.022 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2020-07-18T03:24:52.022 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:24:52.022 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:24:52.022 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:24:52.023 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:24:52.023 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:24:52.023 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:24:52.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2020-07-18T03:24:52.024 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-07-18T03:24:52.069 INFO:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-07-18T03:24:52.145 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-07-18T03:24:52.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-07-18T03:24:52.217 INFO:tasks.ceph:role: osd.3 2020-07-18T03:24:52.217 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:24:52.218 INFO:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-07-18T03:24:52.310 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:24:52.311 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:24:52.311 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:24:52.311 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2020-07-18T03:24:52.311 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:24:52.312 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:24:52.312 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:24:52.312 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:24:52.312 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:24:52.312 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:24:52.313 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2020-07-18T03:24:52.314 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-07-18T03:24:52.448 INFO:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-07-18T03:24:52.482 INFO:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:52.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:52.571+0000 7f003a939ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-07-18T03:24:52.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:52.571+0000 7f003a939ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-07-18T03:24:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:52.571+0000 7f003a939ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-07-18T03:24:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:52.571+0000 7f003a939ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-07-18T03:24:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:52.571+0000 7f003a939ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-07-18T03:24:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:52.571+0000 7f003a939ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-07-18T03:24:54.914 INFO:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-07-18T03:24:54.951 INFO:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:55.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:55.043+0000 7f7a210fcec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-07-18T03:24:55.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:55.043+0000 7f7a210fcec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-07-18T03:24:55.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:55.043+0000 7f7a210fcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-07-18T03:24:55.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:55.044+0000 7f7a210fcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-07-18T03:24:55.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:55.044+0000 7f7a210fcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-07-18T03:24:55.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:55.044+0000 7f7a210fcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-07-18T03:24:57.391 INFO:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-07-18T03:24:57.427 INFO:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:24:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:57.521+0000 7fb245eaeec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-07-18T03:24:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:57.521+0000 7fb245eaeec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-07-18T03:24:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:57.521+0000 7fb245eaeec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-07-18T03:24:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:57.521+0000 7fb245eaeec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-07-18T03:24:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:57.521+0000 7fb245eaeec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-07-18T03:24:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:24:57.521+0000 7fb245eaeec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-07-18T03:24:59.877 INFO:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-07-18T03:24:59.915 INFO:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:00.007+0000 7fe631126ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-07-18T03:25:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:00.007+0000 7fe631126ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-07-18T03:25:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:00.007+0000 7fe631126ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-07-18T03:25:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:00.007+0000 7fe631126ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-07-18T03:25:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:00.007+0000 7fe631126ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-07-18T03:25:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:00.007+0000 7fe631126ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-07-18T03:25:02.347 INFO:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-07-18T03:25:02.385 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-07-18T03:25:02.423 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-07-18T03:25:02.423 INFO:tasks.ceph:role: osd.4 2020-07-18T03:25:02.424 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:25:02.424 INFO:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-07-18T03:25:02.521 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:25:02.522 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:25:02.522 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:25:02.522 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 2020-07-18T03:25:02.523 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:25:02.523 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:25:02.523 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:25:02.523 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:25:02.523 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:25:02.524 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:25:02.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2020-07-18T03:25:02.525 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-07-18T03:25:02.572 INFO:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-07-18T03:25:02.608 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-07-18T03:25:02.682 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-07-18T03:25:02.683 INFO:tasks.ceph:role: osd.5 2020-07-18T03:25:02.683 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:25:02.684 INFO:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-07-18T03:25:02.778 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:25:02.778 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:25:02.778 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:25:02.778 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 2020-07-18T03:25:02.779 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:25:02.779 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:25:02.779 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:25:02.779 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:25:02.779 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:25:02.780 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:25:02.780 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2020-07-18T03:25:02.781 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-07-18T03:25:02.824 INFO:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-07-18T03:25:02.900 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-07-18T03:25:02.971 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-07-18T03:25:02.972 INFO:tasks.ceph:role: osd.6 2020-07-18T03:25:02.972 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:25:02.973 INFO:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-07-18T03:25:03.065 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:25:03.066 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:25:03.066 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:25:03.066 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 2020-07-18T03:25:03.066 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:25:03.066 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:25:03.067 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:25:03.067 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:25:03.067 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:25:03.067 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:25:03.068 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2020-07-18T03:25:03.069 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-07-18T03:25:03.113 INFO:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-07-18T03:25:03.187 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-07-18T03:25:03.257 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-07-18T03:25:03.258 INFO:tasks.ceph:role: osd.7 2020-07-18T03:25:03.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi018.front.sepia.ceph.com 2020-07-18T03:25:03.259 INFO:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-07-18T03:25:03.352 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-07-18T03:25:03.353 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2020-07-18T03:25:03.353 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-07-18T03:25:03.354 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 2020-07-18T03:25:03.354 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-07-18T03:25:03.355 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2020-07-18T03:25:03.355 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-07-18T03:25:03.355 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-07-18T03:25:03.356 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-07-18T03:25:03.356 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-07-18T03:25:03.358 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2020-07-18T03:25:03.358 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-07-18T03:25:03.402 INFO:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-07-18T03:25:03.477 INFO:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:03.566 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:03.565+0000 7f9c8dffcec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-07-18T03:25:03.567 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:03.565+0000 7f9c8dffcec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-07-18T03:25:03.567 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:03.565+0000 7f9c8dffcec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-07-18T03:25:03.567 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:03.565+0000 7f9c8dffcec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-07-18T03:25:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:03.565+0000 7f9c8dffcec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-07-18T03:25:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:03.565+0000 7f9c8dffcec0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-07-18T03:25:05.913 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-07-18T03:25:05.953 INFO:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:06.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:06.045+0000 7f74348a6ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-07-18T03:25:06.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:06.046+0000 7f74348a6ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-07-18T03:25:06.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:06.046+0000 7f74348a6ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-07-18T03:25:06.048 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:06.046+0000 7f74348a6ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-07-18T03:25:06.048 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:06.046+0000 7f74348a6ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-07-18T03:25:06.048 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:06.046+0000 7f74348a6ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-07-18T03:25:08.389 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-07-18T03:25:08.425 INFO:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:08.518 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:08.516+0000 7fbd64f2cec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-07-18T03:25:08.518 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:08.517+0000 7fbd64f2cec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-07-18T03:25:08.518 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:08.517+0000 7fbd64f2cec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-07-18T03:25:08.519 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:08.517+0000 7fbd64f2cec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-07-18T03:25:08.519 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:08.517+0000 7fbd64f2cec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-07-18T03:25:08.519 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:08.517+0000 7fbd64f2cec0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-07-18T03:25:10.861 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-07-18T03:25:10.898 INFO:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:10.990 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:10.989+0000 7fdd6d149ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-07-18T03:25:10.991 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:10.989+0000 7fdd6d149ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-07-18T03:25:10.991 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:10.989+0000 7fdd6d149ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-07-18T03:25:10.991 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:10.989+0000 7fdd6d149ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-07-18T03:25:10.991 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:10.989+0000 7fdd6d149ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-07-18T03:25:10.991 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:10.989+0000 7fdd6d149ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-07-18T03:25:13.328 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-07-18T03:25:13.364 INFO:tasks.ceph:Reading keys from all nodes... 2020-07-18T03:25:13.365 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:13.383 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.7FFFahCWwD 2020-07-18T03:25:13.384 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.7FFFahCWwD 2020-07-18T03:25:13.460 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.7FFFahCWwD 2020-07-18T03:25:13.593 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.7FFFahCWwD is 56B 2020-07-18T03:25:13.643 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.7FFFahCWwD 2020-07-18T03:25:13.658 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:13.712 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.rqPEhV8s9d 2020-07-18T03:25:13.713 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.rqPEhV8s9d 2020-07-18T03:25:13.785 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.rqPEhV8s9d 2020-07-18T03:25:13.914 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.rqPEhV8s9d is 56B 2020-07-18T03:25:13.963 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.rqPEhV8s9d 2020-07-18T03:25:13.977 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:14.031 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.dFgqKBNnTA 2020-07-18T03:25:14.031 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.dFgqKBNnTA 2020-07-18T03:25:14.102 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.dFgqKBNnTA 2020-07-18T03:25:14.230 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.dFgqKBNnTA is 56B 2020-07-18T03:25:14.277 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.dFgqKBNnTA 2020-07-18T03:25:14.292 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:14.345 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.VgQHWC10Ll 2020-07-18T03:25:14.346 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.VgQHWC10Ll 2020-07-18T03:25:14.416 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.VgQHWC10Ll 2020-07-18T03:25:14.543 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.VgQHWC10Ll is 56B 2020-07-18T03:25:14.591 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.VgQHWC10Ll 2020-07-18T03:25:14.606 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:14.658 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.d7fZYnGBUm 2020-07-18T03:25:14.659 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.d7fZYnGBUm 2020-07-18T03:25:14.730 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.d7fZYnGBUm 2020-07-18T03:25:14.860 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.d7fZYnGBUm is 56B 2020-07-18T03:25:14.915 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.d7fZYnGBUm 2020-07-18T03:25:14.931 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:14.985 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.EsnbhNmzCw 2020-07-18T03:25:14.986 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.EsnbhNmzCw 2020-07-18T03:25:15.056 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.EsnbhNmzCw 2020-07-18T03:25:15.183 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.EsnbhNmzCw is 56B 2020-07-18T03:25:15.231 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.EsnbhNmzCw 2020-07-18T03:25:15.245 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:15.263 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.zkJ51wIAgx 2020-07-18T03:25:15.264 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.zkJ51wIAgx 2020-07-18T03:25:15.340 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.zkJ51wIAgx 2020-07-18T03:25:15.473 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.zkJ51wIAgx is 56B 2020-07-18T03:25:15.525 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.zkJ51wIAgx 2020-07-18T03:25:15.541 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:15.596 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.3YoHwJpwjq 2020-07-18T03:25:15.597 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.3YoHwJpwjq 2020-07-18T03:25:15.668 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.3YoHwJpwjq 2020-07-18T03:25:15.797 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.3YoHwJpwjq is 56B 2020-07-18T03:25:15.844 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.3YoHwJpwjq 2020-07-18T03:25:15.858 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:15.911 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.F0Ptxl5t0P 2020-07-18T03:25:15.912 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.F0Ptxl5t0P 2020-07-18T03:25:15.984 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.F0Ptxl5t0P 2020-07-18T03:25:16.111 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.F0Ptxl5t0P is 56B 2020-07-18T03:25:16.158 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.F0Ptxl5t0P 2020-07-18T03:25:16.173 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:16.228 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.O24JOphtLa 2020-07-18T03:25:16.229 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.O24JOphtLa 2020-07-18T03:25:16.299 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.O24JOphtLa 2020-07-18T03:25:16.426 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.O24JOphtLa is 56B 2020-07-18T03:25:16.475 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.O24JOphtLa 2020-07-18T03:25:16.489 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:16.544 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.EBu00eVQBF 2020-07-18T03:25:16.544 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.EBu00eVQBF 2020-07-18T03:25:16.615 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.EBu00eVQBF 2020-07-18T03:25:16.742 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.EBu00eVQBF is 56B 2020-07-18T03:25:16.789 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.EBu00eVQBF 2020-07-18T03:25:16.803 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:16.856 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.R6sPQLlpJf 2020-07-18T03:25:16.857 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.R6sPQLlpJf 2020-07-18T03:25:16.928 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.R6sPQLlpJf 2020-07-18T03:25:17.055 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.R6sPQLlpJf is 56B 2020-07-18T03:25:17.104 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.R6sPQLlpJf 2020-07-18T03:25:17.138 DEBUG:teuthology.orchestra.remote:smithi047:/etc/ceph/ceph.client.0.keyring is 59B 2020-07-18T03:25:17.233 DEBUG:teuthology.orchestra.remote:smithi047:/etc/ceph/ceph.client.1.keyring is 59B 2020-07-18T03:25:17.298 DEBUG:teuthology.orchestra.remote:smithi018:/etc/ceph/ceph.client.2.keyring is 59B 2020-07-18T03:25:17.392 DEBUG:teuthology.orchestra.remote:smithi018:/etc/ceph/ceph.client.3.keyring is 59B 2020-07-18T03:25:17.440 INFO:tasks.ceph:Adding keys to all mons... 2020-07-18T03:25:17.440 INFO:teuthology.orchestra.run.smithi018:> sudo tee -a /etc/ceph/ceph.keyring 2020-07-18T03:25:17.443 INFO:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2020-07-18T03:25:17.484 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr.y] 2020-07-18T03:25:17.484 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw== 2020-07-18T03:25:17.484 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a] 2020-07-18T03:25:17.485 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g== 2020-07-18T03:25:17.485 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.0] 2020-07-18T03:25:17.485 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA== 2020-07-18T03:25:17.485 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.1] 2020-07-18T03:25:17.486 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ== 2020-07-18T03:25:17.486 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.2] 2020-07-18T03:25:17.486 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA== 2020-07-18T03:25:17.486 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.3] 2020-07-18T03:25:17.487 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ== 2020-07-18T03:25:17.487 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr.x] 2020-07-18T03:25:17.487 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ== 2020-07-18T03:25:17.487 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b] 2020-07-18T03:25:17.488 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ== 2020-07-18T03:25:17.488 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.4] 2020-07-18T03:25:17.488 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ== 2020-07-18T03:25:17.488 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.5] 2020-07-18T03:25:17.488 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA== 2020-07-18T03:25:17.489 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.6] 2020-07-18T03:25:17.489 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg== 2020-07-18T03:25:17.489 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.7] 2020-07-18T03:25:17.489 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg== 2020-07-18T03:25:17.490 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2020-07-18T03:25:17.490 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw== 2020-07-18T03:25:17.490 INFO:teuthology.orchestra.run.smithi018.stdout:[client.1] 2020-07-18T03:25:17.490 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g== 2020-07-18T03:25:17.490 INFO:teuthology.orchestra.run.smithi018.stdout:[client.2] 2020-07-18T03:25:17.491 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w== 2020-07-18T03:25:17.491 INFO:teuthology.orchestra.run.smithi018.stdout:[client.3] 2020-07-18T03:25:17.491 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog== 2020-07-18T03:25:17.493 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2020-07-18T03:25:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw== 2020-07-18T03:25:17.493 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2020-07-18T03:25:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g== 2020-07-18T03:25:17.494 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2020-07-18T03:25:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA== 2020-07-18T03:25:17.494 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2020-07-18T03:25:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ== 2020-07-18T03:25:17.495 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2020-07-18T03:25:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA== 2020-07-18T03:25:17.495 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2020-07-18T03:25:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ== 2020-07-18T03:25:17.496 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2020-07-18T03:25:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ== 2020-07-18T03:25:17.496 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2020-07-18T03:25:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ== 2020-07-18T03:25:17.496 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2020-07-18T03:25:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ== 2020-07-18T03:25:17.497 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2020-07-18T03:25:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA== 2020-07-18T03:25:17.497 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2020-07-18T03:25:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg== 2020-07-18T03:25:17.498 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2020-07-18T03:25:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg== 2020-07-18T03:25:17.498 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2020-07-18T03:25:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw== 2020-07-18T03:25:17.499 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2020-07-18T03:25:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g== 2020-07-18T03:25:17.499 INFO:teuthology.orchestra.run.smithi047.stdout:[client.2] 2020-07-18T03:25:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w== 2020-07-18T03:25:17.500 INFO:teuthology.orchestra.run.smithi047.stdout:[client.3] 2020-07-18T03:25:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog== 2020-07-18T03:25:17.500 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-07-18T03:25:17.535 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-07-18T03:25:17.592 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-07-18T03:25:17.594 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-07-18T03:25:17.652 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.655 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.707 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.710 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.762 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.765 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.817 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.820 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.872 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-07-18T03:25:17.875 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-07-18T03:25:17.927 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-07-18T03:25:17.931 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-07-18T03:25:17.986 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:17.989 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.042 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.045 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.097 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.099 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.151 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.154 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-07-18T03:25:18.207 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.210 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.265 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.268 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.323 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.326 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.382 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.384 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-07-18T03:25:18.439 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-07-18T03:25:18.440 INFO:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-07-18T03:25:18.473 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-07-18T03:25:18.571 INFO:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-07-18T03:25:18.603 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-07-18T03:25:18.635 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-07-18T03:25:18.733 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-07-18T03:25:18.764 INFO:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-07-18T03:25:18.834 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-07-18T03:25:18.934 INFO:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-07-18T03:25:18.965 INFO:teuthology.orchestra.run.smithi018:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:18.984 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:25:19.008 INFO:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-07-18T03:25:19.010 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:25:19.024 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-07-18T03:25:19.025 INFO:tasks.ceph.mon.a:Restarting daemon 2020-07-18T03:25:19.025 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-07-18T03:25:19.066 INFO:tasks.ceph.mon.a:Started 2020-07-18T03:25:19.066 INFO:tasks.ceph.mon.b:Restarting daemon 2020-07-18T03:25:19.067 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-07-18T03:25:19.069 INFO:tasks.ceph.mon.b:Started 2020-07-18T03:25:19.070 INFO:tasks.ceph.mon.c:Restarting daemon 2020-07-18T03:25:19.070 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-07-18T03:25:19.072 INFO:tasks.ceph.mon.c:Started 2020-07-18T03:25:19.072 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-07-18T03:25:19.073 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-07-18T03:25:19.073 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-07-18T03:25:19.075 INFO:tasks.ceph.mgr.y:Started 2020-07-18T03:25:19.076 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-07-18T03:25:19.076 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-07-18T03:25:19.078 INFO:tasks.ceph.mgr.x:Started 2020-07-18T03:25:19.079 INFO:tasks.ceph:Setting crush tunables to default 2020-07-18T03:25:19.079 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2020-07-18T03:25:24.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.233+0000 7f4bf00c0700 1 Processor -- start 2020-07-18T03:25:24.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.233+0000 7f4bf00c0700 1 -- start start 2020-07-18T03:25:24.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bf00c0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bf00c0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bf00c0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be81548a0 con 0x7f4be80625a0 2020-07-18T03:25:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bf00c0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8151960 con 0x7f4be8080700 2020-07-18T03:25:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bf00c0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8154b40 con 0x7f4be80ae4b0 2020-07-18T03:25:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bede5c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bee65d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4be80625a0 0x7f4be80afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38758/0 (socket says 172.21.15.47:38758) 2020-07-18T03:25:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 learned_addr learned my addr 172.21.15.47:0/1888377736 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-07-18T03:25:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-07-18T03:25:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-07-18T03:25:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.234+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.235+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-07-18T03:25:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-07-18T03:25:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-07-18T03:25:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-07-18T03:25:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.436+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-07-18T03:25:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-07-18T03:25:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-07-18T03:25:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-07-18T03:25:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:19.837+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-07-18T03:25:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 msgr2=0x7f4be80ae920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-07-18T03:25:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4be80ae4b0 0x7f4be80ae920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 msgr2=0x7f4be8080b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-07-18T03:25:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4be8080700 0x7f4be8080b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 >> v1:172.21.15.47:6789/0 conn(0x7f4be80625a0 legacy=0x7f4be80afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be8154de0 con 0x7f4bcc005ee0 2020-07-18T03:25:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8151960 con 0x7f4bcc0033d0 2020-07-18T03:25:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.235+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-07-18T03:25:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.436+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.436+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.436+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.436+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.436+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-07-18T03:25:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.837+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.837+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.837+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.837+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:20.837+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-07-18T03:25:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:21.639+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:21.639+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:21.639+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:21.639+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:21.639+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-07-18T03:25:24.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-07-18T03:25:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 >> v1:172.21.15.18:6789/0 conn(0x7f4bcc005ee0 legacy=0x7f4bcc008360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 reap_dead start 2020-07-18T03:25:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc00ad90 con 0x7f4bcc009d50 2020-07-18T03:25:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8151960 con 0x7f4bcc0033d0 2020-07-18T03:25:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bdffff700 1 -- 172.21.15.47:0/1888377736 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8154b40 con 0x7f4bcc00a160 2020-07-18T03:25:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-07-18T03:25:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-07-18T03:25:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.235+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-07-18T03:25:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-07-18T03:25:24.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-07-18T03:25:24.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.436+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-07-18T03:25:24.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-07-18T03:25:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-07-18T03:25:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-07-18T03:25:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4bee65d700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-07-18T03:25:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-07-18T03:25:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:23.837+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-07-18T03:25:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.396+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555882414 0 0) 0x7f4bcc00ad90 con 0x7f4bcc009d50 2020-07-18T03:25:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.396+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd4003240 con 0x7f4bcc009d50 2020-07-18T03:25:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3937834258 0 0) 0x7f4bd4003240 con 0x7f4bcc009d50 2020-07-18T03:25:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 msgr2=0x7f4bcc00a610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-07-18T03:25:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.47:3300/0 conn(0x7f4bcc00a160 0x7f4bcc00a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 msgr2=0x7f4bcc005850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-07-18T03:25:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 --2- 172.21.15.47:0/1888377736 >> v2:172.21.15.18:3300/0 conn(0x7f4bcc0033d0 0x7f4bcc005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc00e1d0 con 0x7f4bcc009d50 2020-07-18T03:25:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.398+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4bd800b330 con 0x7f4bcc009d50 2020-07-18T03:25:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 >> v1:172.21.15.47:6789/0 conn(0x7f4bcc009d50 legacy=0x7f4bcc008360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd4003730 0x7f4bd4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd40040b0 0x7f4bd4004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4004b80 0x7f4bd400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8151960 con 0x7f4bd4004b80 2020-07-18T03:25:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8154b40 con 0x7f4bd40040b0 2020-07-18T03:25:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4005320 con 0x7f4bd4003730 2020-07-18T03:25:24.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd40040b0 0x7f4bd4004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bee65d700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd4003730 0x7f4bd4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4004b80 0x7f4bd400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd4003730 msgr2=0x7f4bd4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd4003730 0x7f4bd4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4004b80 msgr2=0x7f4bd400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4004b80 0x7f4bd400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 -- 172.21.15.47:0/1888377736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4003240 con 0x7f4bd40040b0 2020-07-18T03:25:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.399+0000 7f4bede5c700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd40040b0 0x7f4bd4004580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f4bd40054b0 tx=0x7f4bd8008040).ready entity=mon.1 client_cookie=74b2b4f12e41a242 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4beee5e700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4004b80 0x7f4bd400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:25:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4beee5e700 1 -- 172.21.15.47:0/1888377736 reap_dead start 2020-07-18T03:25:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bd8014600 con 0x7f4bd40040b0 2020-07-18T03:25:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd8014760 con 0x7f4bd40040b0 2020-07-18T03:25:24.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bed65b700 1 -- 172.21.15.47:0/1888377736 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bd8014ac0 con 0x7f4bd40040b0 2020-07-18T03:25:24.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 -- 172.21.15.47:0/1888377736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd40040b0 msgr2=0x7f4bd4004580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd40040b0 0x7f4bd4004580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f4bd40054b0 tx=0x7f4bd8008040).stop 2020-07-18T03:25:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 -- 172.21.15.47:0/1888377736 shutdown_connections 2020-07-18T03:25:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 --2- 172.21.15.47:0/1888377736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd40040b0 0x7f4bd4004580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 -- 172.21.15.47:0/1888377736 >> 172.21.15.47:0/1888377736 conn(0x7f4be805f5a0 msgr2=0x7f4be805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 -- 172.21.15.47:0/1888377736 shutdown_connections 2020-07-18T03:25:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.400+0000 7f4bf00c0700 1 -- 172.21.15.47:0/1888377736 wait complete. 2020-07-18T03:25:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.403+0000 7f4bf00c0700 1 Processor -- start 2020-07-18T03:25:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.403+0000 7f4bf00c0700 1 -- start start 2020-07-18T03:25:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bf00c0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4be80b2020 0x7f4be80b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bf00c0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4be8063940 0x7f4be80b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bf00c0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be8062f60 con 0x7f4be80b8100 2020-07-18T03:25:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bf00c0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8063200 con 0x7f4be80b2020 2020-07-18T03:25:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bf00c0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80634a0 con 0x7f4be8063940 2020-07-18T03:25:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4beee5e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4be8063940 0x7f4be80b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bee65d700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f4be80b8100 0x7f4be8152460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:57616/0 (socket says 172.21.15.47:57616) 2020-07-18T03:25:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bee65d700 1 -- 172.21.15.47:0/172129792 learned_addr learned my addr 172.21.15.47:0/172129792 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4bede5c700 1 --2- 172.21.15.47:0/172129792 >> v2:172.21.15.18:3300/0 conn(0x7f4be80b2020 0x7f4be80b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4beee5e700 1 -- 172.21.15.47:0/172129792 >> v2:172.21.15.18:3300/0 conn(0x7f4be80b2020 msgr2=0x7f4be80b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4beee5e700 1 --2- 172.21.15.47:0/172129792 >> v2:172.21.15.18:3300/0 conn(0x7f4be80b2020 0x7f4be80b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4beee5e700 1 -- 172.21.15.47:0/172129792 >> v1:172.21.15.18:6789/0 conn(0x7f4be80b8100 legacy=0x7f4be8152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4beee5e700 1 -- 172.21.15.47:0/172129792 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8063dd0 con 0x7f4be8063940 2020-07-18T03:25:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.404+0000 7f4beee5e700 1 --2- 172.21.15.47:0/172129792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be8063940 0x7f4be80b04f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f4be000a000 tx=0x7f4be0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be0015a10 con 0x7f4be8063940 2020-07-18T03:25:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0015b70 con 0x7f4be8063940 2020-07-18T03:25:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be80b09e0 con 0x7f4be8063940 2020-07-18T03:25:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be0015ed0 con 0x7f4be8063940 2020-07-18T03:25:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be80b0ed0 con 0x7f4be8063940 2020-07-18T03:25:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f4be002a070 con 0x7f4be8063940 2020-07-18T03:25:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.405+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f4be0023dd0 con 0x7f4be8063940 2020-07-18T03:25:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.406+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8057450 con 0x7f4be8063940 2020-07-18T03:25:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.409+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7f4be0028060 con 0x7f4be8063940 2020-07-18T03:25:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.492+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4be8068c50 con 0x7f4be8063940 2020-07-18T03:25:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.495+0000 7f4bddffb700 1 -- 172.21.15.47:0/172129792 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4be0023530 con 0x7f4be8063940 2020-07-18T03:25:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.495+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be8063940 msgr2=0x7f4be80b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.495+0000 7f4bf00c0700 1 --2- 172.21.15.47:0/172129792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be8063940 0x7f4be80b04f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f4be000a000 tx=0x7f4be0008040).stop 2020-07-18T03:25:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.496+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 shutdown_connections 2020-07-18T03:25:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.496+0000 7f4bf00c0700 1 --2- 172.21.15.47:0/172129792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be8063940 0x7f4be80b04f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.496+0000 7f4bf00c0700 1 --2- 172.21.15.47:0/172129792 >> v2:172.21.15.18:3300/0 conn(0x7f4be80b2020 0x7f4be80b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:24.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.496+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 >> 172.21.15.47:0/172129792 conn(0x7f4be805f5a0 msgr2=0x7f4be80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:24.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.496+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 shutdown_connections 2020-07-18T03:25:24.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:24.496+0000 7f4bf00c0700 1 -- 172.21.15.47:0/172129792 wait complete. 2020-07-18T03:25:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2020-07-18T03:25:24.515 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-07-18T03:25:24.516 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:24.532 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.n2k5PO0Dd3 2020-07-18T03:25:24.533 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.n2k5PO0Dd3 2020-07-18T03:25:24.604 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.n2k5PO0Dd3 2020-07-18T03:25:24.735 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.n2k5PO0Dd3 is 37B 2020-07-18T03:25:24.783 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.n2k5PO0Dd3 2020-07-18T03:25:24.798 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:24.852 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.xVWzm6tjL0 2020-07-18T03:25:24.852 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.xVWzm6tjL0 2020-07-18T03:25:24.925 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.xVWzm6tjL0 2020-07-18T03:25:25.053 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.xVWzm6tjL0 is 37B 2020-07-18T03:25:25.100 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.xVWzm6tjL0 2020-07-18T03:25:25.115 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:25.169 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.rt5tbXKUJw 2020-07-18T03:25:25.170 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.rt5tbXKUJw 2020-07-18T03:25:25.243 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.rt5tbXKUJw 2020-07-18T03:25:25.375 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.rt5tbXKUJw is 37B 2020-07-18T03:25:25.423 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.rt5tbXKUJw 2020-07-18T03:25:25.440 INFO:teuthology.orchestra.run.smithi047:> mktemp 2020-07-18T03:25:25.504 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.xumpB0UvVE 2020-07-18T03:25:25.504 INFO:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.xumpB0UvVE 2020-07-18T03:25:25.582 INFO:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.xumpB0UvVE 2020-07-18T03:25:25.710 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.xumpB0UvVE is 37B 2020-07-18T03:25:25.758 INFO:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.xumpB0UvVE 2020-07-18T03:25:25.773 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:25.787 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.l5mHpw6q4p 2020-07-18T03:25:25.788 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.l5mHpw6q4p 2020-07-18T03:25:25.863 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.l5mHpw6q4p 2020-07-18T03:25:25.993 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.l5mHpw6q4p is 37B 2020-07-18T03:25:26.040 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.l5mHpw6q4p 2020-07-18T03:25:26.055 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:26.109 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.r5iA9EOi4I 2020-07-18T03:25:26.110 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.r5iA9EOi4I 2020-07-18T03:25:26.182 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.r5iA9EOi4I 2020-07-18T03:25:26.314 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.r5iA9EOi4I is 37B 2020-07-18T03:25:26.361 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.r5iA9EOi4I 2020-07-18T03:25:26.375 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:26.429 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.iglrFiyTNG 2020-07-18T03:25:26.429 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.iglrFiyTNG 2020-07-18T03:25:26.500 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.iglrFiyTNG 2020-07-18T03:25:26.631 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.iglrFiyTNG is 37B 2020-07-18T03:25:26.687 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.iglrFiyTNG 2020-07-18T03:25:26.703 INFO:teuthology.orchestra.run.smithi018:> mktemp 2020-07-18T03:25:26.762 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.bYFbuCpCdP 2020-07-18T03:25:26.763 INFO:teuthology.orchestra.run.smithi018:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.bYFbuCpCdP 2020-07-18T03:25:26.837 INFO:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.bYFbuCpCdP 2020-07-18T03:25:27.124 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.bYFbuCpCdP is 37B 2020-07-18T03:25:27.144 INFO:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.bYFbuCpCdP 2020-07-18T03:25:27.158 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e 0 2020-07-18T03:25:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 Processor -- start 2020-07-18T03:25:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 -- start start 2020-07-18T03:25:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 0x7f3b8c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 0x7f3b8c080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8c1548a0 con 0x7f3b8c157420 2020-07-18T03:25:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c154b40 con 0x7f3b8c0625a0 2020-07-18T03:25:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b90b2d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c154de0 con 0x7f3b8c080700 2020-07-18T03:25:27.318 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.313+0000 7f3b8a59c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 0x7f3b8c080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.318 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8ad9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 0x7f3b8c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.318 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 0x7f3b8c080b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:49950/0 (socket says 172.21.15.18:49950) 2020-07-18T03:25:27.318 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 -- 172.21.15.18:0/4252023161 learned_addr learned my addr 172.21.15.18:0/4252023161 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:27.318 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8ad9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 0x7f3b8c0afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57962/0 (socket says 172.21.15.18:57962) 2020-07-18T03:25:27.319 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 -- 172.21.15.18:0/4252023161 >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 msgr2=0x7f3b8c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:27.319 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 --2- 172.21.15.18:0/4252023161 >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 0x7f3b8c0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.319 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 -- 172.21.15.18:0/4252023161 >> v1:172.21.15.47:6789/0 conn(0x7f3b8c157420 legacy=0x7f3b8c1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.319 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 -- 172.21.15.18:0/4252023161 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c1545a0 con 0x7f3b8c080700 2020-07-18T03:25:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8ad9d700 1 --2- 172.21.15.18:0/4252023161 >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 0x7f3b8c0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b8a59c700 1 --2- 172.21.15.18:0/4252023161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 0x7f3b8c080b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3b74002f30 tx=0x7f3b74006e90).ready entity=mon.2 client_cookie=fd8fbac05820535f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b89d9b700 1 -- 172.21.15.18:0/4252023161 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b74009850 con 0x7f3b8c080700 2020-07-18T03:25:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b89d9b700 1 -- 172.21.15.18:0/4252023161 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b740099b0 con 0x7f3b8c080700 2020-07-18T03:25:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b89d9b700 1 -- 172.21.15.18:0/4252023161 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b74009d10 con 0x7f3b8c080700 2020-07-18T03:25:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 -- 172.21.15.18:0/4252023161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 msgr2=0x7f3b8c080b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 --2- 172.21.15.18:0/4252023161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 0x7f3b8c080b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3b74002f30 tx=0x7f3b74006e90).stop 2020-07-18T03:25:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 -- 172.21.15.18:0/4252023161 shutdown_connections 2020-07-18T03:25:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 --2- 172.21.15.18:0/4252023161 >> v2:172.21.15.18:3300/0 conn(0x7f3b8c0625a0 0x7f3b8c0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 --2- 172.21.15.18:0/4252023161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c080700 0x7f3b8c080b80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.322 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 -- 172.21.15.18:0/4252023161 >> 172.21.15.18:0/4252023161 conn(0x7f3b8c05f5a0 msgr2=0x7f3b8c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:27.322 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 -- 172.21.15.18:0/4252023161 shutdown_connections 2020-07-18T03:25:27.322 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.314+0000 7f3b90b2d700 1 -- 172.21.15.18:0/4252023161 wait complete. 2020-07-18T03:25:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.316+0000 7f3b90b2d700 1 Processor -- start 2020-07-18T03:25:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.316+0000 7f3b90b2d700 1 -- start start 2020-07-18T03:25:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b90b2d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 0x7f3b8c0b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b90b2d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b8c151d80 0x7f3b8c153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b90b2d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8c0b0ae0 con 0x7f3b8c157420 2020-07-18T03:25:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b90b2d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c0b08d0 con 0x7f3b8c151d80 2020-07-18T03:25:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b90b2d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c151370 con 0x7f3b8c0625a0 2020-07-18T03:25:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 0x7f3b8c0b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b8c151d80 0x7f3b8c153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b8c151d80 0x7f3b8c153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.18:46720/0 (socket says 172.21.15.18:46720) 2020-07-18T03:25:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 0x7f3b8c0b7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:49954/0 (socket says 172.21.15.18:49954) 2020-07-18T03:25:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8b59e700 1 -- 172.21.15.18:0/363265262 learned_addr learned my addr 172.21.15.18:0/363265262 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8ad9d700 1 -- 172.21.15.18:0/363265262 >> v2:172.21.15.47:3300/0 conn(0x7f3b8c151d80 msgr2=0x7f3b8c153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8ad9d700 1 --2- 172.21.15.18:0/363265262 >> v2:172.21.15.47:3300/0 conn(0x7f3b8c151d80 0x7f3b8c153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.317+0000 7f3b8ad9d700 1 -- 172.21.15.18:0/363265262 >> v1:172.21.15.47:6789/0 conn(0x7f3b8c157420 legacy=0x7f3b8c152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b8ad9d700 1 -- 172.21.15.18:0/363265262 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0b7780 con 0x7f3b8c0625a0 2020-07-18T03:25:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b8ad9d700 1 --2- 172.21.15.18:0/363265262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 0x7f3b8c0b7290 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3b7c00c000 tx=0x7f3b7c006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7c00cb40 con 0x7f3b8c0625a0 2020-07-18T03:25:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00cca0 con 0x7f3b8c0625a0 2020-07-18T03:25:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c13a9f0 con 0x7f3b8c0625a0 2020-07-18T03:25:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7c0123f0 con 0x7f3b8c0625a0 2020-07-18T03:25:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c13aec0 con 0x7f3b8c0625a0 2020-07-18T03:25:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f3b7c012620 con 0x7f3b8c0625a0 2020-07-18T03:25:27.330 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.318+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f3b7c01c7d0 con 0x7f3b8c0625a0 2020-07-18T03:25:27.330 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.319+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c0b0c70 con 0x7f3b8c0625a0 2020-07-18T03:25:27.330 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.322+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7f3b7c012a40 con 0x7f3b8c0625a0 2020-07-18T03:25:27.419 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.412+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", "id": 0} v 0) v1 -- 0x7f3b8c057450 con 0x7f3b8c0625a0 2020-07-18T03:25:27.419 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.415+0000 7f3b827fc700 1 -- 172.21.15.18:0/363265262 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3b7c012be0 con 0x7f3b8c0625a0 2020-07-18T03:25:27.419 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 msgr2=0x7f3b8c0b7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.420 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 --2- 172.21.15.18:0/363265262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 0x7f3b8c0b7290 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3b7c00c000 tx=0x7f3b7c006dd0).stop 2020-07-18T03:25:27.420 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 shutdown_connections 2020-07-18T03:25:27.420 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 --2- 172.21.15.18:0/363265262 >> v2:172.21.15.47:3300/0 conn(0x7f3b8c151d80 0x7f3b8c153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.420 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 --2- 172.21.15.18:0/363265262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b8c0625a0 0x7f3b8c0b7290 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.420 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 >> 172.21.15.18:0/363265262 conn(0x7f3b8c05f5a0 msgr2=0x7f3b8c0afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:27.421 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 shutdown_connections 2020-07-18T03:25:27.421 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.416+0000 7f3b90b2d700 1 -- 172.21.15.18:0/363265262 wait complete. 2020-07-18T03:25:27.421 INFO:teuthology.orchestra.run.smithi018.stdout:0 2020-07-18T03:25:27.428 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new ec8353f5-7ad4-45bf-b8e5-99115dd53e24 1 2020-07-18T03:25:27.545 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 Processor -- start 2020-07-18T03:25:27.545 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 -- start start 2020-07-18T03:25:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 0x7fb1500afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb150080700 0x7fb150080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1501548a0 con 0x7fb150157420 2020-07-18T03:25:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb150154b40 con 0x7fb150080700 2020-07-18T03:25:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.542+0000 7fb155bde700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb150154de0 con 0x7fb1500625a0 2020-07-18T03:25:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 0x7fb1500afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 0x7fb1500afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:49960/0 (socket says 172.21.15.18:49960) 2020-07-18T03:25:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 -- 172.21.15.18:0/2263387244 learned_addr learned my addr 172.21.15.18:0/2263387244 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb147fff700 1 --2- 172.21.15.18:0/2263387244 >> v2:172.21.15.47:3300/0 conn(0x7fb150080700 0x7fb150080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.548 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 -- 172.21.15.18:0/2263387244 >> v2:172.21.15.47:3300/0 conn(0x7fb150080700 msgr2=0x7fb150080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:27.548 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 --2- 172.21.15.18:0/2263387244 >> v2:172.21.15.47:3300/0 conn(0x7fb150080700 0x7fb150080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.548 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 -- 172.21.15.18:0/2263387244 >> v1:172.21.15.47:6789/0 conn(0x7fb150157420 legacy=0x7fb1501578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.548 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 -- 172.21.15.18:0/2263387244 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1501545a0 con 0x7fb1500625a0 2020-07-18T03:25:27.548 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14ffff700 1 --2- 172.21.15.18:0/2263387244 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 0x7fb1500afa90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fb140002f30 tx=0x7fb140006e90).ready entity=mon.2 client_cookie=f7ca40789f781ca3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:27.549 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14f7fe700 1 -- 172.21.15.18:0/2263387244 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb140009850 con 0x7fb1500625a0 2020-07-18T03:25:27.549 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14f7fe700 1 -- 172.21.15.18:0/2263387244 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1400099b0 con 0x7fb1500625a0 2020-07-18T03:25:27.549 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb14f7fe700 1 -- 172.21.15.18:0/2263387244 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb140009d10 con 0x7fb1500625a0 2020-07-18T03:25:27.549 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 -- 172.21.15.18:0/2263387244 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 msgr2=0x7fb1500afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.549 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 --2- 172.21.15.18:0/2263387244 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 0x7fb1500afa90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fb140002f30 tx=0x7fb140006e90).stop 2020-07-18T03:25:27.552 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 -- 172.21.15.18:0/2263387244 shutdown_connections 2020-07-18T03:25:27.556 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 --2- 172.21.15.18:0/2263387244 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb1500625a0 0x7fb1500afa90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.556 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 --2- 172.21.15.18:0/2263387244 >> v2:172.21.15.47:3300/0 conn(0x7fb150080700 0x7fb150080b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.557 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 -- 172.21.15.18:0/2263387244 >> 172.21.15.18:0/2263387244 conn(0x7fb15005f5a0 msgr2=0x7fb15005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:27.557 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.543+0000 7fb155bde700 1 -- 172.21.15.18:0/2263387244 shutdown_connections 2020-07-18T03:25:27.557 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.544+0000 7fb155bde700 1 -- 172.21.15.18:0/2263387244 wait complete. 2020-07-18T03:25:27.559 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.545+0000 7fb155bde700 1 Processor -- start 2020-07-18T03:25:27.560 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.545+0000 7fb155bde700 1 -- start start 2020-07-18T03:25:27.560 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb155bde700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 0x7fb150120d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.560 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb155bde700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb150157420 0x7fb150121d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.560 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb155bde700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500b0d30 con 0x7fb150153430 2020-07-18T03:25:27.561 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb155bde700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500b0b20 con 0x7fb150080700 2020-07-18T03:25:27.561 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb155bde700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500b0fd0 con 0x7fb150157420 2020-07-18T03:25:27.561 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb14ffff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 0x7fb150120d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.561 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb14ffff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 0x7fb150120d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57978/0 (socket says 172.21.15.18:57978) 2020-07-18T03:25:27.561 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb14ffff700 1 -- 172.21.15.18:0/3195310336 learned_addr learned my addr 172.21.15.18:0/3195310336 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:27.562 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.546+0000 7fb147fff700 1 --2- 172.21.15.18:0/3195310336 >> v2:172.21.15.47:3300/0 conn(0x7fb150157420 0x7fb150121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.562 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb147fff700 1 -- 172.21.15.18:0/3195310336 >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 msgr2=0x7fb150120d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.568 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb147fff700 1 --2- 172.21.15.18:0/3195310336 >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 0x7fb150120d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.568 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb147fff700 1 -- 172.21.15.18:0/3195310336 >> v1:172.21.15.47:6789/0 conn(0x7fb150153430 legacy=0x7fb1500b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.568 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb147fff700 1 -- 172.21.15.18:0/3195310336 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150122230 con 0x7fb150157420 2020-07-18T03:25:27.568 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649324094 0 0) 0x7fb1500b0d30 con 0x7fb150153430 2020-07-18T03:25:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb14ffff700 1 --2- 172.21.15.18:0/3195310336 >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 0x7fb150120d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb147fff700 1 --2- 172.21.15.18:0/3195310336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb150157420 0x7fb150121d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb1500709f0 tx=0x7fb138008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13800c820 con 0x7fb150157420 2020-07-18T03:25:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800c980 con 0x7fb150157420 2020-07-18T03:25:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1500b2df0 con 0x7fb150157420 2020-07-18T03:25:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13800cce0 con 0x7fb150157420 2020-07-18T03:25:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1500b16b0 con 0x7fb150157420 2020-07-18T03:25:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7fb138022070 con 0x7fb150157420 2020-07-18T03:25:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.547+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fb13800ce40 con 0x7fb150157420 2020-07-18T03:25:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.548+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb150057450 con 0x7fb150157420 2020-07-18T03:25:27.571 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.550+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7fb138039a00 con 0x7fb150157420 2020-07-18T03:25:27.641 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.640+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", "id": 1} v 0) v1 -- 0x7fb1500b1140 con 0x7fb150157420 2020-07-18T03:25:27.644 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.644+0000 7fb1477fe700 1 -- 172.21.15.18:0/3195310336 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb138029030 con 0x7fb150157420 2020-07-18T03:25:27.645 INFO:teuthology.orchestra.run.smithi018.stdout:1 2020-07-18T03:25:27.646 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.644+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb150157420 msgr2=0x7fb150121d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.646 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.644+0000 7fb155bde700 1 --2- 172.21.15.18:0/3195310336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb150157420 0x7fb150121d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb1500709f0 tx=0x7fb138008040).stop 2020-07-18T03:25:27.646 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.645+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 shutdown_connections 2020-07-18T03:25:27.647 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.645+0000 7fb155bde700 1 --2- 172.21.15.18:0/3195310336 >> v2:172.21.15.18:3300/0 conn(0x7fb150080700 0x7fb150120d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.647 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.645+0000 7fb155bde700 1 --2- 172.21.15.18:0/3195310336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb150157420 0x7fb150121d40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.647 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.645+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 >> 172.21.15.18:0/3195310336 conn(0x7fb15005f5a0 msgr2=0x7fb1500b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:27.647 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.645+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 shutdown_connections 2020-07-18T03:25:27.647 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.645+0000 7fb155bde700 1 -- 172.21.15.18:0/3195310336 wait complete. 2020-07-18T03:25:27.656 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new dc0b1a97-efca-483d-839c-37c286699283 2 2020-07-18T03:25:27.774 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.770+0000 7f5fac4e3700 1 Processor -- start 2020-07-18T03:25:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.770+0000 7f5fac4e3700 1 -- start start 2020-07-18T03:25:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fac4e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fa4080700 0x7f5fa4080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fac4e3700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5fa40ae4b0 0x7f5fa40ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fac4e3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa41548a0 con 0x7f5fa40625a0 2020-07-18T03:25:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fac4e3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa4151960 con 0x7f5fa4080700 2020-07-18T03:25:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fac4e3700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4154b40 con 0x7f5fa40ae4b0 2020-07-18T03:25:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5faaa80700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f5fa40625a0 0x7f5fa40afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:39086/0 (socket says 172.21.15.18:39086) 2020-07-18T03:25:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5faaa80700 1 -- 172.21.15.18:0/2392405521 learned_addr learned my addr 172.21.15.18:0/2392405521 (peer_addr_for_me v1:172.21.15.18:0/0) 2020-07-18T03:25:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fab281700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5fa40ae4b0 0x7f5fa40ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5faa27f700 1 --2- 172.21.15.18:0/2392405521 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4080700 0x7f5fa4080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fa9a7e700 1 -- 172.21.15.18:0/2392405521 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909054164 0 0) 0x7f5fa41548a0 con 0x7f5fa40625a0 2020-07-18T03:25:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fa9a7e700 1 -- 172.21.15.18:0/2392405521 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f98003240 con 0x7f5fa40625a0 2020-07-18T03:25:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fab281700 1 -- 172.21.15.18:0/2392405521 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4080700 msgr2=0x7f5fa4080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fab281700 1 --2- 172.21.15.18:0/2392405521 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4080700 0x7f5fa4080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fab281700 1 -- 172.21.15.18:0/2392405521 >> v1:172.21.15.18:6789/0 conn(0x7f5fa40625a0 legacy=0x7f5fa40afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.771+0000 7f5fab281700 1 -- 172.21.15.18:0/2392405521 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4154de0 con 0x7f5fa40ae4b0 2020-07-18T03:25:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fab281700 1 --2- 172.21.15.18:0/2392405521 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5fa40ae4b0 0x7f5fa40ae920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f5f9c0032b0 tx=0x7f5f9c0091d0).ready entity=mon.2 client_cookie=bd30c78aa6389e6f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fa9a7e700 1 -- 172.21.15.18:0/2392405521 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f9c00b9a0 con 0x7f5fa40ae4b0 2020-07-18T03:25:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fa9a7e700 1 -- 172.21.15.18:0/2392405521 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00bb00 con 0x7f5fa40ae4b0 2020-07-18T03:25:27.786 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fa9a7e700 1 -- 172.21.15.18:0/2392405521 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f9c00bc70 con 0x7f5fa40ae4b0 2020-07-18T03:25:27.786 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 -- 172.21.15.18:0/2392405521 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5fa40ae4b0 msgr2=0x7f5fa40ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.786 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 --2- 172.21.15.18:0/2392405521 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5fa40ae4b0 0x7f5fa40ae920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f5f9c0032b0 tx=0x7f5f9c0091d0).stop 2020-07-18T03:25:27.786 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 -- 172.21.15.18:0/2392405521 shutdown_connections 2020-07-18T03:25:27.786 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 --2- 172.21.15.18:0/2392405521 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5fa40ae4b0 0x7f5fa40ae920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 --2- 172.21.15.18:0/2392405521 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4080700 0x7f5fa4080b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 -- 172.21.15.18:0/2392405521 >> 172.21.15.18:0/2392405521 conn(0x7f5fa405f5a0 msgr2=0x7f5fa405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 -- 172.21.15.18:0/2392405521 shutdown_connections 2020-07-18T03:25:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.772+0000 7f5fac4e3700 1 -- 172.21.15.18:0/2392405521 wait complete. 2020-07-18T03:25:27.790 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.774+0000 7f5fac4e3700 1 Processor -- start 2020-07-18T03:25:27.790 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.774+0000 7f5fac4e3700 1 -- start start 2020-07-18T03:25:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5fa4080700 0x7f5fa4066360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fa4150af0 0x7f5fa4150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa4157250 con 0x7f5fa40625a0 2020-07-18T03:25:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa4157480 con 0x7f5fa4080700 2020-07-18T03:25:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa4157720 con 0x7f5fa4150af0 2020-07-18T03:25:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faaa80700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f5fa40625a0 0x7f5fa41560f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:39092/0 (socket says 172.21.15.18:39092) 2020-07-18T03:25:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faaa80700 1 -- 172.21.15.18:0/487788202 learned_addr learned my addr 172.21.15.18:0/487788202 (peer_addr_for_me v1:172.21.15.18:0/0) 2020-07-18T03:25:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faa27f700 1 --2- 172.21.15.18:0/487788202 >> v2:172.21.15.18:3300/0 conn(0x7f5fa4080700 0x7f5fa4066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fab281700 1 --2- 172.21.15.18:0/487788202 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4150af0 0x7f5fa4150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748436876 0 0) 0x7f5fa4157250 con 0x7f5fa40625a0 2020-07-18T03:25:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f80003240 con 0x7f5fa40625a0 2020-07-18T03:25:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faa27f700 1 -- 172.21.15.18:0/487788202 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4150af0 msgr2=0x7f5fa4150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faa27f700 1 --2- 172.21.15.18:0/487788202 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4150af0 0x7f5fa4150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faa27f700 1 -- 172.21.15.18:0/487788202 >> v1:172.21.15.18:6789/0 conn(0x7f5fa40625a0 legacy=0x7f5fa41560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faa27f700 1 -- 172.21.15.18:0/487788202 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4151450 con 0x7f5fa4080700 2020-07-18T03:25:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fab281700 1 --2- 172.21.15.18:0/487788202 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4150af0 0x7f5fa4150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5faa27f700 1 --2- 172.21.15.18:0/487788202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fa4080700 0x7f5fa4066360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f5f9400a000 tx=0x7f5f94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f940117d0 con 0x7f5fa4080700 2020-07-18T03:25:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f94011930 con 0x7f5fa4080700 2020-07-18T03:25:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa41579c0 con 0x7f5fa4080700 2020-07-18T03:25:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f94011c90 con 0x7f5fa4080700 2020-07-18T03:25:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.775+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa4064480 con 0x7f5fa4080700 2020-07-18T03:25:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.776+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 2) v1 ==== 31010+0+0 (secure 0 0 0) 0x7f5f94012420 con 0x7f5fa4080700 2020-07-18T03:25:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.776+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f5f940385b0 con 0x7f5fa4080700 2020-07-18T03:25:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.776+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa4057450 con 0x7f5fa4080700 2020-07-18T03:25:27.796 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.778+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7f5f94011e40 con 0x7f5fa4080700 2020-07-18T03:25:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.867+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dc0b1a97-efca-483d-839c-37c286699283", "id": 2} v 0) v1 -- 0x7f5fa4068c50 con 0x7f5fa4080700 2020-07-18T03:25:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.869+0000 7f5f92ffd700 1 -- 172.21.15.18:0/487788202 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc0b1a97-efca-483d-839c-37c286699283", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f94011e40 con 0x7f5fa4080700 2020-07-18T03:25:27.870 INFO:teuthology.orchestra.run.smithi018.stdout:2 2020-07-18T03:25:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fa4080700 msgr2=0x7f5fa4066360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 --2- 172.21.15.18:0/487788202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fa4080700 0x7f5fa4066360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f5f9400a000 tx=0x7f5f94008040).stop 2020-07-18T03:25:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 shutdown_connections 2020-07-18T03:25:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 --2- 172.21.15.18:0/487788202 >> v2:172.21.15.47:3300/0 conn(0x7f5fa4150af0 0x7f5fa4150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 --2- 172.21.15.18:0/487788202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fa4080700 0x7f5fa4066360 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 >> 172.21.15.18:0/487788202 conn(0x7f5fa405f5a0 msgr2=0x7f5fa40ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 shutdown_connections 2020-07-18T03:25:27.873 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.870+0000 7f5fac4e3700 1 -- 172.21.15.18:0/487788202 wait complete. 2020-07-18T03:25:27.881 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 3f949e90-b2a0-4ef4-a63b-90f457a32c86 3 2020-07-18T03:25:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.992+0000 7ff085bf9700 1 Processor -- start 2020-07-18T03:25:27.996 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.992+0000 7ff085bf9700 1 -- start start 2020-07-18T03:25:27.996 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff085bf9700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff0800625a0 0x7ff0800afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.996 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff085bf9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff080080700 0x7ff080080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:27.997 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff085bf9700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0801548a0 con 0x7ff080157420 2020-07-18T03:25:27.997 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff085bf9700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff080154b40 con 0x7ff0800625a0 2020-07-18T03:25:27.997 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff085bf9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff080154de0 con 0x7ff080080700 2020-07-18T03:25:27.997 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07ffff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff0800625a0 0x7ff0800afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.998 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff084997700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ff080157420 0x7ff080157890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:39102/0 (socket says 172.21.15.18:39102) 2020-07-18T03:25:27.998 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff084997700 1 -- 172.21.15.18:0/3716343940 learned_addr learned my addr 172.21.15.18:0/3716343940 (peer_addr_for_me v1:172.21.15.18:0/0) 2020-07-18T03:25:27.998 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07f7fe700 1 --2- 172.21.15.18:0/3716343940 >> v2:172.21.15.47:3300/0 conn(0x7ff080080700 0x7ff080080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:27.998 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07ffff700 1 -- 172.21.15.18:0/3716343940 >> v2:172.21.15.47:3300/0 conn(0x7ff080080700 msgr2=0x7ff080080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:27.999 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07ffff700 1 --2- 172.21.15.18:0/3716343940 >> v2:172.21.15.47:3300/0 conn(0x7ff080080700 0x7ff080080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:27.999 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07ffff700 1 -- 172.21.15.18:0/3716343940 >> v1:172.21.15.18:6789/0 conn(0x7ff080157420 legacy=0x7ff080157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:27.999 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07effd700 1 -- 172.21.15.18:0/3716343940 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329770761 0 0) 0x7ff0801548a0 con 0x7ff080157420 2020-07-18T03:25:27.999 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.993+0000 7ff07ffff700 1 -- 172.21.15.18:0/3716343940 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0801545a0 con 0x7ff0800625a0 2020-07-18T03:25:27.999 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff07ffff700 1 --2- 172.21.15.18:0/3716343940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0800625a0 0x7ff0800afa90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff070002f30 tx=0x7ff070006dd0).ready entity=mon.0 client_cookie=76e43489a4db9017 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.000 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff07effd700 1 -- 172.21.15.18:0/3716343940 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0700097b0 con 0x7ff0800625a0 2020-07-18T03:25:28.000 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff07effd700 1 -- 172.21.15.18:0/3716343940 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070009910 con 0x7ff0800625a0 2020-07-18T03:25:28.000 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff07effd700 1 -- 172.21.15.18:0/3716343940 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff070009c70 con 0x7ff0800625a0 2020-07-18T03:25:28.000 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 -- 172.21.15.18:0/3716343940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0800625a0 msgr2=0x7ff0800afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.000 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 --2- 172.21.15.18:0/3716343940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0800625a0 0x7ff0800afa90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff070002f30 tx=0x7ff070006dd0).stop 2020-07-18T03:25:28.001 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 -- 172.21.15.18:0/3716343940 shutdown_connections 2020-07-18T03:25:28.001 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 --2- 172.21.15.18:0/3716343940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0800625a0 0x7ff0800afa90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.001 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 --2- 172.21.15.18:0/3716343940 >> v2:172.21.15.47:3300/0 conn(0x7ff080080700 0x7ff080080b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.001 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 -- 172.21.15.18:0/3716343940 >> 172.21.15.18:0/3716343940 conn(0x7ff08005f5a0 msgr2=0x7ff08005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.001 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 -- 172.21.15.18:0/3716343940 shutdown_connections 2020-07-18T03:25:28.002 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.994+0000 7ff085bf9700 1 -- 172.21.15.18:0/3716343940 wait complete. 2020-07-18T03:25:28.004 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 Processor -- start 2020-07-18T03:25:28.005 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 -- start start 2020-07-18T03:25:28.005 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 0x7ff080138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.005 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff080157420 0x7ff0800b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.005 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0800b31e0 con 0x7ff080152b80 2020-07-18T03:25:28.005 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff080124670 con 0x7ff080157420 2020-07-18T03:25:28.006 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff085bf9700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800b3480 con 0x7ff080080700 2020-07-18T03:25:28.006 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 0x7ff080138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.006 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff084997700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ff080152b80 0x7ff0800b1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:39106/0 (socket says 172.21.15.18:39106) 2020-07-18T03:25:28.006 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 0x7ff080138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:49990/0 (socket says 172.21.15.18:49990) 2020-07-18T03:25:28.006 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff084997700 1 -- 172.21.15.18:0/223328349 learned_addr learned my addr 172.21.15.18:0/223328349 (peer_addr_for_me v1:172.21.15.18:0/0) 2020-07-18T03:25:28.007 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.996+0000 7ff07f7fe700 1 --2- 172.21.15.18:0/223328349 >> v2:172.21.15.47:3300/0 conn(0x7ff080157420 0x7ff0800b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.007 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729564407 0 0) 0x7ff0800b31e0 con 0x7ff080152b80 2020-07-18T03:25:28.007 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff058003240 con 0x7ff080152b80 2020-07-18T03:25:28.007 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff07ffff700 1 -- 172.21.15.18:0/223328349 >> v2:172.21.15.47:3300/0 conn(0x7ff080157420 msgr2=0x7ff0800b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.007 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff07ffff700 1 --2- 172.21.15.18:0/223328349 >> v2:172.21.15.47:3300/0 conn(0x7ff080157420 0x7ff0800b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.008 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff07ffff700 1 -- 172.21.15.18:0/223328349 >> v1:172.21.15.18:6789/0 conn(0x7ff080152b80 legacy=0x7ff0800b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.008 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff07ffff700 1 -- 172.21.15.18:0/223328349 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800b2ed0 con 0x7ff080080700 2020-07-18T03:25:28.008 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff07f7fe700 1 --2- 172.21.15.18:0/223328349 >> v2:172.21.15.47:3300/0 conn(0x7ff080157420 0x7ff0800b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:28.008 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff07ffff700 1 --2- 172.21.15.18:0/223328349 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 0x7ff080138260 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ff070002f30 tx=0x7ff070015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.008 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff070021070 con 0x7ff080080700 2020-07-18T03:25:28.009 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0700106c0 con 0x7ff080080700 2020-07-18T03:25:28.009 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080063ef0 con 0x7ff080080700 2020-07-18T03:25:28.009 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff070010a20 con 0x7ff080080700 2020-07-18T03:25:28.009 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.997+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080153f00 con 0x7ff080080700 2020-07-18T03:25:28.009 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.998+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 2) v1 ==== 31010+0+0 (secure 0 0 0) 0x7ff07000f070 con 0x7ff080080700 2020-07-18T03:25:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.998+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1475+0+0 (secure 0 0 0) 0x7ff070039dd0 con 0x7ff080080700 2020-07-18T03:25:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:27.998+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff080057450 con 0x7ff080080700 2020-07-18T03:25:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.000+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7ff07001b7b0 con 0x7ff080080700 2020-07-18T03:25:28.090 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.089+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", "id": 3} v 0) v1 -- 0x7ff080068c50 con 0x7ff080080700 2020-07-18T03:25:28.094 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.093+0000 7ff05f7fe700 1 -- 172.21.15.18:0/223328349 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff07001b7b0 con 0x7ff080080700 2020-07-18T03:25:28.094 INFO:teuthology.orchestra.run.smithi018.stdout:3 2020-07-18T03:25:28.095 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.093+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 msgr2=0x7ff080138260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.095 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.093+0000 7ff085bf9700 1 --2- 172.21.15.18:0/223328349 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 0x7ff080138260 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ff070002f30 tx=0x7ff070015000).stop 2020-07-18T03:25:28.096 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.094+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 shutdown_connections 2020-07-18T03:25:28.096 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.094+0000 7ff085bf9700 1 --2- 172.21.15.18:0/223328349 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff080080700 0x7ff080138260 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.096 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.094+0000 7ff085bf9700 1 --2- 172.21.15.18:0/223328349 >> v2:172.21.15.47:3300/0 conn(0x7ff080157420 0x7ff0800b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.096 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.094+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 >> 172.21.15.18:0/223328349 conn(0x7ff08005f5a0 msgr2=0x7ff08005f980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.096 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.094+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 shutdown_connections 2020-07-18T03:25:28.097 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.094+0000 7ff085bf9700 1 -- 172.21.15.18:0/223328349 wait complete. 2020-07-18T03:25:28.105 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 83234bc7-02d5-405a-af0d-2c30c2f80681 4 2020-07-18T03:25:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 Processor -- start 2020-07-18T03:25:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 -- start start 2020-07-18T03:25:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbca80625a0 0x7fbca80afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbca8080b80 0x7fbca8157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca81548a0 con 0x7fbca8080700 2020-07-18T03:25:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca8151960 con 0x7fbca8080b80 2020-07-18T03:25:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.219+0000 7fbcafa25700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8154b40 con 0x7fbca80625a0 2020-07-18T03:25:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcadfc2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbca80625a0 0x7fbca80afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcae7c3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbca8080b80 0x7fbca8157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcadfc2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbca80625a0 0x7fbca80afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:49996/0 (socket says 172.21.15.18:49996) 2020-07-18T03:25:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcadfc2700 1 -- 172.21.15.18:0/4079421150 learned_addr learned my addr 172.21.15.18:0/4079421150 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcae7c3700 1 -- 172.21.15.18:0/4079421150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbca80625a0 msgr2=0x7fbca80afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcae7c3700 1 --2- 172.21.15.18:0/4079421150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbca80625a0 0x7fbca80afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcae7c3700 1 -- 172.21.15.18:0/4079421150 >> v1:172.21.15.18:6789/0 conn(0x7fbca8080700 legacy=0x7fbca8157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcae7c3700 1 -- 172.21.15.18:0/4079421150 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8154de0 con 0x7fbca8080b80 2020-07-18T03:25:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcae7c3700 1 --2- 172.21.15.18:0/4079421150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8080b80 0x7fbca8157b30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbca80ae390 tx=0x7fbca00091d0).ready entity=mon.1 client_cookie=b0f4b678e01f6cae server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcacfc0700 1 -- 172.21.15.18:0/4079421150 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca000b960 con 0x7fbca8080b80 2020-07-18T03:25:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcacfc0700 1 -- 172.21.15.18:0/4079421150 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000bac0 con 0x7fbca8080b80 2020-07-18T03:25:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcacfc0700 1 -- 172.21.15.18:0/4079421150 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca000bc30 con 0x7fbca8080b80 2020-07-18T03:25:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcafa25700 1 -- 172.21.15.18:0/4079421150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8080b80 msgr2=0x7fbca8157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcafa25700 1 --2- 172.21.15.18:0/4079421150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8080b80 0x7fbca8157b30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbca80ae390 tx=0x7fbca00091d0).stop 2020-07-18T03:25:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcafa25700 1 -- 172.21.15.18:0/4079421150 shutdown_connections 2020-07-18T03:25:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcafa25700 1 --2- 172.21.15.18:0/4079421150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8080b80 0x7fbca8157b30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcafa25700 1 --2- 172.21.15.18:0/4079421150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbca80625a0 0x7fbca80afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.220+0000 7fbcafa25700 1 -- 172.21.15.18:0/4079421150 >> 172.21.15.18:0/4079421150 conn(0x7fbca805f5a0 msgr2=0x7fbca805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.221+0000 7fbcafa25700 1 -- 172.21.15.18:0/4079421150 shutdown_connections 2020-07-18T03:25:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.221+0000 7fbcafa25700 1 -- 172.21.15.18:0/4079421150 wait complete. 2020-07-18T03:25:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.222+0000 7fbcafa25700 1 Processor -- start 2020-07-18T03:25:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.222+0000 7fbcafa25700 1 -- start start 2020-07-18T03:25:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcafa25700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbca80625a0 0x7fbca8136d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcafa25700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbca815c390 0x7fbca8137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcafa25700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80b2b70 con 0x7fbca8080700 2020-07-18T03:25:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcafa25700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca80b2e10 con 0x7fbca815c390 2020-07-18T03:25:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcafa25700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca8137770 con 0x7fbca80625a0 2020-07-18T03:25:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcae7c3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbca815c390 0x7fbca8137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcae7c3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbca815c390 0x7fbca8137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58016/0 (socket says 172.21.15.18:58016) 2020-07-18T03:25:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcae7c3700 1 -- 172.21.15.18:0/843019804 learned_addr learned my addr 172.21.15.18:0/843019804 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.223+0000 7fbcadfc2700 1 --2- 172.21.15.18:0/843019804 >> v2:172.21.15.47:3300/0 conn(0x7fbca80625a0 0x7fbca8136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbcae7c3700 1 -- 172.21.15.18:0/843019804 >> v2:172.21.15.47:3300/0 conn(0x7fbca80625a0 msgr2=0x7fbca8136d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbcae7c3700 1 --2- 172.21.15.18:0/843019804 >> v2:172.21.15.47:3300/0 conn(0x7fbca80625a0 0x7fbca8136d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbcae7c3700 1 -- 172.21.15.18:0/843019804 >> v1:172.21.15.47:6789/0 conn(0x7fbca8080700 legacy=0x7fbca813b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023748910 0 0) 0x7fbca80b2b70 con 0x7fbca8080700 2020-07-18T03:25:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbcae7c3700 1 -- 172.21.15.18:0/843019804 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8152b90 con 0x7fbca815c390 2020-07-18T03:25:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbcadfc2700 1 --2- 172.21.15.18:0/843019804 >> v2:172.21.15.47:3300/0 conn(0x7fbca80625a0 0x7fbca8136d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.224+0000 7fbcae7c3700 1 --2- 172.21.15.18:0/843019804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbca815c390 0x7fbca8137280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbca80ae390 tx=0x7fbca0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.277 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.276+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca0010870 con 0x7fbca815c390 2020-07-18T03:25:28.277 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.276+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca00109d0 con 0x7fbca815c390 2020-07-18T03:25:28.277 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.276+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca0010d30 con 0x7fbca815c390 2020-07-18T03:25:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.276+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8152d20 con 0x7fbca815c390 2020-07-18T03:25:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.276+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca813b980 con 0x7fbca815c390 2020-07-18T03:25:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.276+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 2) v1 ==== 31010+0+0 (secure 0 0 0) 0x7fbca0027030 con 0x7fbca815c390 2020-07-18T03:25:28.279 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.277+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1614+0+0 (secure 0 0 0) 0x7fbca0039b50 con 0x7fbca815c390 2020-07-18T03:25:28.279 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.277+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80adad0 con 0x7fbca815c390 2020-07-18T03:25:28.279 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.279+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7fbca0015080 con 0x7fbca815c390 2020-07-18T03:25:28.368 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.367+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", "id": 4} v 0) v1 -- 0x7fbca8057450 con 0x7fbca815c390 2020-07-18T03:25:28.371 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.370+0000 7fbc997fa700 1 -- 172.21.15.18:0/843019804 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbca0025070 con 0x7fbca815c390 2020-07-18T03:25:28.371 INFO:teuthology.orchestra.run.smithi018.stdout:4 2020-07-18T03:25:28.372 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbca815c390 msgr2=0x7fbca8137280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.373 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 --2- 172.21.15.18:0/843019804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbca815c390 0x7fbca8137280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbca80ae390 tx=0x7fbca0016000).stop 2020-07-18T03:25:28.373 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 shutdown_connections 2020-07-18T03:25:28.373 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 --2- 172.21.15.18:0/843019804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbca815c390 0x7fbca8137280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.373 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 --2- 172.21.15.18:0/843019804 >> v2:172.21.15.47:3300/0 conn(0x7fbca80625a0 0x7fbca8136d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.374 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 >> 172.21.15.18:0/843019804 conn(0x7fbca805f5a0 msgr2=0x7fbca805d410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.374 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 shutdown_connections 2020-07-18T03:25:28.374 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.371+0000 7fbcafa25700 1 -- 172.21.15.18:0/843019804 wait complete. 2020-07-18T03:25:28.382 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new a8bc62ad-b199-4f39-bbd5-10a71cd953c8 5 2020-07-18T03:25:28.499 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.495+0000 7f784234c700 1 Processor -- start 2020-07-18T03:25:28.499 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.495+0000 7f784234c700 1 -- start start 2020-07-18T03:25:28.499 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f784234c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f783c0625a0 0x7f783c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.499 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f784234c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f783c080700 0x7f783c080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.500 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f784234c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f783c157420 0x7f783c157890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.500 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f784234c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c1548a0 con 0x7f783c080700 2020-07-18T03:25:28.500 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f784234c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c154b40 con 0x7f783c0625a0 2020-07-18T03:25:28.500 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f784234c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c154de0 con 0x7f783c157420 2020-07-18T03:25:28.500 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f783c080700 0x7f783c080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.501 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f78410ea700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f783c157420 0x7f783c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.501 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f783c080700 0x7f783c080b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58018/0 (socket says 172.21.15.18:58018) 2020-07-18T03:25:28.501 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f78410ea700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f783c157420 0x7f783c157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50010/0 (socket says 172.21.15.18:50010) 2020-07-18T03:25:28.501 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 -- 172.21.15.18:0/2636471639 learned_addr learned my addr 172.21.15.18:0/2636471639 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:28.502 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f78408e9700 1 --2- 172.21.15.18:0/2636471639 >> v2:172.21.15.47:3300/0 conn(0x7f783c0625a0 0x7f783c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.502 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 -- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f783c157420 msgr2=0x7f783c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:28.502 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 --2- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f783c157420 0x7f783c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.502 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 -- 172.21.15.18:0/2636471639 >> v2:172.21.15.47:3300/0 conn(0x7f783c0625a0 msgr2=0x7f783c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:28.502 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 --2- 172.21.15.18:0/2636471639 >> v2:172.21.15.47:3300/0 conn(0x7f783c0625a0 0x7f783c0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.503 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 -- 172.21.15.18:0/2636471639 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c1545a0 con 0x7f783c080700 2020-07-18T03:25:28.503 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.496+0000 7f783bfff700 1 --2- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f783c080700 0x7f783c080b80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f782c002f30 tx=0x7f782c006e90).ready entity=mon.0 client_cookie=b8a73a1af3156767 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.503 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f783b7fe700 1 -- 172.21.15.18:0/2636471639 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c009850 con 0x7f783c080700 2020-07-18T03:25:28.503 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f783b7fe700 1 -- 172.21.15.18:0/2636471639 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c0099b0 con 0x7f783c080700 2020-07-18T03:25:28.503 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f783b7fe700 1 -- 172.21.15.18:0/2636471639 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c009d10 con 0x7f783c080700 2020-07-18T03:25:28.504 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 -- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f783c080700 msgr2=0x7f783c080b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.504 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 --2- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f783c080700 0x7f783c080b80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f782c002f30 tx=0x7f782c006e90).stop 2020-07-18T03:25:28.504 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 -- 172.21.15.18:0/2636471639 shutdown_connections 2020-07-18T03:25:28.504 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 --2- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f783c157420 0x7f783c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.504 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 --2- 172.21.15.18:0/2636471639 >> v2:172.21.15.47:3300/0 conn(0x7f783c0625a0 0x7f783c0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.505 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 --2- 172.21.15.18:0/2636471639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f783c080700 0x7f783c080b80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.505 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 -- 172.21.15.18:0/2636471639 >> 172.21.15.18:0/2636471639 conn(0x7f783c05f5a0 msgr2=0x7f783c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.505 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 -- 172.21.15.18:0/2636471639 shutdown_connections 2020-07-18T03:25:28.505 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.497+0000 7f784234c700 1 -- 172.21.15.18:0/2636471639 wait complete. 2020-07-18T03:25:28.508 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.498+0000 7f784234c700 1 Processor -- start 2020-07-18T03:25:28.508 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f784234c700 1 -- start start 2020-07-18T03:25:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f784234c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f783c0639d0 0x7f783c1487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f784234c700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c063670 con 0x7f783c157420 2020-07-18T03:25:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f784234c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0633d0 con 0x7f783c0625a0 2020-07-18T03:25:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f784234c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c124400 con 0x7f783c0639d0 2020-07-18T03:25:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f783bfff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f783c157420 0x7f783c11f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:39128/0 (socket says 172.21.15.18:39128) 2020-07-18T03:25:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f783bfff700 1 -- 172.21.15.18:0/3722559159 learned_addr learned my addr 172.21.15.18:0/3722559159 (peer_addr_for_me v1:172.21.15.18:0/0) 2020-07-18T03:25:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.499+0000 7f78410ea700 1 --2- 172.21.15.18:0/3722559159 >> v2:172.21.15.47:3300/0 conn(0x7f783c0639d0 0x7f783c1487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940318531 0 0) 0x7f783c063670 con 0x7f783c157420 2020-07-18T03:25:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7818003240 con 0x7f783c157420 2020-07-18T03:25:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906572609 0 0) 0x7f783c0633d0 con 0x7f783c0625a0 2020-07-18T03:25:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f783c063670 con 0x7f783c0625a0 2020-07-18T03:25:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3503959307 0 0) 0x7f7818003240 con 0x7f783c157420 2020-07-18T03:25:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f78410ea700 1 -- 172.21.15.18:0/3722559159 >> v1:172.21.15.47:6789/0 conn(0x7f783c0625a0 legacy=0x7f783c0b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f78410ea700 1 -- 172.21.15.18:0/3722559159 >> v1:172.21.15.18:6789/0 conn(0x7f783c157420 legacy=0x7f783c11f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f78410ea700 1 -- 172.21.15.18:0/3722559159 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c0665a0 con 0x7f783c0639d0 2020-07-18T03:25:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f78410ea700 1 --2- 172.21.15.18:0/3722559159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0639d0 0x7f783c1487f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f782c00b870 tx=0x7f78340091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c009850 con 0x7f783c0639d0 2020-07-18T03:25:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783400c870 con 0x7f783c0639d0 2020-07-18T03:25:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c066060 con 0x7f783c0639d0 2020-07-18T03:25:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f783400c9e0 con 0x7f783c0639d0 2020-07-18T03:25:28.513 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.500+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c0b1db0 con 0x7f783c0639d0 2020-07-18T03:25:28.513 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.501+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 2) v1 ==== 31010+0+0 (secure 0 0 0) 0x7f7834022070 con 0x7f783c0639d0 2020-07-18T03:25:28.513 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.501+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1729+0+0 (secure 0 0 0) 0x7f7834038900 con 0x7f783c0639d0 2020-07-18T03:25:28.513 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.501+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c057450 con 0x7f783c0639d0 2020-07-18T03:25:28.513 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.503+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68080 (secure 0 0 0) 0x7f7834016070 con 0x7f783c0639d0 2020-07-18T03:25:28.593 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.592+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", "id": 5} v 0) v1 -- 0x7f783c124570 con 0x7f783c0639d0 2020-07-18T03:25:28.598 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.596+0000 7f7833fff700 1 -- 172.21.15.18:0/3722559159 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7834016070 con 0x7f783c0639d0 2020-07-18T03:25:28.599 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0639d0 msgr2=0x7f783c1487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.602 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 --2- 172.21.15.18:0/3722559159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0639d0 0x7f783c1487f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f782c00b870 tx=0x7f78340091d0).stop 2020-07-18T03:25:28.602 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 shutdown_connections 2020-07-18T03:25:28.602 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 --2- 172.21.15.18:0/3722559159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0639d0 0x7f783c1487f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.602 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 >> 172.21.15.18:0/3722559159 conn(0x7f783c05f5a0 msgr2=0x7f783c153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.603 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 shutdown_connections 2020-07-18T03:25:28.603 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.597+0000 7f784234c700 1 -- 172.21.15.18:0/3722559159 wait complete. 2020-07-18T03:25:28.603 INFO:teuthology.orchestra.run.smithi018.stdout:5 2020-07-18T03:25:28.611 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new 3561e1c4-c30d-462b-a364-ae0be63149c9 6 2020-07-18T03:25:28.729 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.725+0000 7f9b2232f700 1 Processor -- start 2020-07-18T03:25:28.729 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.725+0000 7f9b2232f700 1 -- start start 2020-07-18T03:25:28.729 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b2232f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9b1c0625a0 0x7f9b1c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.730 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b2232f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.730 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b2232f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c1548a0 con 0x7f9b1c080700 2020-07-18T03:25:28.730 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b2232f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c151960 con 0x7f9b1c0625a0 2020-07-18T03:25:28.730 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b2232f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c154b40 con 0x7f9b1c080b80 2020-07-18T03:25:28.730 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9b1c0625a0 0x7f9b1c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.731 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9b1c0625a0 0x7f9b1c0afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58036/0 (socket says 172.21.15.18:58036) 2020-07-18T03:25:28.731 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 -- 172.21.15.18:0/152552078 learned_addr learned my addr 172.21.15.18:0/152552078 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:28.731 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b210cd700 1 --2- 172.21.15.18:0/152552078 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.731 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 -- 172.21.15.18:0/152552078 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 msgr2=0x7f9b1c157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:28.731 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 --2- 172.21.15.18:0/152552078 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.732 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 -- 172.21.15.18:0/152552078 >> v1:172.21.15.47:6789/0 conn(0x7f9b1c080700 legacy=0x7f9b1c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.732 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 -- 172.21.15.18:0/152552078 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c154de0 con 0x7f9b1c0625a0 2020-07-18T03:25:28.732 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b208cc700 1 --2- 172.21.15.18:0/152552078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b1c0625a0 0x7f9b1c0afa90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9b0c002f30 tx=0x7f9b0c006e90).ready entity=mon.0 client_cookie=d630e50aef5a4260 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.732 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.726+0000 7f9b1b7fe700 1 -- 172.21.15.18:0/152552078 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b0c0099c0 con 0x7f9b1c0625a0 2020-07-18T03:25:28.732 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b1b7fe700 1 -- 172.21.15.18:0/152552078 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c009b20 con 0x7f9b1c0625a0 2020-07-18T03:25:28.733 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b1b7fe700 1 -- 172.21.15.18:0/152552078 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b0c009e80 con 0x7f9b1c0625a0 2020-07-18T03:25:28.733 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 -- 172.21.15.18:0/152552078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b1c0625a0 msgr2=0x7f9b1c0afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.733 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 --2- 172.21.15.18:0/152552078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b1c0625a0 0x7f9b1c0afa90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9b0c002f30 tx=0x7f9b0c006e90).stop 2020-07-18T03:25:28.733 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 -- 172.21.15.18:0/152552078 shutdown_connections 2020-07-18T03:25:28.733 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 --2- 172.21.15.18:0/152552078 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.734 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 --2- 172.21.15.18:0/152552078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b1c0625a0 0x7f9b1c0afa90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.734 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 -- 172.21.15.18:0/152552078 >> 172.21.15.18:0/152552078 conn(0x7f9b1c05f5a0 msgr2=0x7f9b1c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.734 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 -- 172.21.15.18:0/152552078 shutdown_connections 2020-07-18T03:25:28.734 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.727+0000 7f9b2232f700 1 -- 172.21.15.18:0/152552078 wait complete. 2020-07-18T03:25:28.737 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.728+0000 7f9b2232f700 1 Processor -- start 2020-07-18T03:25:28.737 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b2232f700 1 -- start start 2020-07-18T03:25:28.737 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b2232f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.738 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b2232f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 0x7f9b1c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.738 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b2232f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c151aa0 con 0x7f9b1c080700 2020-07-18T03:25:28.738 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b2232f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c151d40 con 0x7f9b1c080b80 2020-07-18T03:25:28.738 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b2232f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c151fe0 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.738 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b210cd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 0x7f9b1c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.739 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b210cd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 0x7f9b1c0b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50034/0 (socket says 172.21.15.18:50034) 2020-07-18T03:25:28.739 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b210cd700 1 -- 172.21.15.18:0/3301758682 learned_addr learned my addr 172.21.15.18:0/3301758682 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:28.739 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.729+0000 7f9b1bfff700 1 --2- 172.21.15.18:0/3301758682 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.739 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b210cd700 1 -- 172.21.15.18:0/3301758682 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 msgr2=0x7f9b1c070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.739 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082531608 0 0) 0x7f9b1c151aa0 con 0x7f9b1c080700 2020-07-18T03:25:28.739 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b210cd700 1 --2- 172.21.15.18:0/3301758682 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.740 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b210cd700 1 -- 172.21.15.18:0/3301758682 >> v1:172.21.15.47:6789/0 conn(0x7f9b1c080700 legacy=0x7f9b1c147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.740 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b210cd700 1 -- 172.21.15.18:0/3301758682 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c152280 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.740 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b210cd700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 0x7f9b1c0b0610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9b1400a000 tx=0x7f9b14008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.740 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1400b9e0 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.740 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1400bb40 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.741 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c071680 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.741 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1400bea0 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.741 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.730+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c071bd0 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.741 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.731+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c152410 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.741 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.731+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 3) v1 ==== 61955+0+0 (secure 0 0 0) 0x7f9b14013510 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.742 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.731+0000 7f9afbfff700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9af404fe40 0x7f9af4052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:28.742 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.731+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1844+0+0 (secure 0 0 0) 0x7f9b14067400 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.742 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.732+0000 7f9b208cc700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9af404fe40 0x7f9af4052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:28.742 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.732+0000 7f9b208cc700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9af404fe40 0x7f9af4052300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f9b0c003070 tx=0x7f9b0c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:28.742 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.735+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9b14040df0 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.910 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.909+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", "id": 6} v 0) v1 -- 0x7f9b1c057450 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.913 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.912+0000 7f9afbfff700 1 -- 172.21.15.18:0/3301758682 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b14044410 con 0x7f9b1c0b2c40 2020-07-18T03:25:28.913 INFO:teuthology.orchestra.run.smithi018.stdout:6 2020-07-18T03:25:28.915 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9af404fe40 msgr2=0x7f9af4052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.915 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9af404fe40 0x7f9af4052300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f9b0c003070 tx=0x7f9b0c015000).stop 2020-07-18T03:25:28.915 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 msgr2=0x7f9b1c0b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:28.916 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 0x7f9b1c0b0610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9b1400a000 tx=0x7f9b14008040).stop 2020-07-18T03:25:28.916 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 shutdown_connections 2020-07-18T03:25:28.916 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9b1c0b2c40 0x7f9b1c0b0610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.916 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 --2- 172.21.15.18:0/3301758682 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9af404fe40 0x7f9af4052300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.916 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 --2- 172.21.15.18:0/3301758682 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c080b80 0x7f9b1c070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:28.917 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 >> 172.21.15.18:0/3301758682 conn(0x7f9b1c05f5a0 msgr2=0x7f9b1c0b9390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:28.917 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 shutdown_connections 2020-07-18T03:25:28.917 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:28.914+0000 7f9b2232f700 1 -- 172.21.15.18:0/3301758682 wait complete. 2020-07-18T03:25:28.925 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd new cb66e979-d9f3-40eb-946a-20ccb8c8002b 7 2020-07-18T03:25:29.040 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.037+0000 7efe6ff13700 1 Processor -- start 2020-07-18T03:25:29.041 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.037+0000 7efe6ff13700 1 -- start start 2020-07-18T03:25:29.041 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe680625a0 0x7efe680afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:29.041 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe68080700 0x7efe68080b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:29.042 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 0x7efe68157890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:29.042 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe681548a0 con 0x7efe680625a0 2020-07-18T03:25:29.042 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe68154b40 con 0x7efe68080700 2020-07-18T03:25:29.042 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe68154de0 con 0x7efe68157420 2020-07-18T03:25:29.042 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe680625a0 0x7efe680afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:29.043 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ecb1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 0x7efe68157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:29.043 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe680625a0 0x7efe680afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58050/0 (socket says 172.21.15.18:58050) 2020-07-18T03:25:29.043 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ecb1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 0x7efe68157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.18:50038/0 (socket says 172.21.15.18:50038) 2020-07-18T03:25:29.043 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 -- 172.21.15.18:0/26679018 learned_addr learned my addr 172.21.15.18:0/26679018 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:29.043 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6dcaf700 1 --2- 172.21.15.18:0/26679018 >> v2:172.21.15.47:3300/0 conn(0x7efe68080700 0x7efe68080b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:29.044 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 -- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 msgr2=0x7efe68157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:29.044 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 --2- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 0x7efe68157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.044 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 -- 172.21.15.18:0/26679018 >> v2:172.21.15.47:3300/0 conn(0x7efe68080700 msgr2=0x7efe68080b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:29.044 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 --2- 172.21.15.18:0/26679018 >> v2:172.21.15.47:3300/0 conn(0x7efe68080700 0x7efe68080b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.044 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 -- 172.21.15.18:0/26679018 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe681545a0 con 0x7efe680625a0 2020-07-18T03:25:29.045 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ecb1700 1 --2- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 0x7efe68157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:29.045 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6e4b0700 1 --2- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680625a0 0x7efe680afa90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7efe64002f30 tx=0x7efe64006e90).ready entity=mon.0 client_cookie=ffb7f3278c9d9a01 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:29.045 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6d4ae700 1 -- 172.21.15.18:0/26679018 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe64009850 con 0x7efe680625a0 2020-07-18T03:25:29.045 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6d4ae700 1 -- 172.21.15.18:0/26679018 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe640099b0 con 0x7efe680625a0 2020-07-18T03:25:29.045 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6d4ae700 1 -- 172.21.15.18:0/26679018 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe64009d10 con 0x7efe680625a0 2020-07-18T03:25:29.046 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 -- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680625a0 msgr2=0x7efe680afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:29.046 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.038+0000 7efe6ff13700 1 --2- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680625a0 0x7efe680afa90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7efe64002f30 tx=0x7efe64006e90).stop 2020-07-18T03:25:29.046 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 -- 172.21.15.18:0/26679018 shutdown_connections 2020-07-18T03:25:29.046 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 --2- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe68157420 0x7efe68157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.046 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 --2- 172.21.15.18:0/26679018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680625a0 0x7efe680afa90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 --2- 172.21.15.18:0/26679018 >> v2:172.21.15.47:3300/0 conn(0x7efe68080700 0x7efe68080b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 -- 172.21.15.18:0/26679018 >> 172.21.15.18:0/26679018 conn(0x7efe6805f5a0 msgr2=0x7efe6805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:29.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 -- 172.21.15.18:0/26679018 shutdown_connections 2020-07-18T03:25:29.047 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.039+0000 7efe6ff13700 1 -- 172.21.15.18:0/26679018 wait complete. 2020-07-18T03:25:29.050 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.040+0000 7efe6ff13700 1 Processor -- start 2020-07-18T03:25:29.050 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.040+0000 7efe6ff13700 1 -- start start 2020-07-18T03:25:29.050 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ff13700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe68157420 0x7efe680b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:29.051 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ff13700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe680b2420 0x7efe68149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:29.051 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ff13700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe68148f30 con 0x7efe68080700 2020-07-18T03:25:29.051 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ff13700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe681491d0 con 0x7efe68157420 2020-07-18T03:25:29.051 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ff13700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe680b6590 con 0x7efe680b2420 2020-07-18T03:25:29.051 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe68157420 0x7efe680b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:29.052 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe68157420 0x7efe680b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58056/0 (socket says 172.21.15.18:58056) 2020-07-18T03:25:29.052 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 -- 172.21.15.18:0/2701509121 learned_addr learned my addr 172.21.15.18:0/2701509121 (peer_addr_for_me v2:172.21.15.18:0/0) 2020-07-18T03:25:29.052 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ecb1700 1 --2- 172.21.15.18:0/2701509121 >> v2:172.21.15.47:3300/0 conn(0x7efe680b2420 0x7efe68149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:29.052 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 -- 172.21.15.18:0/2701509121 >> v2:172.21.15.47:3300/0 conn(0x7efe680b2420 msgr2=0x7efe68149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:29.052 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 --2- 172.21.15.18:0/2701509121 >> v2:172.21.15.47:3300/0 conn(0x7efe680b2420 0x7efe68149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.053 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 -- 172.21.15.18:0/2701509121 >> v1:172.21.15.47:6789/0 conn(0x7efe68080700 legacy=0x7efe680b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:29.053 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 -- 172.21.15.18:0/2701509121 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe680709d0 con 0x7efe68157420 2020-07-18T03:25:29.053 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6ecb1700 1 --2- 172.21.15.18:0/2701509121 >> v2:172.21.15.47:3300/0 conn(0x7efe680b2420 0x7efe68149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:25:29.053 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.041+0000 7efe6dcaf700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe68157420 0x7efe680b16c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7efe680b3420 tx=0x7efe5c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:29.053 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.042+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe5c009980 con 0x7efe68157420 2020-07-18T03:25:29.054 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.042+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c009ae0 con 0x7efe68157420 2020-07-18T03:25:29.054 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.042+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe68070c20 con 0x7efe68157420 2020-07-18T03:25:29.054 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.042+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe5c009e40 con 0x7efe68157420 2020-07-18T03:25:29.054 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.042+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe680710f0 con 0x7efe68157420 2020-07-18T03:25:29.054 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.043+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 3) v1 ==== 61955+0+0 (secure 0 0 0) 0x7efe5c013480 con 0x7efe68157420 2020-07-18T03:25:29.055 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.043+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe68057450 con 0x7efe68157420 2020-07-18T03:25:29.055 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.043+0000 7efe59ffb700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4804ff80 0x7efe48052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:29.055 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.043+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1959+0+0 (secure 0 0 0) 0x7efe5c066a20 con 0x7efe68157420 2020-07-18T03:25:29.055 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.043+0000 7efe6e4b0700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4804ff80 0x7efe48052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:29.055 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.043+0000 7efe6e4b0700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4804ff80 0x7efe48052440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7efe64002f30 tx=0x7efe64015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:29.056 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.047+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7efe5c040450 con 0x7efe68157420 2020-07-18T03:25:29.223 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.222+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", "id": 7} v 0) v1 -- 0x7efe68068c50 con 0x7efe68157420 2020-07-18T03:25:29.226 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.225+0000 7efe59ffb700 1 -- 172.21.15.18:0/2701509121 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7efe5c043a70 con 0x7efe68157420 2020-07-18T03:25:29.226 INFO:teuthology.orchestra.run.smithi018.stdout:7 2020-07-18T03:25:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4804ff80 msgr2=0x7efe48052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4804ff80 0x7efe48052440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7efe64002f30 tx=0x7efe64015000).stop 2020-07-18T03:25:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe68157420 msgr2=0x7efe680b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe68157420 0x7efe680b16c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7efe680b3420 tx=0x7efe5c008040).stop 2020-07-18T03:25:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 shutdown_connections 2020-07-18T03:25:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 --2- 172.21.15.18:0/2701509121 >> v2:172.21.15.47:3300/0 conn(0x7efe680b2420 0x7efe68149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4804ff80 0x7efe48052440 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 --2- 172.21.15.18:0/2701509121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe68157420 0x7efe680b16c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:29.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 >> 172.21.15.18:0/2701509121 conn(0x7efe6805f5a0 msgr2=0x7efe680b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:29.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 shutdown_connections 2020-07-18T03:25:29.230 INFO:teuthology.orchestra.run.smithi018.stderr:2020-07-18T03:25:29.227+0000 7efe6ff13700 1 -- 172.21.15.18:0/2701509121 wait complete. 2020-07-18T03:25:29.238 INFO:tasks.ceph.osd.0:Restarting daemon 2020-07-18T03:25:29.238 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-07-18T03:25:29.241 INFO:tasks.ceph.osd.0:Started 2020-07-18T03:25:29.242 INFO:tasks.ceph.osd.1:Restarting daemon 2020-07-18T03:25:29.242 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-07-18T03:25:29.245 INFO:tasks.ceph.osd.1:Started 2020-07-18T03:25:29.245 INFO:tasks.ceph.osd.2:Restarting daemon 2020-07-18T03:25:29.245 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-07-18T03:25:29.248 INFO:tasks.ceph.osd.2:Started 2020-07-18T03:25:29.249 INFO:tasks.ceph.osd.3:Restarting daemon 2020-07-18T03:25:29.249 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-07-18T03:25:29.251 INFO:tasks.ceph.osd.3:Started 2020-07-18T03:25:29.252 INFO:tasks.ceph.osd.4:Restarting daemon 2020-07-18T03:25:29.252 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-07-18T03:25:29.255 INFO:tasks.ceph.osd.4:Started 2020-07-18T03:25:29.255 INFO:tasks.ceph.osd.5:Restarting daemon 2020-07-18T03:25:29.256 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-07-18T03:25:29.258 INFO:tasks.ceph.osd.5:Started 2020-07-18T03:25:29.259 INFO:tasks.ceph.osd.6:Restarting daemon 2020-07-18T03:25:29.259 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-07-18T03:25:29.261 INFO:tasks.ceph.osd.6:Started 2020-07-18T03:25:29.262 INFO:tasks.ceph.osd.7:Restarting daemon 2020-07-18T03:25:29.262 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-07-18T03:25:29.264 INFO:tasks.ceph.osd.7:Started 2020-07-18T03:25:29.265 INFO:tasks.ceph:Waiting for OSDs to come up 2020-07-18T03:25:29.859 INFO:tasks.ceph.osd.5.smithi018.stderr:2020-07-18T03:25:29.858+0000 7f39f3ba9ec0 -1 Falling back to public interface 2020-07-18T03:25:29.863 INFO:tasks.ceph.osd.4.smithi018.stderr:2020-07-18T03:25:29.862+0000 7f1c3d956ec0 -1 Falling back to public interface 2020-07-18T03:25:29.864 INFO:tasks.ceph.osd.1.smithi047.stderr:2020-07-18T03:25:29.863+0000 7fad15f3cec0 -1 Falling back to public interface 2020-07-18T03:25:29.867 INFO:tasks.ceph.osd.6.smithi018.stderr:2020-07-18T03:25:29.866+0000 7f92c6500ec0 -1 Falling back to public interface 2020-07-18T03:25:29.868 INFO:tasks.ceph.osd.7.smithi018.stderr:2020-07-18T03:25:29.867+0000 7fbef8a2dec0 -1 Falling back to public interface 2020-07-18T03:25:29.871 INFO:tasks.ceph.osd.3.smithi047.stderr:2020-07-18T03:25:29.871+0000 7f7c505c5ec0 -1 Falling back to public interface 2020-07-18T03:25:29.892 INFO:tasks.ceph.osd.2.smithi047.stderr:2020-07-18T03:25:29.891+0000 7fb90b030ec0 -1 Falling back to public interface 2020-07-18T03:25:29.893 INFO:tasks.ceph.osd.0.smithi047.stderr:2020-07-18T03:25:29.892+0000 7f660a0fcec0 -1 Falling back to public interface 2020-07-18T03:25:30.072 INFO:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-07-18T03:25:30.193 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.188+0000 7efe736ba700 1 Processor -- start 2020-07-18T03:25:30.193 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.189+0000 7efe736ba700 1 -- start start 2020-07-18T03:25:30.194 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe736ba700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe6c15c120 0x7efe6c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:30.194 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe736ba700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c154070 con 0x7efe6c0b9e30 2020-07-18T03:25:30.194 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe736ba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c154310 con 0x7efe6c0b9a50 2020-07-18T03:25:30.194 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe736ba700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c063a20 con 0x7efe6c15c120 2020-07-18T03:25:30.194 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe71c57700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efe6c0b9a50 0x7efe6c153110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38934/0 (socket says 172.21.15.47:38934) 2020-07-18T03:25:30.195 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe71c57700 1 -- 172.21.15.47:0/363734056 learned_addr learned my addr 172.21.15.47:0/363734056 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:30.195 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe72458700 1 --2- 172.21.15.47:0/363734056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe6c15c120 0x7efe6c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:30.195 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.190+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698989151 0 0) 0x7efe6c154310 con 0x7efe6c0b9a50 2020-07-18T03:25:30.195 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe58003240 con 0x7efe6c0b9a50 2020-07-18T03:25:30.195 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703490175 0 0) 0x7efe6c154070 con 0x7efe6c0b9e30 2020-07-18T03:25:30.196 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6c154310 con 0x7efe6c0b9e30 2020-07-18T03:25:30.196 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe72458700 1 -- 172.21.15.47:0/363734056 >> v1:172.21.15.47:6789/0 conn(0x7efe6c0b9a50 legacy=0x7efe6c153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.196 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe72458700 1 -- 172.21.15.47:0/363734056 >> v1:172.21.15.18:6789/0 conn(0x7efe6c0b9e30 legacy=0x7efe6c153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.196 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe72458700 1 -- 172.21.15.47:0/363734056 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c1545b0 con 0x7efe6c15c120 2020-07-18T03:25:30.196 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe72458700 1 --2- 172.21.15.47:0/363734056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe6c15c120 0x7efe6c15e510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7efe640032b0 tx=0x7efe640091d0).ready entity=mon.2 client_cookie=41c20e27cf5797f7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:30.197 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe6400b920 con 0x7efe6c15c120 2020-07-18T03:25:30.197 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6400ba80 con 0x7efe6c15c120 2020-07-18T03:25:30.197 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe70c55700 1 -- 172.21.15.47:0/363734056 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe6400bbf0 con 0x7efe6c15c120 2020-07-18T03:25:30.197 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 -- 172.21.15.47:0/363734056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe6c15c120 msgr2=0x7efe6c15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.197 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 --2- 172.21.15.47:0/363734056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe6c15c120 0x7efe6c15e510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7efe640032b0 tx=0x7efe640091d0).stop 2020-07-18T03:25:30.198 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 -- 172.21.15.47:0/363734056 shutdown_connections 2020-07-18T03:25:30.198 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 --2- 172.21.15.47:0/363734056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efe6c15c120 0x7efe6c15e510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:30.198 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 -- 172.21.15.47:0/363734056 >> 172.21.15.47:0/363734056 conn(0x7efe6c06b070 msgr2=0x7efe6c0b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:30.198 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 -- 172.21.15.47:0/363734056 shutdown_connections 2020-07-18T03:25:30.198 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.191+0000 7efe736ba700 1 -- 172.21.15.47:0/363734056 wait complete. 2020-07-18T03:25:30.201 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.194+0000 7efe736ba700 1 Processor -- start 2020-07-18T03:25:30.201 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.194+0000 7efe736ba700 1 -- start start 2020-07-18T03:25:30.202 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe736ba700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe6c0b9a50 0x7efe6c0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:30.202 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe736ba700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe6c0af2c0 0x7efe6c0654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:30.202 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe736ba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c0afa70 con 0x7efe6c0b01b0 2020-07-18T03:25:30.202 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe736ba700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6c0af860 con 0x7efe6c0af2c0 2020-07-18T03:25:30.202 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe736ba700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6c0afd10 con 0x7efe6c0b9a50 2020-07-18T03:25:30.203 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe71c57700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe6c0b9a50 0x7efe6c0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:30.203 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe72458700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efe6c0af2c0 0x7efe6c0654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:30.203 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe71c57700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe6c0b9a50 0x7efe6c0b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57852/0 (socket says 172.21.15.47:57852) 2020-07-18T03:25:30.203 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe71c57700 1 -- 172.21.15.47:0/3670268451 learned_addr learned my addr 172.21.15.47:0/3670268451 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:30.203 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.195+0000 7efe71456700 1 --1- 172.21.15.47:0/3670268451 >> v1:172.21.15.47:6789/0 conn(0x7efe6c0b01b0 0x7efe6c064db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38942/0 (socket says 172.21.15.47:38942) 2020-07-18T03:25:30.204 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe72458700 1 -- 172.21.15.47:0/3670268451 >> v2:172.21.15.47:3300/0 conn(0x7efe6c0b9a50 msgr2=0x7efe6c0b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.204 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe72458700 1 --2- 172.21.15.47:0/3670268451 >> v2:172.21.15.47:3300/0 conn(0x7efe6c0b9a50 0x7efe6c0b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:30.204 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe72458700 1 -- 172.21.15.47:0/3670268451 >> v1:172.21.15.47:6789/0 conn(0x7efe6c0b01b0 legacy=0x7efe6c064db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.204 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe72458700 1 -- 172.21.15.47:0/3670268451 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0b5870 con 0x7efe6c0af2c0 2020-07-18T03:25:30.204 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172950548 0 0) 0x7efe6c0afa70 con 0x7efe6c0b01b0 2020-07-18T03:25:30.205 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe72458700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe6c0af2c0 0x7efe6c0654c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7efe640032b0 tx=0x7efe64016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:30.205 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe640107a0 con 0x7efe6c0af2c0 2020-07-18T03:25:30.205 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64010900 con 0x7efe6c0af2c0 2020-07-18T03:25:30.205 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe64010c60 con 0x7efe6c0af2c0 2020-07-18T03:25:30.205 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c0659b0 con 0x7efe6c0af2c0 2020-07-18T03:25:30.206 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.196+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c15bce0 con 0x7efe6c0af2c0 2020-07-18T03:25:30.206 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.198+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 3) v1 ==== 61955+0+0 (secure 0 0 0) 0x7efe6401e070 con 0x7efe6c0af2c0 2020-07-18T03:25:30.206 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.198+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c065c90 con 0x7efe6c0af2c0 2020-07-18T03:25:30.206 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.198+0000 7efe617fa700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4404fe40 0x7efe44052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:30.206 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.198+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7efe64067c60 con 0x7efe6c0af2c0 2020-07-18T03:25:30.207 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.199+0000 7efe71c57700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4404fe40 0x7efe44052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:30.207 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.199+0000 7efe71c57700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4404fe40 0x7efe44052300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7efe6800d000 tx=0x7efe68009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:30.207 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.205+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7efe64041570 con 0x7efe6c0af2c0 2020-07-18T03:25:30.388 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.387+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe6c153860 con 0x7efe6c0af2c0 2020-07-18T03:25:30.389 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.388+0000 7efe617fa700 1 -- 172.21.15.47:0/3670268451 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8277 (secure 0 0 0) 0x7efe64015090 con 0x7efe6c0af2c0 2020-07-18T03:25:30.389 INFO:teuthology.misc.health.smithi047.stdout: 2020-07-18T03:25:30.390 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:29.223509+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"6","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:30.391 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4404fe40 msgr2=0x7efe44052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.391 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4404fe40 0x7efe44052300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7efe6800d000 tx=0x7efe68009040).stop 2020-07-18T03:25:30.392 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe6c0af2c0 msgr2=0x7efe6c0654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:30.392 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe6c0af2c0 0x7efe6c0654c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7efe640032b0 tx=0x7efe64016000).stop 2020-07-18T03:25:30.392 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 shutdown_connections 2020-07-18T03:25:30.392 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe6c0af2c0 0x7efe6c0654c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:30.392 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 --2- 172.21.15.47:0/3670268451 >> v2:172.21.15.47:3300/0 conn(0x7efe6c0b9a50 0x7efe6c0b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:30.393 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 --2- 172.21.15.47:0/3670268451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efe4404fe40 0x7efe44052300 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:30.393 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 >> 172.21.15.47:0/3670268451 conn(0x7efe6c06b070 msgr2=0x7efe6c0b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:30.393 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 shutdown_connections 2020-07-18T03:25:30.393 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:30.390+0000 7efe736ba700 1 -- 172.21.15.47:0/3670268451 wait complete. 2020-07-18T03:25:30.400 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-07-18T03:25:30.934 INFO:tasks.ceph.osd.5.smithi018.stderr:2020-07-18T03:25:30.932+0000 7f39f3ba9ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-07-18T03:25:30.936 INFO:tasks.ceph.osd.1.smithi047.stderr:2020-07-18T03:25:30.935+0000 7fad15f3cec0 -1 osd.1 0 log_to_monitors {default=true} 2020-07-18T03:25:30.943 INFO:tasks.ceph.osd.4.smithi018.stderr:2020-07-18T03:25:30.941+0000 7f1c3d956ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-07-18T03:25:30.948 INFO:tasks.ceph.osd.3.smithi047.stderr:2020-07-18T03:25:30.947+0000 7f7c505c5ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-07-18T03:25:30.951 INFO:tasks.ceph.osd.7.smithi018.stderr:2020-07-18T03:25:30.950+0000 7fbef8a2dec0 -1 osd.7 0 log_to_monitors {default=true} 2020-07-18T03:25:30.958 INFO:tasks.ceph.osd.6.smithi018.stderr:2020-07-18T03:25:30.953+0000 7f92c6500ec0 -1 osd.6 0 log_to_monitors {default=true} 2020-07-18T03:25:30.966 INFO:tasks.ceph.osd.2.smithi047.stderr:2020-07-18T03:25:30.965+0000 7fb90b030ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-07-18T03:25:30.982 INFO:tasks.ceph.osd.0.smithi047.stderr:2020-07-18T03:25:30.980+0000 7f660a0fcec0 -1 osd.0 0 log_to_monitors {default=true} 2020-07-18T03:25:32.486 INFO:tasks.ceph.osd.6.smithi018.stderr:2020-07-18T03:25:32.485+0000 7f92bcd16700 -1 osd.6 0 waiting for initial osdmap 2020-07-18T03:25:32.487 INFO:tasks.ceph.osd.4.smithi018.stderr:2020-07-18T03:25:32.486+0000 7f1c3416c700 -1 osd.4 0 waiting for initial osdmap 2020-07-18T03:25:32.487 INFO:tasks.ceph.osd.3.smithi047.stderr:2020-07-18T03:25:32.487+0000 7f7c485de700 -1 osd.3 0 waiting for initial osdmap 2020-07-18T03:25:32.489 INFO:tasks.ceph.osd.2.smithi047.stderr:2020-07-18T03:25:32.488+0000 7fb903049700 -1 osd.2 0 waiting for initial osdmap 2020-07-18T03:25:32.489 INFO:tasks.ceph.osd.0.smithi047.stderr:2020-07-18T03:25:32.488+0000 7f6600912700 -1 osd.0 0 waiting for initial osdmap 2020-07-18T03:25:32.489 INFO:tasks.ceph.osd.1.smithi047.stderr:2020-07-18T03:25:32.488+0000 7fad0c752700 -1 osd.1 0 waiting for initial osdmap 2020-07-18T03:25:32.490 INFO:tasks.ceph.osd.7.smithi018.stderr:2020-07-18T03:25:32.486+0000 7fbef0a46700 -1 osd.7 0 waiting for initial osdmap 2020-07-18T03:25:32.490 INFO:tasks.ceph.osd.5.smithi018.stderr:2020-07-18T03:25:32.489+0000 7f39ebbc2700 -1 osd.5 0 waiting for initial osdmap 2020-07-18T03:25:37.212 INFO:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-07-18T03:25:37.326 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.323+0000 7f97a22a0700 1 Processor -- start 2020-07-18T03:25:37.327 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.323+0000 7f97a22a0700 1 -- start start 2020-07-18T03:25:37.327 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f979c0b5cb0 0x7f979c14d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.327 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f979c14d870 0x7f979c156050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.327 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c154500 con 0x7f979c064b30 2020-07-18T03:25:37.327 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c1547a0 con 0x7f979c0b5cb0 2020-07-18T03:25:37.328 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0b6140 con 0x7f979c14d870 2020-07-18T03:25:37.328 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f979c14d870 0x7f979c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.328 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f979c14d870 0x7f979c156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58080/0 (socket says 172.21.15.47:58080) 2020-07-18T03:25:37.328 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a083d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f979c064b30 0x7f979c153a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39164/0 (socket says 172.21.15.47:39164) 2020-07-18T03:25:37.328 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 -- 172.21.15.47:0/1348487684 learned_addr learned my addr 172.21.15.47:0/1348487684 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:37.329 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f979bfff700 1 --2- 172.21.15.47:0/1348487684 >> v2:172.21.15.18:3300/0 conn(0x7f979c0b5cb0 0x7f979c14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.329 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 -- 172.21.15.47:0/1348487684 >> v2:172.21.15.18:3300/0 conn(0x7f979c0b5cb0 msgr2=0x7f979c14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:37.329 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 --2- 172.21.15.47:0/1348487684 >> v2:172.21.15.18:3300/0 conn(0x7f979c0b5cb0 0x7f979c14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.329 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 -- 172.21.15.47:0/1348487684 >> v1:172.21.15.47:6789/0 conn(0x7f979c064b30 legacy=0x7f979c153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.329 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 -- 172.21.15.47:0/1348487684 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0b6280 con 0x7f979c14d870 2020-07-18T03:25:37.330 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f979b7fe700 1 -- 172.21.15.47:0/1348487684 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540256228 0 0) 0x7f979c154500 con 0x7f979c064b30 2020-07-18T03:25:37.330 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a103e700 1 --2- 172.21.15.47:0/1348487684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c14d870 0x7f979c156050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f97940032b0 tx=0x7f9794009270).ready entity=mon.1 client_cookie=d5e37d7b3cfbe416 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:37.330 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f979b7fe700 1 -- 172.21.15.47:0/1348487684 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9794009e60 con 0x7f979c14d870 2020-07-18T03:25:37.330 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f979b7fe700 1 -- 172.21.15.47:0/1348487684 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979400bc00 con 0x7f979c14d870 2020-07-18T03:25:37.330 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f979b7fe700 1 -- 172.21.15.47:0/1348487684 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979400bd70 con 0x7f979c14d870 2020-07-18T03:25:37.331 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 -- 172.21.15.47:0/1348487684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c14d870 msgr2=0x7f979c156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.331 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.324+0000 7f97a22a0700 1 --2- 172.21.15.47:0/1348487684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c14d870 0x7f979c156050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f97940032b0 tx=0x7f9794009270).stop 2020-07-18T03:25:37.331 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.325+0000 7f97a22a0700 1 -- 172.21.15.47:0/1348487684 shutdown_connections 2020-07-18T03:25:37.331 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.325+0000 7f97a22a0700 1 --2- 172.21.15.47:0/1348487684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c14d870 0x7f979c156050 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.331 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.325+0000 7f97a22a0700 1 --2- 172.21.15.47:0/1348487684 >> v2:172.21.15.18:3300/0 conn(0x7f979c0b5cb0 0x7f979c14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.332 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.325+0000 7f97a22a0700 1 -- 172.21.15.47:0/1348487684 >> 172.21.15.47:0/1348487684 conn(0x7f979c06b070 msgr2=0x7f979c05b920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:37.332 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.325+0000 7f97a22a0700 1 -- 172.21.15.47:0/1348487684 shutdown_connections 2020-07-18T03:25:37.332 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.325+0000 7f97a22a0700 1 -- 172.21.15.47:0/1348487684 wait complete. 2020-07-18T03:25:37.335 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.326+0000 7f97a22a0700 1 Processor -- start 2020-07-18T03:25:37.335 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.326+0000 7f97a22a0700 1 -- start start 2020-07-18T03:25:37.335 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a22a0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f979c064b30 0x7f979c0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.336 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a22a0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979c0b5cb0 0x7f979c0b9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.336 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a22a0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f979c0b1680 0x7f979c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.336 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a22a0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0b4b90 con 0x7f979c064b30 2020-07-18T03:25:37.336 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a22a0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c14ed60 con 0x7f979c0b1680 2020-07-18T03:25:37.336 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a22a0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c14f000 con 0x7f979c0b5cb0 2020-07-18T03:25:37.337 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a103e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f979c0b1680 0x7f979c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.337 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a103e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f979c0b1680 0x7f979c14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58086/0 (socket says 172.21.15.47:58086) 2020-07-18T03:25:37.337 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a103e700 1 -- 172.21.15.47:0/4063921265 learned_addr learned my addr 172.21.15.47:0/4063921265 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:37.337 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f979bfff700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979c0b5cb0 0x7f979c0b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.338 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.327+0000 7f97a083d700 1 --2- 172.21.15.47:0/4063921265 >> v2:172.21.15.18:3300/0 conn(0x7f979c064b30 0x7f979c0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.338 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a103e700 1 -- 172.21.15.47:0/4063921265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979c0b5cb0 msgr2=0x7f979c0b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.338 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a103e700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979c0b5cb0 0x7f979c0b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.338 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a103e700 1 -- 172.21.15.47:0/4063921265 >> v2:172.21.15.18:3300/0 conn(0x7f979c064b30 msgr2=0x7f979c0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.338 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a103e700 1 --2- 172.21.15.47:0/4063921265 >> v2:172.21.15.18:3300/0 conn(0x7f979c064b30 0x7f979c0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.339 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a103e700 1 -- 172.21.15.47:0/4063921265 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0b9870 con 0x7f979c0b1680 2020-07-18T03:25:37.339 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a103e700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c0b1680 0x7f979c14ffb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f97940032b0 tx=0x7f9794016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:37.339 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9794010870 con 0x7f979c0b1680 2020-07-18T03:25:37.339 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97940109d0 con 0x7f979c0b1680 2020-07-18T03:25:37.339 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c1504a0 con 0x7f979c0b1680 2020-07-18T03:25:37.340 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9794010d30 con 0x7f979c0b1680 2020-07-18T03:25:37.340 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.328+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c14f4a0 con 0x7f979c0b1680 2020-07-18T03:25:37.340 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.329+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c0690d0 con 0x7f979c0b1680 2020-07-18T03:25:37.340 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.329+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f979400f070 con 0x7f979c0b1680 2020-07-18T03:25:37.340 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.330+0000 7f9793fff700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97780500f0 0x7f97780525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.341 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.330+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f9794068b90 con 0x7f979c0b1680 2020-07-18T03:25:37.341 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.330+0000 7f97a083d700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97780500f0 0x7f97780525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.341 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.330+0000 7f97a083d700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97780500f0 0x7f97780525b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f978400d000 tx=0x7f9784009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:37.341 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.333+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f97940419c0 con 0x7f979c0b1680 2020-07-18T03:25:37.496 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.495+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f979c14f170 con 0x7f979c0b1680 2020-07-18T03:25:37.497 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.496+0000 7f9793fff700 1 -- 172.21.15.47:0/4063921265 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11808 (secure 0 0 0) 0x7f9794015070 con 0x7f979c0b1680 2020-07-18T03:25:37.497 INFO:teuthology.misc.health.smithi047.stdout: 2020-07-18T03:25:37.498 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":15,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:35.494971+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:37.499 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97780500f0 msgr2=0x7f97780525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.499 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97780500f0 0x7f97780525b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f978400d000 tx=0x7f9784009040).stop 2020-07-18T03:25:37.499 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c0b1680 msgr2=0x7f979c14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.499 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c0b1680 0x7f979c14ffb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f97940032b0 tx=0x7f9794016000).stop 2020-07-18T03:25:37.500 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 shutdown_connections 2020-07-18T03:25:37.500 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97780500f0 0x7f97780525b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.500 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f979c0b1680 0x7f979c14ffb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.500 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 --2- 172.21.15.47:0/4063921265 >> v2:172.21.15.18:3300/0 conn(0x7f979c064b30 0x7f979c0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.500 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 --2- 172.21.15.47:0/4063921265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979c0b5cb0 0x7f979c0b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.501 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 >> 172.21.15.47:0/4063921265 conn(0x7f979c06b070 msgr2=0x7f979c0b89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:37.501 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 shutdown_connections 2020-07-18T03:25:37.501 INFO:teuthology.misc.health.smithi047.stderr:2020-07-18T03:25:37.497+0000 7f97a22a0700 1 -- 172.21.15.47:0/4063921265 wait complete. 2020-07-18T03:25:37.508 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-07-18T03:25:37.509 INFO:tasks.ceph:Creating RBD pool 2020-07-18T03:25:37.509 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool create rbd 8 2020-07-18T03:25:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.622+0000 7f8e88753700 1 Processor -- start 2020-07-18T03:25:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.622+0000 7f8e88753700 1 -- start start 2020-07-18T03:25:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e88753700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8e80080510 0x7f8e800b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e88753700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e80153ad0 0x7f8e801575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e88753700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e80153f80 0x7f8e80157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e88753700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e80080bb0 con 0x7f8e80080510 2020-07-18T03:25:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e88753700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e80157140 con 0x7f8e80153f80 2020-07-18T03:25:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e88753700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800808f0 con 0x7f8e80153ad0 2020-07-18T03:25:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e80153f80 0x7f8e80157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e80153f80 0x7f8e80157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58094/0 (socket says 172.21.15.47:58094) 2020-07-18T03:25:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 -- 172.21.15.47:0/2053373897 learned_addr learned my addr 172.21.15.47:0/2053373897 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e864ef700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e80153ad0 0x7f8e801575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e86cf0700 1 --2- 172.21.15.47:0/2053373897 >> v2:172.21.15.18:3300/0 conn(0x7f8e80080510 0x7f8e800b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 -- 172.21.15.47:0/2053373897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e80153ad0 msgr2=0x7f8e801575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e80153ad0 0x7f8e801575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 -- 172.21.15.47:0/2053373897 >> v2:172.21.15.18:3300/0 conn(0x7f8e80080510 msgr2=0x7f8e800b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 --2- 172.21.15.47:0/2053373897 >> v2:172.21.15.18:3300/0 conn(0x7f8e80080510 0x7f8e800b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e874f1700 1 -- 172.21.15.47:0/2053373897 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80080e50 con 0x7f8e80153f80 2020-07-18T03:25:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e864ef700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e80153ad0 0x7f8e801575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.623+0000 7f8e86cf0700 1 --2- 172.21.15.47:0/2053373897 >> v2:172.21.15.18:3300/0 conn(0x7f8e80080510 0x7f8e800b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e874f1700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80153f80 0x7f8e80157ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8e780032b0 tx=0x7f8e78009270).ready entity=mon.1 client_cookie=71d374151b880a12 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e85cee700 1 -- 172.21.15.47:0/2053373897 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e7800bc00 con 0x7f8e80153f80 2020-07-18T03:25:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e85cee700 1 -- 172.21.15.47:0/2053373897 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7800bd60 con 0x7f8e80153f80 2020-07-18T03:25:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e85cee700 1 -- 172.21.15.47:0/2053373897 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e78011400 con 0x7f8e80153f80 2020-07-18T03:25:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 -- 172.21.15.47:0/2053373897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80153f80 msgr2=0x7f8e80157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80153f80 0x7f8e80157ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8e780032b0 tx=0x7f8e78009270).stop 2020-07-18T03:25:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 -- 172.21.15.47:0/2053373897 shutdown_connections 2020-07-18T03:25:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80153f80 0x7f8e80157ad0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 --2- 172.21.15.47:0/2053373897 >> v2:172.21.15.18:3300/0 conn(0x7f8e80080510 0x7f8e800b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 --2- 172.21.15.47:0/2053373897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e80153ad0 0x7f8e801575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 -- 172.21.15.47:0/2053373897 >> 172.21.15.47:0/2053373897 conn(0x7f8e8005c620 msgr2=0x7f8e800b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 -- 172.21.15.47:0/2053373897 shutdown_connections 2020-07-18T03:25:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.624+0000 7f8e88753700 1 -- 172.21.15.47:0/2053373897 wait complete. 2020-07-18T03:25:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.626+0000 7f8e88753700 1 Processor -- start 2020-07-18T03:25:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.626+0000 7f8e88753700 1 -- start start 2020-07-18T03:25:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e80080510 0x7f8e800adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e800b59c0 0x7f8e8013d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e80132f20 con 0x7f8e80153ad0 2020-07-18T03:25:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e80132c80 con 0x7f8e80080510 2020-07-18T03:25:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80130dd0 con 0x7f8e800b59c0 2020-07-18T03:25:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e80080510 0x7f8e800adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e80080510 0x7f8e800adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58096/0 (socket says 172.21.15.47:58096) 2020-07-18T03:25:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 -- 172.21.15.47:0/1892731537 learned_addr learned my addr 172.21.15.47:0/1892731537 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e874f1700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e800b59c0 0x7f8e8013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 -- 172.21.15.47:0/1892731537 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e800b59c0 msgr2=0x7f8e8013d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e800b59c0 0x7f8e8013d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 -- 172.21.15.47:0/1892731537 >> v1:172.21.15.18:6789/0 conn(0x7f8e80153ad0 legacy=0x7f8e800af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 -- 172.21.15.47:0/1892731537 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80132940 con 0x7f8e80080510 2020-07-18T03:25:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e874f1700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e800b59c0 0x7f8e8013d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e86cf0700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80080510 0x7f8e800adb80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8e7c006920 tx=0x7f8e7c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e7c009980 con 0x7f8e80080510 2020-07-18T03:25:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c009ae0 con 0x7f8e80080510 2020-07-18T03:25:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e7c009e40 con 0x7f8e80080510 2020-07-18T03:25:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e80133890 con 0x7f8e80080510 2020-07-18T03:25:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.627+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8013d500 con 0x7f8e80080510 2020-07-18T03:25:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.628+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e80132ad0 con 0x7f8e80080510 2020-07-18T03:25:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.628+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8e7c013480 con 0x7f8e80080510 2020-07-18T03:25:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.629+0000 7f8e6effd700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e5c04ffe0 0x7f8e5c0524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.629+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f8e7c067df0 con 0x7f8e80080510 2020-07-18T03:25:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.629+0000 7f8e864ef700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e5c04ffe0 0x7f8e5c0524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.629+0000 7f8e864ef700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e5c04ffe0 0x7f8e5c0524a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8e800b6740 tx=0x7f8e70008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.632+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8e7c040b10 con 0x7f8e80080510 2020-07-18T03:25:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:37.792+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f8e800b8ec0 con 0x7f8e80080510 2020-07-18T03:25:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.510+0000 7f8e6effd700 1 -- 172.21.15.47:0/1892731537 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f8e7c044130 con 0x7f8e80080510 2020-07-18T03:25:38.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e5c04ffe0 msgr2=0x7f8e5c0524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e5c04ffe0 0x7f8e5c0524a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8e800b6740 tx=0x7f8e70008040).stop 2020-07-18T03:25:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80080510 msgr2=0x7f8e800adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80080510 0x7f8e800adb80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8e7c006920 tx=0x7f8e7c008040).stop 2020-07-18T03:25:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 shutdown_connections 2020-07-18T03:25:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e5c04ffe0 0x7f8e5c0524a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8e800b59c0 0x7f8e8013d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 --2- 172.21.15.47:0/1892731537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80080510 0x7f8e800adb80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 >> 172.21.15.47:0/1892731537 conn(0x7f8e8005c620 msgr2=0x7f8e800643a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 shutdown_connections 2020-07-18T03:25:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.513+0000 7f8e88753700 1 -- 172.21.15.47:0/1892731537 wait complete. 2020-07-18T03:25:38.517 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'rbd' created 2020-07-18T03:25:38.528 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-07-18T03:25:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.651+0000 7f0d05126700 1 Processor -- start 2020-07-18T03:25:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.651+0000 7f0d05126700 1 -- start start 2020-07-18T03:25:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.652+0000 7f0d05126700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0d00153ad0 0x7f0d001575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.652+0000 7f0d05126700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 0x7f0d00157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.652+0000 7f0d05126700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d00066c20 con 0x7f0d00080510 2020-07-18T03:25:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.652+0000 7f0d05126700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d00080bb0 con 0x7f0d00153ad0 2020-07-18T03:25:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.652+0000 7f0d05126700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00157140 con 0x7f0d00153f80 2020-07-18T03:25:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.652+0000 7f0d04925700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 0x7f0d00157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 0x7f0d00157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:50712/0 (socket says 172.21.15.47:50712) 2020-07-18T03:25:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 -- 172.21.15.47:0/2636253842 learned_addr learned my addr 172.21.15.47:0/2636253842 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0cfed9d700 1 --2- 172.21.15.47:0/2636253842 >> v2:172.21.15.18:3300/0 conn(0x7f0d00153ad0 0x7f0d001575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 -- 172.21.15.47:0/2636253842 >> v2:172.21.15.18:3300/0 conn(0x7f0d00153ad0 msgr2=0x7f0d001575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 --2- 172.21.15.47:0/2636253842 >> v2:172.21.15.18:3300/0 conn(0x7f0d00153ad0 0x7f0d001575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 -- 172.21.15.47:0/2636253842 >> v1:172.21.15.18:6789/0 conn(0x7f0d00080510 legacy=0x7f0d000b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 -- 172.21.15.47:0/2636253842 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000808f0 con 0x7f0d00153f80 2020-07-18T03:25:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d04925700 1 --2- 172.21.15.47:0/2636253842 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 0x7f0d00157ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f0cf80032b0 tx=0x7f0cf8009270).ready entity=mon.2 client_cookie=203bd527ef5f7847 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0cfe59c700 1 -- 172.21.15.47:0/2636253842 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf8009e60 con 0x7f0d00153f80 2020-07-18T03:25:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0cfe59c700 1 -- 172.21.15.47:0/2636253842 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf800bc00 con 0x7f0d00153f80 2020-07-18T03:25:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0cfe59c700 1 -- 172.21.15.47:0/2636253842 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf800bd70 con 0x7f0d00153f80 2020-07-18T03:25:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 -- 172.21.15.47:0/2636253842 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 msgr2=0x7f0d00157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 --2- 172.21.15.47:0/2636253842 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 0x7f0d00157ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f0cf80032b0 tx=0x7f0cf8009270).stop 2020-07-18T03:25:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 -- 172.21.15.47:0/2636253842 shutdown_connections 2020-07-18T03:25:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 --2- 172.21.15.47:0/2636253842 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00153f80 0x7f0d00157ad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 --2- 172.21.15.47:0/2636253842 >> v2:172.21.15.18:3300/0 conn(0x7f0d00153ad0 0x7f0d001575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 -- 172.21.15.47:0/2636253842 >> 172.21.15.47:0/2636253842 conn(0x7f0d0005c620 msgr2=0x7f0d000b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 -- 172.21.15.47:0/2636253842 shutdown_connections 2020-07-18T03:25:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.653+0000 7f0d05126700 1 -- 172.21.15.47:0/2636253842 wait complete. 2020-07-18T03:25:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.655+0000 7f0d05126700 1 Processor -- start 2020-07-18T03:25:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.655+0000 7f0d05126700 1 -- start start 2020-07-18T03:25:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0d05126700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00080510 0x7f0d000635d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0d05126700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000ae9f0 con 0x7f0d00143e40 2020-07-18T03:25:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0d05126700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000aec90 con 0x7f0d00153ad0 2020-07-18T03:25:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0d05126700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0012d290 con 0x7f0d00080510 2020-07-18T03:25:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0cfed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d00153ad0 0x7f0d0012b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39208/0 (socket says 172.21.15.47:39208) 2020-07-18T03:25:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0cfed9d700 1 -- 172.21.15.47:0/870779396 learned_addr learned my addr 172.21.15.47:0/870779396 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0cff59e700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00080510 0x7f0d000635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944295967 0 0) 0x7f0d000aec90 con 0x7f0d00153ad0 2020-07-18T03:25:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cdc003240 con 0x7f0d00153ad0 2020-07-18T03:25:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.656+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068002934 0 0) 0x7f0d000ae9f0 con 0x7f0d00143e40 2020-07-18T03:25:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d000aec90 con 0x7f0d00143e40 2020-07-18T03:25:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0cff59e700 1 -- 172.21.15.47:0/870779396 >> v1:172.21.15.47:6789/0 conn(0x7f0d00153ad0 legacy=0x7f0d0012b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0cff59e700 1 -- 172.21.15.47:0/870779396 >> v1:172.21.15.18:6789/0 conn(0x7f0d00143e40 legacy=0x7f0d000af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0cff59e700 1 -- 172.21.15.47:0/870779396 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00063ac0 con 0x7f0d00080510 2020-07-18T03:25:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0cff59e700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00080510 0x7f0d000635d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f0d000b90e0 tx=0x7f0cf0007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:38.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf0013410 con 0x7f0d00080510 2020-07-18T03:25:38.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0013570 con 0x7f0d00080510 2020-07-18T03:25:38.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0012c010 con 0x7f0d00080510 2020-07-18T03:25:38.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf00138d0 con 0x7f0d00080510 2020-07-18T03:25:38.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.657+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d000ad460 con 0x7f0d00080510 2020-07-18T03:25:38.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.658+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d000b9ff0 con 0x7f0d00080510 2020-07-18T03:25:38.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.658+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0cf0019020 con 0x7f0d00080510 2020-07-18T03:25:38.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.659+0000 7f0ce6ffd700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0cdc050490 0x7f0cdc052950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:38.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.659+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f0cf0067040 con 0x7f0d00080510 2020-07-18T03:25:38.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.659+0000 7f0cfed9d700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0cdc050490 0x7f0cdc052950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:38.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.659+0000 7f0cfed9d700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0cdc050490 0x7f0cdc052950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0cf400a000 tx=0x7f0cf4006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:38.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.663+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0cf003fc70 con 0x7f0d00080510 2020-07-18T03:25:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:38.821+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d00061d20 con 0x7f0d00080510 2020-07-18T03:25:39.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.511+0000 7f0ce6ffd700 1 -- 172.21.15.47:0/870779396 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0cf0043290 con 0x7f0d00080510 2020-07-18T03:25:39.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0cdc050490 msgr2=0x7f0cdc052950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0cdc050490 0x7f0cdc052950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0cf400a000 tx=0x7f0cf4006dd0).stop 2020-07-18T03:25:39.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00080510 msgr2=0x7f0d000635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00080510 0x7f0d000635d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f0d000b90e0 tx=0x7f0cf0007b70).stop 2020-07-18T03:25:39.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 shutdown_connections 2020-07-18T03:25:39.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d00080510 0x7f0d000635d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 --2- 172.21.15.47:0/870779396 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0cdc050490 0x7f0cdc052950 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 >> 172.21.15.47:0/870779396 conn(0x7f0d0005c620 msgr2=0x7f0d0012d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 shutdown_connections 2020-07-18T03:25:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.514+0000 7f0d05126700 1 -- 172.21.15.47:0/870779396 wait complete. 2020-07-18T03:25:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rbd' on pool 'rbd' 2020-07-18T03:25:39.528 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-07-18T03:25:39.530 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.651+0000 7f7a45209700 1 Processor -- start 2020-07-18T03:25:39.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.651+0000 7f7a45209700 1 -- start start 2020-07-18T03:25:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a45209700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a400ad9e0 0x7f7a401428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a45209700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a400b3ac0 con 0x7f7a400b5370 2020-07-18T03:25:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a45209700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a400b3d20 con 0x7f7a400b41c0 2020-07-18T03:25:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a45209700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a400b5a10 con 0x7f7a400ad9e0 2020-07-18T03:25:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a400ad9e0 0x7f7a401428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a400b41c0 0x7f7a400ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39214/0 (socket says 172.21.15.47:39214) 2020-07-18T03:25:39.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a400ad9e0 0x7f7a401428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58128/0 (socket says 172.21.15.47:58128) 2020-07-18T03:25:39.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3e59c700 1 -- 172.21.15.47:0/968317977 learned_addr learned my addr 172.21.15.47:0/968317977 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:39.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3f59e700 1 -- 172.21.15.47:0/968317977 >> v1:172.21.15.47:6789/0 conn(0x7f7a400b41c0 legacy=0x7f7a400ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3f59e700 1 -- 172.21.15.47:0/968317977 >> v1:172.21.15.18:6789/0 conn(0x7f7a400b5370 legacy=0x7f7a4005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3f59e700 1 -- 172.21.15.47:0/968317977 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400b5750 con 0x7f7a400ad9e0 2020-07-18T03:25:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3f59e700 1 --2- 172.21.15.47:0/968317977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400ad9e0 0x7f7a401428f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f7a400af270 tx=0x7f7a380091d0).ready entity=mon.1 client_cookie=82a7b791976e601 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3dd9b700 1 -- 172.21.15.47:0/968317977 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a38009e80 con 0x7f7a400ad9e0 2020-07-18T03:25:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.652+0000 7f7a3dd9b700 1 -- 172.21.15.47:0/968317977 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3800bb80 con 0x7f7a400ad9e0 2020-07-18T03:25:39.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a3dd9b700 1 -- 172.21.15.47:0/968317977 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3800bcf0 con 0x7f7a400ad9e0 2020-07-18T03:25:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 -- 172.21.15.47:0/968317977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400ad9e0 msgr2=0x7f7a401428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 --2- 172.21.15.47:0/968317977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400ad9e0 0x7f7a401428f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f7a400af270 tx=0x7f7a380091d0).stop 2020-07-18T03:25:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 -- 172.21.15.47:0/968317977 shutdown_connections 2020-07-18T03:25:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 --2- 172.21.15.47:0/968317977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400ad9e0 0x7f7a401428f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 -- 172.21.15.47:0/968317977 >> 172.21.15.47:0/968317977 conn(0x7f7a400b9340 msgr2=0x7f7a400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:39.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 -- 172.21.15.47:0/968317977 shutdown_connections 2020-07-18T03:25:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.653+0000 7f7a45209700 1 -- 172.21.15.47:0/968317977 wait complete. 2020-07-18T03:25:39.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.654+0000 7f7a45209700 1 Processor -- start 2020-07-18T03:25:39.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.654+0000 7f7a45209700 1 -- start start 2020-07-18T03:25:39.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a45209700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a400b41c0 0x7f7a40133050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a45209700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a40147ef0 con 0x7f7a400b5370 2020-07-18T03:25:39.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a45209700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a40148190 con 0x7f7a40149580 2020-07-18T03:25:39.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a45209700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a40134030 con 0x7f7a400b41c0 2020-07-18T03:25:39.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a3ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a400b41c0 0x7f7a40133050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a3f59e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a40149580 0x7f7a4014aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39222/0 (socket says 172.21.15.47:39222) 2020-07-18T03:25:39.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a3f59e700 1 -- 172.21.15.47:0/2328063598 learned_addr learned my addr 172.21.15.47:0/2328063598 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:39.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223348186 0 0) 0x7f7a40148190 con 0x7f7a40149580 2020-07-18T03:25:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a1c003240 con 0x7f7a40149580 2020-07-18T03:25:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a3ed9d700 1 -- 172.21.15.47:0/2328063598 >> v1:172.21.15.47:6789/0 conn(0x7f7a40149580 legacy=0x7f7a4014aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a3ed9d700 1 -- 172.21.15.47:0/2328063598 >> v1:172.21.15.18:6789/0 conn(0x7f7a400b5370 legacy=0x7f7a40133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.655+0000 7f7a3ed9d700 1 -- 172.21.15.47:0/2328063598 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40133540 con 0x7f7a400b41c0 2020-07-18T03:25:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.656+0000 7f7a3ed9d700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400b41c0 0x7f7a40133050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7a28001f50 tx=0x7f7a28007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.656+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a28013470 con 0x7f7a400b41c0 2020-07-18T03:25:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.656+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280135d0 con 0x7f7a400b41c0 2020-07-18T03:25:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.656+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4014bee0 con 0x7f7a400b41c0 2020-07-18T03:25:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.656+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a28013930 con 0x7f7a400b41c0 2020-07-18T03:25:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.656+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a40148c40 con 0x7f7a400b41c0 2020-07-18T03:25:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.657+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a40066f90 con 0x7f7a400b41c0 2020-07-18T03:25:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.657+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7a28010030 con 0x7f7a400b41c0 2020-07-18T03:25:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.657+0000 7f7a36ffd700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a1c0503c0 0x7f7a1c052880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.657+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7a28067580 con 0x7f7a400b41c0 2020-07-18T03:25:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.657+0000 7f7a3e59c700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a1c0503c0 0x7f7a1c052880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.658+0000 7f7a3e59c700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a1c0503c0 0x7f7a1c052880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7a400b33b0 tx=0x7f7a300077f0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.661+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7a28040140 con 0x7f7a400b41c0 2020-07-18T03:25:39.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.820+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a4014c0d0 con 0x7f7a400b41c0 2020-07-18T03:25:39.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.821+0000 7f7a36ffd700 1 -- 172.21.15.47:0/2328063598 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13194 (secure 0 0 0) 0x7f7a28043760 con 0x7f7a400b41c0 2020-07-18T03:25:39.822 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:39.823 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:39.508083+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a1c0503c0 msgr2=0x7f7a1c052880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a1c0503c0 0x7f7a1c052880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7a400b33b0 tx=0x7f7a300077f0).stop 2020-07-18T03:25:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400b41c0 msgr2=0x7f7a40133050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400b41c0 0x7f7a40133050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7a28001f50 tx=0x7f7a28007b50).stop 2020-07-18T03:25:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 shutdown_connections 2020-07-18T03:25:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a1c0503c0 0x7f7a1c052880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 --2- 172.21.15.47:0/2328063598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a400b41c0 0x7f7a40133050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 >> 172.21.15.47:0/2328063598 conn(0x7f7a400b9340 msgr2=0x7f7a400b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 shutdown_connections 2020-07-18T03:25:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.822+0000 7f7a45209700 1 -- 172.21.15.47:0/2328063598 wait complete. 2020-07-18T03:25:39.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-07-18T03:25:39.835 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:25:39.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.952+0000 7f567a5f3700 1 Processor -- start 2020-07-18T03:25:39.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.952+0000 7f567a5f3700 1 -- start start 2020-07-18T03:25:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.952+0000 7f567a5f3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5674061c60 0x7f56740620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.952+0000 7f567a5f3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56740af4b0 con 0x7f5674142a20 2020-07-18T03:25:39.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.952+0000 7f567a5f3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56740b6d80 con 0x7f56740b3af0 2020-07-18T03:25:39.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.952+0000 7f567a5f3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56740af710 con 0x7f5674061c60 2020-07-18T03:25:39.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5678b90700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f56740b3af0 0x7f56740b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39230/0 (socket says 172.21.15.47:39230) 2020-07-18T03:25:39.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5678b90700 1 -- 172.21.15.47:0/4053238023 learned_addr learned my addr 172.21.15.47:0/4053238023 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:39.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f56737fe700 1 --2- 172.21.15.47:0/4053238023 >> v2:172.21.15.18:3300/0 conn(0x7f5674061c60 0x7f56740620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453473062 0 0) 0x7f56740b6d80 con 0x7f56740b3af0 2020-07-18T03:25:39.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5660003240 con 0x7f56740b3af0 2020-07-18T03:25:39.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336020148 0 0) 0x7f56740af4b0 con 0x7f5674142a20 2020-07-18T03:25:39.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56740b6d80 con 0x7f5674142a20 2020-07-18T03:25:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1252865167 0 0) 0x7f5660003240 con 0x7f56740b3af0 2020-07-18T03:25:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 >> v2:172.21.15.18:3300/0 conn(0x7f5674061c60 msgr2=0x7f56740620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 --2- 172.21.15.47:0/4053238023 >> v2:172.21.15.18:3300/0 conn(0x7f5674061c60 0x7f56740620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 >> v1:172.21.15.18:6789/0 conn(0x7f5674142a20 legacy=0x7f5674142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56740af9b0 con 0x7f56740b3af0 2020-07-18T03:25:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f566c001a50 con 0x7f56740b3af0 2020-07-18T03:25:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 >> v1:172.21.15.47:6789/0 conn(0x7f56740b3af0 legacy=0x7f56740b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56600038c0 0x7f5660003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56600044e0 0x7f5660004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660004e50 0x7f566000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56740af710 con 0x7f56600038c0 2020-07-18T03:25:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566000bc20 con 0x7f5660004e50 2020-07-18T03:25:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566000bde0 con 0x7f56600044e0 2020-07-18T03:25:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5678b90700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660004e50 0x7f566000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f56737fe700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56600044e0 0x7f5660004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.953+0000 7f5673fff700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56600038c0 0x7f5660003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 -- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56600044e0 msgr2=0x7f5660004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56600044e0 0x7f5660004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 -- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56600038c0 msgr2=0x7f5660003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56600038c0 0x7f5660003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 -- 172.21.15.47:0/4053238023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5660003240 con 0x7f5660004e50 2020-07-18T03:25:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 -- 172.21.15.47:0/4053238023 reap_dead start 2020-07-18T03:25:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5678b90700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660004e50 0x7f566000b730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f56600054b0 tx=0x7f56740629a0).ready entity=mon.1 client_cookie=9475a72a1e8f33f7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566c005b80 con 0x7f5660004e50 2020-07-18T03:25:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c005ce0 con 0x7f5660004e50 2020-07-18T03:25:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f5672ffd700 1 -- 172.21.15.47:0/4053238023 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566c0072c0 con 0x7f5660004e50 2020-07-18T03:25:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 -- 172.21.15.47:0/4053238023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660004e50 msgr2=0x7f566000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660004e50 0x7f566000b730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f56600054b0 tx=0x7f56740629a0).stop 2020-07-18T03:25:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 -- 172.21.15.47:0/4053238023 shutdown_connections 2020-07-18T03:25:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4053238023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660004e50 0x7f566000b730 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 -- 172.21.15.47:0/4053238023 >> 172.21.15.47:0/4053238023 conn(0x7f567405f5a0 msgr2=0x7f567405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 -- 172.21.15.47:0/4053238023 shutdown_connections 2020-07-18T03:25:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.954+0000 7f567a5f3700 1 -- 172.21.15.47:0/4053238023 wait complete. 2020-07-18T03:25:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.955+0000 7f567a5f3700 1 Processor -- start 2020-07-18T03:25:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f567a5f3700 1 -- start start 2020-07-18T03:25:39.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f567a5f3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5674063420 0x7f5674143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f567a5f3700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f567412b100 0x7f5674144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f567a5f3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5674146600 con 0x7f5674063040 2020-07-18T03:25:39.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f567a5f3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5674136730 con 0x7f5674063420 2020-07-18T03:25:39.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f567a5f3700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5674144950 con 0x7f567412b100 2020-07-18T03:25:39.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f5678b90700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f567412b100 0x7f5674144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f5673fff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f5674063040 0x7f5674136020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58006/0 (socket says 172.21.15.47:58006) 2020-07-18T03:25:39.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.956+0000 7f5673fff700 1 -- 172.21.15.47:0/4274385719 learned_addr learned my addr 172.21.15.47:0/4274385719 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:39.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f56737fe700 1 --2- 172.21.15.47:0/4274385719 >> v2:172.21.15.18:3300/0 conn(0x7f5674063420 0x7f5674143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5678b90700 1 -- 172.21.15.47:0/4274385719 >> v2:172.21.15.18:3300/0 conn(0x7f5674063420 msgr2=0x7f5674143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5678b90700 1 --2- 172.21.15.47:0/4274385719 >> v2:172.21.15.18:3300/0 conn(0x7f5674063420 0x7f5674143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:39.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5678b90700 1 -- 172.21.15.47:0/4274385719 >> v1:172.21.15.18:6789/0 conn(0x7f5674063040 legacy=0x7f5674136020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:39.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5678b90700 1 -- 172.21.15.47:0/4274385719 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674145360 con 0x7f567412b100 2020-07-18T03:25:39.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f56737fe700 1 --2- 172.21.15.47:0/4274385719 >> v2:172.21.15.18:3300/0 conn(0x7f5674063420 0x7f5674143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:39.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5678b90700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f567412b100 0x7f5674144460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f5674137aa0 tx=0x7f566c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:39.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566c005b80 con 0x7f567412b100 2020-07-18T03:25:39.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c013d80 con 0x7f567412b100 2020-07-18T03:25:39.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56741348e0 con 0x7f567412b100 2020-07-18T03:25:39.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566c01f5f0 con 0x7f567412b100 2020-07-18T03:25:39.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.957+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5674134d50 con 0x7f567412b100 2020-07-18T03:25:39.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.958+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567405b120 con 0x7f567412b100 2020-07-18T03:25:39.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.958+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f566c026030 con 0x7f567412b100 2020-07-18T03:25:39.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.959+0000 7f5653fff700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f564c04ffb0 0x7f564c052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:39.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.959+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f566c068ba0 con 0x7f567412b100 2020-07-18T03:25:39.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.959+0000 7f5673fff700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f564c04ffb0 0x7f564c052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:39.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.959+0000 7f5673fff700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f564c04ffb0 0x7f564c052470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f567412c6a0 tx=0x7f5664007ae0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:39.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:39.963+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f566c0417c0 con 0x7f567412b100 2020-07-18T03:25:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.121+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f56740690d0 con 0x7f567412b100 2020-07-18T03:25:40.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.122+0000 7f5653fff700 1 -- 172.21.15.47:0/4274385719 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f566c044de0 con 0x7f567412b100 2020-07-18T03:25:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:25:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f564c04ffb0 msgr2=0x7f564c052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f564c04ffb0 0x7f564c052470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f567412c6a0 tx=0x7f5664007ae0).stop 2020-07-18T03:25:40.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f567412b100 msgr2=0x7f5674144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f567412b100 0x7f5674144460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f5674137aa0 tx=0x7f566c015000).stop 2020-07-18T03:25:40.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 shutdown_connections 2020-07-18T03:25:40.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f564c04ffb0 0x7f564c052470 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4274385719 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f567412b100 0x7f5674144460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 --2- 172.21.15.47:0/4274385719 >> v2:172.21.15.18:3300/0 conn(0x7f5674063420 0x7f5674143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 >> 172.21.15.47:0/4274385719 conn(0x7f567405f5a0 msgr2=0x7f56740aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 shutdown_connections 2020-07-18T03:25:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.123+0000 7f567a5f3700 1 -- 172.21.15.47:0/4274385719 wait complete. 2020-07-18T03:25:40.134 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:25:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.250+0000 7f384713f700 1 Processor -- start 2020-07-18T03:25:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.250+0000 7f384713f700 1 -- start start 2020-07-18T03:25:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f384713f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 0x7f3840143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f384713f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f38400b3a10 0x7f38400b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f384713f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3840147de0 con 0x7f38400b59b0 2020-07-18T03:25:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f384713f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38400ae000 con 0x7f38400b3a10 2020-07-18T03:25:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f384713f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3840148010 con 0x7f38400ad330 2020-07-18T03:25:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 0x7f3840143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f38456dc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f38400b3a10 0x7f38400b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 0x7f3840143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:50756/0 (socket says 172.21.15.47:50756) 2020-07-18T03:25:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 -- 172.21.15.47:0/1126053613 learned_addr learned my addr 172.21.15.47:0/1126053613 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f383ffff700 1 --1- 172.21.15.47:0/1126053613 >> v1:172.21.15.18:6789/0 conn(0x7f38400b59b0 0x7f38400b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58012/0 (socket says 172.21.15.47:58012) 2020-07-18T03:25:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 -- 172.21.15.47:0/1126053613 >> v2:172.21.15.18:3300/0 conn(0x7f38400b3a10 msgr2=0x7f38400b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 --2- 172.21.15.47:0/1126053613 >> v2:172.21.15.18:3300/0 conn(0x7f38400b3a10 0x7f38400b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 -- 172.21.15.47:0/1126053613 >> v1:172.21.15.18:6789/0 conn(0x7f38400b59b0 legacy=0x7f38400b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.251+0000 7f3844edb700 1 -- 172.21.15.47:0/1126053613 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400ad710 con 0x7f38400ad330 2020-07-18T03:25:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f3844edb700 1 --2- 172.21.15.47:0/1126053613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 0x7f3840143640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3830002f30 tx=0x7f3830006e90).ready entity=mon.2 client_cookie=7bdf65fd9fc63cbb server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f383f7fe700 1 -- 172.21.15.47:0/1126053613 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3830009850 con 0x7f38400ad330 2020-07-18T03:25:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f383f7fe700 1 -- 172.21.15.47:0/1126053613 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300099b0 con 0x7f38400ad330 2020-07-18T03:25:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f383f7fe700 1 -- 172.21.15.47:0/1126053613 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3830009d10 con 0x7f38400ad330 2020-07-18T03:25:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 -- 172.21.15.47:0/1126053613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 msgr2=0x7f3840143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 --2- 172.21.15.47:0/1126053613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 0x7f3840143640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3830002f30 tx=0x7f3830006e90).stop 2020-07-18T03:25:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 -- 172.21.15.47:0/1126053613 shutdown_connections 2020-07-18T03:25:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 --2- 172.21.15.47:0/1126053613 >> v2:172.21.15.18:3300/0 conn(0x7f38400b3a10 0x7f38400b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 --2- 172.21.15.47:0/1126053613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400ad330 0x7f3840143640 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 -- 172.21.15.47:0/1126053613 >> 172.21.15.47:0/1126053613 conn(0x7f384005f5a0 msgr2=0x7f384005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 -- 172.21.15.47:0/1126053613 shutdown_connections 2020-07-18T03:25:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.252+0000 7f384713f700 1 -- 172.21.15.47:0/1126053613 wait complete. 2020-07-18T03:25:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.253+0000 7f384713f700 1 Processor -- start 2020-07-18T03:25:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.253+0000 7f384713f700 1 -- start start 2020-07-18T03:25:40.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f384713f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400b3a10 0x7f3840159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f384713f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38400b59b0 0x7f384015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f384713f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3840156e70 con 0x7f3840065840 2020-07-18T03:25:40.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f384713f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384015d8d0 con 0x7f38400b59b0 2020-07-18T03:25:40.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f384713f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3840159540 con 0x7f38400b3a10 2020-07-18T03:25:40.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f383ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38400b59b0 0x7f384015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f38456dc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3840065840 0x7f3840141aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39254/0 (socket says 172.21.15.47:39254) 2020-07-18T03:25:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f383ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38400b59b0 0x7f384015d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58164/0 (socket says 172.21.15.47:58164) 2020-07-18T03:25:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f38456dc700 1 -- 172.21.15.47:0/3243704143 learned_addr learned my addr 172.21.15.47:0/3243704143 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.254+0000 7f3844edb700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400b3a10 0x7f3840159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383ffff700 1 -- 172.21.15.47:0/3243704143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400b3a10 msgr2=0x7f3840159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383ffff700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400b3a10 0x7f3840159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383ffff700 1 -- 172.21.15.47:0/3243704143 >> v1:172.21.15.47:6789/0 conn(0x7f3840065840 legacy=0x7f3840141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383ffff700 1 -- 172.21.15.47:0/3243704143 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38401597e0 con 0x7f38400b59b0 2020-07-18T03:25:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192840186 0 0) 0x7f3840156e70 con 0x7f3840065840 2020-07-18T03:25:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383ffff700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38400b59b0 0x7f384015d3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f38400ae4e0 tx=0x7f3834007b40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38340072e0 con 0x7f38400b59b0 2020-07-18T03:25:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834005530 con 0x7f38400b59b0 2020-07-18T03:25:40.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3840159a80 con 0x7f38400b59b0 2020-07-18T03:25:40.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3834005890 con 0x7f38400b59b0 2020-07-18T03:25:40.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.255+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384013eda0 con 0x7f38400b59b0 2020-07-18T03:25:40.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.256+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3840159c10 con 0x7f38400b59b0 2020-07-18T03:25:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.256+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3834010070 con 0x7f38400b59b0 2020-07-18T03:25:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.256+0000 7f383cff9700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f381804ff00 0x7f38180523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.256+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3834066df0 con 0x7f38400b59b0 2020-07-18T03:25:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.256+0000 7f3844edb700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f381804ff00 0x7f38180523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.257+0000 7f3844edb700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f381804ff00 0x7f38180523c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f383001c000 tx=0x7f3830015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.260+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f383403f930 con 0x7f38400b59b0 2020-07-18T03:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.423+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f384005b120 con 0x7f38400b59b0 2020-07-18T03:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.423+0000 7f383cff9700 1 -- 172.21.15.47:0/3243704143 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3834042f50 con 0x7f38400b59b0 2020-07-18T03:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f381804ff00 msgr2=0x7f38180523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f381804ff00 0x7f38180523c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f383001c000 tx=0x7f3830015000).stop 2020-07-18T03:25:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38400b59b0 msgr2=0x7f384015d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38400b59b0 0x7f384015d3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f38400ae4e0 tx=0x7f3834007b40).stop 2020-07-18T03:25:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 shutdown_connections 2020-07-18T03:25:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f381804ff00 0x7f38180523c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38400b3a10 0x7f3840159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 --2- 172.21.15.47:0/3243704143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38400b59b0 0x7f384015d3e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 >> 172.21.15.47:0/3243704143 conn(0x7f384005f5a0 msgr2=0x7f38400b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 shutdown_connections 2020-07-18T03:25:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.425+0000 7f384713f700 1 -- 172.21.15.47:0/3243704143 wait complete. 2020-07-18T03:25:40.433 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:40.436 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-07-18T03:25:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.551+0000 7fa047ff2700 1 Processor -- start 2020-07-18T03:25:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.551+0000 7fa047ff2700 1 -- start start 2020-07-18T03:25:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa047ff2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa0400b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa047ff2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa0400b7420 0x7fa0400b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa047ff2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0400b95a0 con 0x7fa0400b82f0 2020-07-18T03:25:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa047ff2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0400b5a90 con 0x7fa0400b7420 2020-07-18T03:25:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa047ff2700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400b5c80 con 0x7fa0400b42a0 2020-07-18T03:25:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa04658f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa0400b82f0 0x7fa040061800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39260/0 (socket says 172.21.15.47:39260) 2020-07-18T03:25:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa04658f700 1 -- 172.21.15.47:0/3789487672 learned_addr learned my addr 172.21.15.47:0/3789487672 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa04558d700 1 --2- 172.21.15.47:0/3789487672 >> v2:172.21.15.18:3300/0 conn(0x7fa0400b7420 0x7fa0400b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379961921 0 0) 0x7fa0400b95a0 con 0x7fa0400b82f0 2020-07-18T03:25:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa045d8e700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa0400b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.552+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa028003240 con 0x7fa0400b82f0 2020-07-18T03:25:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1675096016 0 0) 0x7fa028003240 con 0x7fa0400b82f0 2020-07-18T03:25:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 msgr2=0x7fa0400b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa0400b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 >> v2:172.21.15.18:3300/0 conn(0x7fa0400b7420 msgr2=0x7fa0400b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 --2- 172.21.15.47:0/3789487672 >> v2:172.21.15.18:3300/0 conn(0x7fa0400b7420 0x7fa0400b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400b5f20 con 0x7fa0400b82f0 2020-07-18T03:25:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fa038001a50 con 0x7fa0400b82f0 2020-07-18T03:25:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 >> v1:172.21.15.47:6789/0 conn(0x7fa0400b82f0 legacy=0x7fa040061800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa028003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa028003c40 0x7fa0280044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0280049d0 0x7fa028004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400b5c80 con 0x7fa0280049d0 2020-07-18T03:25:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280041d0 con 0x7fa028003c40 2020-07-18T03:25:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02800b730 con 0x7fa0400b42a0 2020-07-18T03:25:40.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 --2- 172.21.15.47:0/3789487672 >> v2:172.21.15.18:3300/0 conn(0x7fa0400b7420 0x7fa0400b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa045d8e700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa028003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa028003c40 0x7fa0280044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04658f700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0280049d0 0x7fa028004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 -- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 msgr2=0x7fa028003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa028003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 -- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0280049d0 msgr2=0x7fa028004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0280049d0 0x7fa028004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 -- 172.21.15.47:0/3789487672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028003240 con 0x7fa028003c40 2020-07-18T03:25:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa045d8e700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0400b42a0 0x7fa028003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa04558d700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa028003c40 0x7fa0280044e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa03c00c000 tx=0x7fa03c008040).ready entity=mon.1 client_cookie=1ac540c2ad28de7b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.553+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa03c00ebd0 con 0x7fa028003c40 2020-07-18T03:25:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c00ed30 con 0x7fa028003c40 2020-07-18T03:25:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa044d8c700 1 -- 172.21.15.47:0/3789487672 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa03c00eea0 con 0x7fa028003c40 2020-07-18T03:25:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa047ff2700 1 -- 172.21.15.47:0/3789487672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa028003c40 msgr2=0x7fa0280044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa047ff2700 1 --2- 172.21.15.47:0/3789487672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa028003c40 0x7fa0280044e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa03c00c000 tx=0x7fa03c008040).stop 2020-07-18T03:25:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa04658f700 1 -- 172.21.15.47:0/3789487672 reap_dead start 2020-07-18T03:25:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa047ff2700 1 -- 172.21.15.47:0/3789487672 shutdown_connections 2020-07-18T03:25:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa047ff2700 1 -- 172.21.15.47:0/3789487672 >> 172.21.15.47:0/3789487672 conn(0x7fa04005f5a0 msgr2=0x7fa04005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa047ff2700 1 -- 172.21.15.47:0/3789487672 shutdown_connections 2020-07-18T03:25:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.554+0000 7fa047ff2700 1 -- 172.21.15.47:0/3789487672 wait complete. 2020-07-18T03:25:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.555+0000 7fa047ff2700 1 Processor -- start 2020-07-18T03:25:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.555+0000 7fa047ff2700 1 -- start start 2020-07-18T03:25:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa047ff2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0401461f0 0x7fa040116740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa047ff2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0401465d0 0x7fa0400ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa047ff2700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa040116d70 con 0x7fa0400b1480 2020-07-18T03:25:40.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa047ff2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0400ae090 con 0x7fa0401461f0 2020-07-18T03:25:40.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa047ff2700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400ae330 con 0x7fa0401465d0 2020-07-18T03:25:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0401461f0 0x7fa040116740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0401461f0 0x7fa040116740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58180/0 (socket says 172.21.15.47:58180) 2020-07-18T03:25:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 -- 172.21.15.47:0/2718674016 learned_addr learned my addr 172.21.15.47:0/2718674016 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa04558d700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0401465d0 0x7fa0400ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 -- 172.21.15.47:0/2718674016 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0401465d0 msgr2=0x7fa0400ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0401465d0 0x7fa0400ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 -- 172.21.15.47:0/2718674016 >> v1:172.21.15.18:6789/0 conn(0x7fa0400b1480 legacy=0x7fa040158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 -- 172.21.15.47:0/2718674016 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400aecc0 con 0x7fa0401461f0 2020-07-18T03:25:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa04558d700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0401465d0 0x7fa0400ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa045d8e700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0401461f0 0x7fa040116740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa030008000 tx=0x7fa030008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa03000fca0 con 0x7fa0401461f0 2020-07-18T03:25:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03000fe00 con 0x7fa0401461f0 2020-07-18T03:25:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0400b0d00 con 0x7fa0401461f0 2020-07-18T03:25:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.556+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa030014440 con 0x7fa0401461f0 2020-07-18T03:25:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.557+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0400b11d0 con 0x7fa0401461f0 2020-07-18T03:25:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.558+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0400b6b40 con 0x7fa0401461f0 2020-07-18T03:25:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.558+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa030014670 con 0x7fa0401461f0 2020-07-18T03:25:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.558+0000 7fa02dffb700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa018050040 0x7fa018052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.558+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa030067f00 con 0x7fa0401461f0 2020-07-18T03:25:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.558+0000 7fa04558d700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa018050040 0x7fa018052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.558+0000 7fa04558d700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa018050040 0x7fa018052500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa03c01a000 tx=0x7fa03c019000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.561+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa030040b20 con 0x7fa0401461f0 2020-07-18T03:25:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.721+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa0400aeeb0 con 0x7fa0401461f0 2020-07-18T03:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.721+0000 7fa02dffb700 1 -- 172.21.15.47:0/2718674016 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa030044140 con 0x7fa0401461f0 2020-07-18T03:25:40.723 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:40.723 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2020-07-18T03:25:40.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:25:40.724 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2020-07-18T03:25:40.724 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2020-07-18T03:25:40.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:25:40.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:25:40.725 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2020-07-18T03:25:40.726 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2020-07-18T03:25:40.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:25:40.726 INFO:teuthology.orchestra.run.smithi047.stdout:] 2020-07-18T03:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa018050040 msgr2=0x7fa018052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa018050040 0x7fa018052500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa03c01a000 tx=0x7fa03c019000).stop 2020-07-18T03:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0401461f0 msgr2=0x7fa040116740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0401461f0 0x7fa040116740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa030008000 tx=0x7fa030008040).stop 2020-07-18T03:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 shutdown_connections 2020-07-18T03:25:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa018050040 0x7fa018052500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0401461f0 0x7fa040116740 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 --2- 172.21.15.47:0/2718674016 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa0401465d0 0x7fa0400ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 >> 172.21.15.47:0/2718674016 conn(0x7fa04005f5a0 msgr2=0x7fa040142680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 shutdown_connections 2020-07-18T03:25:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.723+0000 7fa047ff2700 1 -- 172.21.15.47:0/2718674016 wait complete. 2020-07-18T03:25:40.734 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-07-18T03:25:40.735 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-07-18T03:25:40.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.851+0000 7f4bfdfdd700 1 Processor -- start 2020-07-18T03:25:40.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.851+0000 7f4bfdfdd700 1 -- start start 2020-07-18T03:25:40.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bfdfdd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4bf80ad330 0x7f4bf8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bfdfdd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bf80b59b0 0x7f4bf80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bfdfdd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8147de0 con 0x7f4bf80b6300 2020-07-18T03:25:40.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bfdfdd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf80ae000 con 0x7f4bf80ad330 2020-07-18T03:25:40.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bfdfdd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8148010 con 0x7f4bf80b59b0 2020-07-18T03:25:40.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bf6ffd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bf80b59b0 0x7f4bf80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bf77fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4bf80ad330 0x7f4bf8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bf7fff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f4bf80b6300 0x7f4bf80b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58042/0 (socket says 172.21.15.47:58042) 2020-07-18T03:25:40.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bf77fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4bf80ad330 0x7f4bf8143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:43868/0 (socket says 172.21.15.47:43868) 2020-07-18T03:25:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bf6ffd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bf80b59b0 0x7f4bf80b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:50786/0 (socket says 172.21.15.47:50786) 2020-07-18T03:25:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.852+0000 7f4bf7fff700 1 -- 172.21.15.47:0/3850850857 learned_addr learned my addr 172.21.15.47:0/3850850857 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf77fe700 1 -- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bf80b59b0 msgr2=0x7f4bf80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf77fe700 1 --2- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bf80b59b0 0x7f4bf80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf77fe700 1 -- 172.21.15.47:0/3850850857 >> v1:172.21.15.18:6789/0 conn(0x7f4bf80b6300 legacy=0x7f4bf80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf77fe700 1 -- 172.21.15.47:0/3850850857 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80ad710 con 0x7f4bf80ad330 2020-07-18T03:25:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf77fe700 1 --2- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bf80ad330 0x7f4bf8143640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4be8002f30 tx=0x7f4be8006dd0).ready entity=mon.0 client_cookie=b9ae79c874c3991c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf67fc700 1 -- 172.21.15.47:0/3850850857 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be8009760 con 0x7f4bf80ad330 2020-07-18T03:25:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf67fc700 1 -- 172.21.15.47:0/3850850857 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be80098c0 con 0x7f4bf80ad330 2020-07-18T03:25:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bf67fc700 1 -- 172.21.15.47:0/3850850857 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be8009c20 con 0x7f4bf80ad330 2020-07-18T03:25:40.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bf80ad330 msgr2=0x7f4bf8143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bf80ad330 0x7f4bf8143640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4be8002f30 tx=0x7f4be8006dd0).stop 2020-07-18T03:25:40.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/3850850857 shutdown_connections 2020-07-18T03:25:40.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bf80ad330 0x7f4bf8143640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/3850850857 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bf80b59b0 0x7f4bf80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/3850850857 >> 172.21.15.47:0/3850850857 conn(0x7f4bf805f5a0 msgr2=0x7f4bf805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/3850850857 shutdown_connections 2020-07-18T03:25:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.853+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/3850850857 wait complete. 2020-07-18T03:25:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bfdfdd700 1 Processor -- start 2020-07-18T03:25:40.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bfdfdd700 1 -- start start 2020-07-18T03:25:40.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bfdfdd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4bf80b6300 0x7f4bf8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bfdfdd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80637d0 con 0x7f4bf80b59b0 2020-07-18T03:25:40.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bfdfdd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8151d40 con 0x7f4bf80630c0 2020-07-18T03:25:40.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bfdfdd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf8150ef0 con 0x7f4bf80b6300 2020-07-18T03:25:40.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bf7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4bf80630c0 0x7f4bf8151630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39284/0 (socket says 172.21.15.47:39284) 2020-07-18T03:25:40.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.855+0000 7f4bf7fff700 1 -- 172.21.15.47:0/2565008144 learned_addr learned my addr 172.21.15.47:0/2565008144 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:40.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bf6ffd700 1 --2- 172.21.15.47:0/2565008144 >> v2:172.21.15.18:3300/0 conn(0x7f4bf80b6300 0x7f4bf8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216386747 0 0) 0x7f4bf8151d40 con 0x7f4bf80630c0 2020-07-18T03:25:40.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd0003240 con 0x7f4bf80630c0 2020-07-18T03:25:40.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1748369923 0 0) 0x7f4bd0003240 con 0x7f4bf80630c0 2020-07-18T03:25:40.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4bf8151d40 con 0x7f4bf80630c0 2020-07-18T03:25:40.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4bf0001b50 con 0x7f4bf80630c0 2020-07-18T03:25:40.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 >> v2:172.21.15.18:3300/0 conn(0x7f4bf80b6300 msgr2=0x7f4bf8154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 --2- 172.21.15.47:0/2565008144 >> v2:172.21.15.18:3300/0 conn(0x7f4bf80b6300 0x7f4bf8154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 >> v1:172.21.15.47:6789/0 conn(0x7f4bf80630c0 legacy=0x7f4bf8151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 >> v1:172.21.15.18:6789/0 conn(0x7f4bf80b59b0 legacy=0x7f4bf80b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd0003800 0x7f4bd0003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd00044e0 0x7f4bd0004960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd0004e50 0x7f4bd000b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8150ef0 con 0x7f4bd0003800 2020-07-18T03:25:40.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd00052d0 con 0x7f4bd0004e50 2020-07-18T03:25:40.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd000bc20 con 0x7f4bd00044e0 2020-07-18T03:25:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bf6ffd700 1 --2- 172.21.15.47:0/2565008144 >> v2:172.21.15.18:3300/0 conn(0x7f4bf80b6300 0x7f4bf8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bf7fff700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd0004e50 0x7f4bd000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bf77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd0003800 0x7f4bd0003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.856+0000 7f4bf6ffd700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd00044e0 0x7f4bd0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 -- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd00044e0 msgr2=0x7f4bd0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4bd00044e0 0x7f4bd0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 -- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd0003800 msgr2=0x7f4bd0003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:40.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd0003800 0x7f4bd0003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:40.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8151190 con 0x7f4bd0004e50 2020-07-18T03:25:40.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd0003800 0x7f4bd0003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:40.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf80634e0 con 0x7f4bd0004e50 2020-07-18T03:25:40.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf8065040 con 0x7f4bd0004e50 2020-07-18T03:25:40.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 -- 172.21.15.47:0/2565008144 reap_dead start 2020-07-18T03:25:40.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bf7fff700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd0004e50 0x7f4bd000b730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4bf00058d0 tx=0x7f4bf0007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf0005950 con 0x7f4bd0004e50 2020-07-18T03:25:40.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0005ab0 con 0x7f4bd0004e50 2020-07-18T03:25:40.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.857+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf0005e10 con 0x7f4bd0004e50 2020-07-18T03:25:40.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.858+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf80adb40 con 0x7f4bd0004e50 2020-07-18T03:25:40.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.858+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4bf0010030 con 0x7f4bd0004e50 2020-07-18T03:25:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.858+0000 7f4bd77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4bd0053de0 0x7f4bd00562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.858+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4bf0067f20 con 0x7f4bd0004e50 2020-07-18T03:25:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.858+0000 7f4bf77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4bd0053de0 0x7f4bd00562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.859+0000 7f4bf77fe700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4bd0053de0 0x7f4bd00562a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4be8016000 tx=0x7f4be8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:40.862+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4bf0040ac0 con 0x7f4bd0004e50 2020-07-18T03:25:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.017+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4bf805b120 con 0x7f4bd0004e50 2020-07-18T03:25:41.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.520+0000 7f4bd77fe700 1 -- 172.21.15.47:0/2565008144 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4bf00440e0 con 0x7f4bd0004e50 2020-07-18T03:25:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4bd0053de0 msgr2=0x7f4bd00562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4bd0053de0 0x7f4bd00562a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4be8016000 tx=0x7f4be8015000).stop 2020-07-18T03:25:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd0004e50 msgr2=0x7f4bd000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd0004e50 0x7f4bd000b730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4bf00058d0 tx=0x7f4bf0007b50).stop 2020-07-18T03:25:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 shutdown_connections 2020-07-18T03:25:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4bd0053de0 0x7f4bd00562a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 --2- 172.21.15.47:0/2565008144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bd0004e50 0x7f4bd000b730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.523+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 >> 172.21.15.47:0/2565008144 conn(0x7f4bf805f5a0 msgr2=0x7f4bf80b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.524+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 shutdown_connections 2020-07-18T03:25:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.524+0000 7f4bfdfdd700 1 -- 172.21.15.47:0/2565008144 wait complete. 2020-07-18T03:25:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2020-07-18T03:25:41.537 INFO:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2020-07-18T03:25:41.537 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2020-07-18T03:25:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.661+0000 7f2ea44f5700 1 Processor -- start 2020-07-18T03:25:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.661+0000 7f2ea44f5700 1 -- start start 2020-07-18T03:25:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea44f5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea44f5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e9c0b6300 0x7f2e9c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea44f5700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c147de0 con 0x7f2e9c0ad330 2020-07-18T03:25:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea44f5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0ae000 con 0x7f2e9c0b6300 2020-07-18T03:25:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea44f5700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c148010 con 0x7f2e9c0b59b0 2020-07-18T03:25:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e9c0b6300 0x7f2e9c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e9c0b6300 0x7f2e9c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58210/0 (socket says 172.21.15.47:58210) 2020-07-18T03:25:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 -- 172.21.15.47:0/1881185143 learned_addr learned my addr 172.21.15.47:0/1881185143 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 -- 172.21.15.47:0/1881185143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 msgr2=0x7f2e9c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c0b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 -- 172.21.15.47:0/1881185143 >> v1:172.21.15.18:6789/0 conn(0x7f2e9c0ad330 legacy=0x7f2e9c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 -- 172.21.15.47:0/1881185143 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0ad710 con 0x7f2e9c0b6300 2020-07-18T03:25:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.662+0000 7f2ea2a92700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e9c0b6300 0x7f2e9c0b3a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f2e940032b0 tx=0x7f2e94009270).ready entity=mon.1 client_cookie=d2b7898a54730a2b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea128f700 1 -- 172.21.15.47:0/1881185143 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e9400bc00 con 0x7f2e9c0b6300 2020-07-18T03:25:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea128f700 1 -- 172.21.15.47:0/1881185143 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9400bd60 con 0x7f2e9c0b6300 2020-07-18T03:25:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea128f700 1 -- 172.21.15.47:0/1881185143 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e94011400 con 0x7f2e9c0b6300 2020-07-18T03:25:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 -- 172.21.15.47:0/1881185143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e9c0b6300 msgr2=0x7f2e9c0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e9c0b6300 0x7f2e9c0b3a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f2e940032b0 tx=0x7f2e94009270).stop 2020-07-18T03:25:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 -- 172.21.15.47:0/1881185143 shutdown_connections 2020-07-18T03:25:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e9c0b6300 0x7f2e9c0b3a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 --2- 172.21.15.47:0/1881185143 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 -- 172.21.15.47:0/1881185143 >> 172.21.15.47:0/1881185143 conn(0x7f2e9c05f5a0 msgr2=0x7f2e9c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 -- 172.21.15.47:0/1881185143 shutdown_connections 2020-07-18T03:25:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.663+0000 7f2ea44f5700 1 -- 172.21.15.47:0/1881185143 wait complete. 2020-07-18T03:25:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.664+0000 7f2ea44f5700 1 Processor -- start 2020-07-18T03:25:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.664+0000 7f2ea44f5700 1 -- start start 2020-07-18T03:25:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea44f5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2e9c0ad330 0x7f2e9c066500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea44f5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c063070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea44f5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c13b520 con 0x7f2e9c139300 2020-07-18T03:25:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea44f5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c145a30 con 0x7f2e9c0ad330 2020-07-18T03:25:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea44f5700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c13b750 con 0x7f2e9c0b59b0 2020-07-18T03:25:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea2a92700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2e9c139300 0x7f2e9c0b2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39304/0 (socket says 172.21.15.47:39304) 2020-07-18T03:25:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea2a92700 1 -- 172.21.15.47:0/3760125380 learned_addr learned my addr 172.21.15.47:0/3760125380 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c063070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2ea2291700 1 --2- 172.21.15.47:0/3760125380 >> v2:172.21.15.18:3300/0 conn(0x7f2e9c0ad330 0x7f2e9c066500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159763624 0 0) 0x7f2e9c13b520 con 0x7f2e9c139300 2020-07-18T03:25:41.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e74003240 con 0x7f2e9c139300 2020-07-18T03:25:41.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 657076576 0 0) 0x7f2e74003240 con 0x7f2e9c139300 2020-07-18T03:25:41.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e9c13b520 con 0x7f2e9c139300 2020-07-18T03:25:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.665+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f2e9400bc00 con 0x7f2e9c139300 2020-07-18T03:25:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 msgr2=0x7f2e9c063070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e9c063070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 >> v2:172.21.15.18:3300/0 conn(0x7f2e9c0ad330 msgr2=0x7f2e9c066500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 --2- 172.21.15.47:0/3760125380 >> v2:172.21.15.18:3300/0 conn(0x7f2e9c0ad330 0x7f2e9c066500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 >> v1:172.21.15.47:6789/0 conn(0x7f2e9c139300 legacy=0x7f2e9c0b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e74003800 0x7f2e74003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740044e0 0x7f2e74004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e74004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c145a30 con 0x7f2e740044e0 2020-07-18T03:25:41.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c13b750 con 0x7f2e74003800 2020-07-18T03:25:41.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74005300 con 0x7f2e9c0b59b0 2020-07-18T03:25:41.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e74003800 0x7f2e74003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2a92700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e74004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740044e0 0x7f2e74004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 -- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 msgr2=0x7f2e74004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e9c0b59b0 0x7f2e74004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 -- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740044e0 msgr2=0x7f2e74004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:41.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740044e0 0x7f2e74004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:41.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c13b9f0 con 0x7f2e74003800 2020-07-18T03:25:41.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c0b3020 con 0x7f2e74003800 2020-07-18T03:25:41.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740044e0 0x7f2e74004920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:41.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c0632b0 con 0x7f2e74003800 2020-07-18T03:25:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2ea2291700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e74003800 0x7f2e74003c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f2e8c00d000 tx=0x7f2e8c000b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e8c00da10 con 0x7f2e74003800 2020-07-18T03:25:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c00db70 con 0x7f2e74003800 2020-07-18T03:25:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.666+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e8c01c3f0 con 0x7f2e74003800 2020-07-18T03:25:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.667+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c146e70 con 0x7f2e74003800 2020-07-18T03:25:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.667+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2e8c0133f0 con 0x7f2e74003800 2020-07-18T03:25:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.668+0000 7f2e8a7fc700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e74058170 0x7f2e7405a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.668+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f2e8c067360 con 0x7f2e74003800 2020-07-18T03:25:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.668+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e74058170 0x7f2e7405a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.668+0000 7f2ea1a90700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e74058170 0x7f2e7405a630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2e9800c000 tx=0x7f2e98008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.671+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2e8c03fdc0 con 0x7f2e74003800 2020-07-18T03:25:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:41.841+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f2e9c0adb40 con 0x7f2e74003800 2020-07-18T03:25:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.523+0000 7f2e8a7fc700 1 -- 172.21.15.47:0/3760125380 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 154+0+0 (secure 0 0 0) 0x7f2e8c0433e0 con 0x7f2e74003800 2020-07-18T03:25:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.526+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e74058170 msgr2=0x7f2e7405a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.526+0000 7f2ea44f5700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e74058170 0x7f2e7405a630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2e9800c000 tx=0x7f2e98008040).stop 2020-07-18T03:25:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.526+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e74003800 msgr2=0x7f2e74003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.526+0000 7f2ea44f5700 1 --2- 172.21.15.47:0/3760125380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e74003800 0x7f2e74003c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f2e8c00d000 tx=0x7f2e8c000b80).stop 2020-07-18T03:25:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.526+0000 7f2ea2a92700 1 -- 172.21.15.47:0/3760125380 reap_dead start 2020-07-18T03:25:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.527+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 shutdown_connections 2020-07-18T03:25:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.527+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 >> 172.21.15.47:0/3760125380 conn(0x7f2e9c05f5a0 msgr2=0x7f2e9c0b9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.527+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 shutdown_connections 2020-07-18T03:25:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.528+0000 7f2ea44f5700 1 -- 172.21.15.47:0/3760125380 wait complete. 2020-07-18T03:25:42.542 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 erasure cephfs_data 2020-07-18T03:25:42.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f99960700 1 Processor -- start 2020-07-18T03:25:42.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f99960700 1 -- start start 2020-07-18T03:25:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f99960700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f9413efc0 0x7f7f9405f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f99960700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f94149120 con 0x7f7f94148d40 2020-07-18T03:25:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f99960700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f94144f50 con 0x7f7f94145690 2020-07-18T03:25:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f99960700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f94144d60 con 0x7f7f9413efc0 2020-07-18T03:25:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f93fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7f94145690 0x7f7f9413e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39314/0 (socket says 172.21.15.47:39314) 2020-07-18T03:25:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.667+0000 7f7f93fff700 1 -- 172.21.15.47:0/2136675377 learned_addr learned my addr 172.21.15.47:0/2136675377 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114073105 0 0) 0x7f7f94144f50 con 0x7f7f94145690 2020-07-18T03:25:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f9915f700 1 --2- 172.21.15.47:0/2136675377 >> v2:172.21.15.18:3300/0 conn(0x7f7f9413efc0 0x7f7f9405f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f80003240 con 0x7f7f94145690 2020-07-18T03:25:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1753934160 0 0) 0x7f7f80003240 con 0x7f7f94145690 2020-07-18T03:25:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 >> v2:172.21.15.18:3300/0 conn(0x7f7f9413efc0 msgr2=0x7f7f9405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 --2- 172.21.15.47:0/2136675377 >> v2:172.21.15.18:3300/0 conn(0x7f7f9413efc0 0x7f7f9405f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 >> v1:172.21.15.18:6789/0 conn(0x7f7f94148d40 legacy=0x7f7f940619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f941451f0 con 0x7f7f94145690 2020-07-18T03:25:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7f880016d0 con 0x7f7f94145690 2020-07-18T03:25:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 >> v1:172.21.15.47:6789/0 conn(0x7f7f94145690 legacy=0x7f7f9413e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80003730 0x7f7f80003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f800044e0 0x7f7f80004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f80004e50 0x7f7f8000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80004320 con 0x7f7f80003730 2020-07-18T03:25:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8000bc20 con 0x7f7f80004e50 2020-07-18T03:25:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8000bd90 con 0x7f7f800044e0 2020-07-18T03:25:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f9915f700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f80004e50 0x7f7f8000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f9895e700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80003730 0x7f7f80003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.668+0000 7f7f93fff700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f800044e0 0x7f7f80004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 -- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f800044e0 msgr2=0x7f7f80004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f800044e0 0x7f7f80004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 -- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80003730 msgr2=0x7f7f80003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80003730 0x7f7f80003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 -- 172.21.15.47:0/2136675377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f80003240 con 0x7f7f80004e50 2020-07-18T03:25:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9895e700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80003730 0x7f7f80003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 -- 172.21.15.47:0/2136675377 reap_dead start 2020-07-18T03:25:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f9915f700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f80004e50 0x7f7f8000b730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f7f8c00c000 tx=0x7f7f8c0091d0).ready entity=mon.1 client_cookie=115de906936402d8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8c00eb80 con 0x7f7f80004e50 2020-07-18T03:25:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00ece0 con 0x7f7f80004e50 2020-07-18T03:25:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f937fe700 1 -- 172.21.15.47:0/2136675377 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8c0125e0 con 0x7f7f80004e50 2020-07-18T03:25:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 -- 172.21.15.47:0/2136675377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f80004e50 msgr2=0x7f7f8000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f80004e50 0x7f7f8000b730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f7f8c00c000 tx=0x7f7f8c0091d0).stop 2020-07-18T03:25:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 -- 172.21.15.47:0/2136675377 shutdown_connections 2020-07-18T03:25:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 --2- 172.21.15.47:0/2136675377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f80004e50 0x7f7f8000b730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 -- 172.21.15.47:0/2136675377 >> 172.21.15.47:0/2136675377 conn(0x7f7f940b91d0 msgr2=0x7f7f940b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 -- 172.21.15.47:0/2136675377 shutdown_connections 2020-07-18T03:25:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.669+0000 7f7f99960700 1 -- 172.21.15.47:0/2136675377 wait complete. 2020-07-18T03:25:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.670+0000 7f7f99960700 1 Processor -- start 2020-07-18T03:25:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f99960700 1 -- start start 2020-07-18T03:25:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f99960700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f940b6540 0x7f7f940af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f99960700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940afb70 con 0x7f7f940b6920 2020-07-18T03:25:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f99960700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f94141240 con 0x7f7f9413a4a0 2020-07-18T03:25:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f99960700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f941414e0 con 0x7f7f940b6540 2020-07-18T03:25:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f9895e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f940b6540 0x7f7f940af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f9895e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f940b6540 0x7f7f940af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58238/0 (socket says 172.21.15.47:58238) 2020-07-18T03:25:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f9915f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7f9413a4a0 0x7f7f94141980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39328/0 (socket says 172.21.15.47:39328) 2020-07-18T03:25:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.671+0000 7f7f9895e700 1 -- 172.21.15.47:0/2428046286 learned_addr learned my addr 172.21.15.47:0/2428046286 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f9895e700 1 -- 172.21.15.47:0/2428046286 >> v1:172.21.15.47:6789/0 conn(0x7f7f9413a4a0 legacy=0x7f7f94141980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f9895e700 1 -- 172.21.15.47:0/2428046286 >> v1:172.21.15.18:6789/0 conn(0x7f7f940b6920 legacy=0x7f7f94139d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469164553 0 0) 0x7f7f94141240 con 0x7f7f9413a4a0 2020-07-18T03:25:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f9895e700 1 -- 172.21.15.47:0/2428046286 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94147f70 con 0x7f7f940b6540 2020-07-18T03:25:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f9895e700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f940b6540 0x7f7f940af540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7f8800b000 tx=0x7f7f84009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8400d990 con 0x7f7f940b6540 2020-07-18T03:25:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8400daf0 con 0x7f7f940b6540 2020-07-18T03:25:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f94148220 con 0x7f7f940b6540 2020-07-18T03:25:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8400de50 con 0x7f7f940b6540 2020-07-18T03:25:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.672+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f941422d0 con 0x7f7f940b6540 2020-07-18T03:25:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.673+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f940ad390 con 0x7f7f940b6540 2020-07-18T03:25:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.673+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7f840133f0 con 0x7f7f940b6540 2020-07-18T03:25:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.673+0000 7f7f90ff9700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f6c04fd60 0x7f7f6c052220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.673+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5863+0+0 (secure 0 0 0) 0x7f7f84067540 con 0x7f7f940b6540 2020-07-18T03:25:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.673+0000 7f7f93fff700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f6c04fd60 0x7f7f6c052220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.674+0000 7f7f93fff700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f6c04fd60 0x7f7f6c052220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7f880013e0 tx=0x7f7f940ae120).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.677+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7f8403fea0 con 0x7f7f940b6540 2020-07-18T03:25:42.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:42.834+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7f7f940690d0 con 0x7f7f940b6540 2020-07-18T03:25:44.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.530+0000 7f7f90ff9700 1 -- 172.21.15.47:0/2428046286 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f7f840434c0 con 0x7f7f940b6540 2020-07-18T03:25:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.535+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f6c04fd60 msgr2=0x7f7f6c052220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.535+0000 7f7f99960700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f6c04fd60 0x7f7f6c052220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7f880013e0 tx=0x7f7f940ae120).stop 2020-07-18T03:25:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.535+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f940b6540 msgr2=0x7f7f940af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.535+0000 7f7f99960700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f940b6540 0x7f7f940af540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7f8800b000 tx=0x7f7f84009040).stop 2020-07-18T03:25:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.536+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 shutdown_connections 2020-07-18T03:25:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.536+0000 7f7f99960700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f6c04fd60 0x7f7f6c052220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.536+0000 7f7f99960700 1 --2- 172.21.15.47:0/2428046286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f940b6540 0x7f7f940af540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.536+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 >> 172.21.15.47:0/2428046286 conn(0x7f7f940b91d0 msgr2=0x7f7f9413e780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.536+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 shutdown_connections 2020-07-18T03:25:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.536+0000 7f7f99960700 1 -- 172.21.15.47:0/2428046286 wait complete. 2020-07-18T03:25:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2020-07-18T03:25:44.550 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool set cephfs_data allow_ec_overwrites true 2020-07-18T03:25:44.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.675+0000 7f41583b8700 1 Processor -- start 2020-07-18T03:25:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.675+0000 7f41583b8700 1 -- start start 2020-07-18T03:25:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.676+0000 7f41583b8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f41500ad330 0x7f4150143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.676+0000 7f41583b8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4150147de0 con 0x7f41500b59b0 2020-07-18T03:25:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.676+0000 7f41583b8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41500ae000 con 0x7f41500b3a10 2020-07-18T03:25:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.676+0000 7f41583b8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4150148010 con 0x7f41500ad330 2020-07-18T03:25:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4156955700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f41500b3a10 0x7f41500b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39336/0 (socket says 172.21.15.47:39336) 2020-07-18T03:25:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4156955700 1 -- 172.21.15.47:0/3150887960 learned_addr learned my addr 172.21.15.47:0/3150887960 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4156154700 1 --2- 172.21.15.47:0/3150887960 >> v2:172.21.15.18:3300/0 conn(0x7f41500ad330 0x7f4150143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483974693 0 0) 0x7f41500ae000 con 0x7f41500b3a10 2020-07-18T03:25:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4138003240 con 0x7f41500b3a10 2020-07-18T03:25:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1548066302 0 0) 0x7f4138003240 con 0x7f41500b3a10 2020-07-18T03:25:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 >> v2:172.21.15.18:3300/0 conn(0x7f41500ad330 msgr2=0x7f4150143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 --2- 172.21.15.47:0/3150887960 >> v2:172.21.15.18:3300/0 conn(0x7f41500ad330 0x7f4150143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 >> v1:172.21.15.18:6789/0 conn(0x7f41500b59b0 legacy=0x7f41500b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500ad710 con 0x7f41500b3a10 2020-07-18T03:25:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4148001a50 con 0x7f41500b3a10 2020-07-18T03:25:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 >> v1:172.21.15.47:6789/0 conn(0x7f41500b3a10 legacy=0x7f41500b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4138003730 0x7f4138003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41380044e0 0x7f4138004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138004e50 0x7f413800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150148010 con 0x7f4138003730 2020-07-18T03:25:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413800bc20 con 0x7f4138004e50 2020-07-18T03:25:44.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413800bd60 con 0x7f41380044e0 2020-07-18T03:25:44.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4156955700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138004e50 0x7f413800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4156154700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4138003730 0x7f4138003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.677+0000 7f4155953700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41380044e0 0x7f4138004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 -- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41380044e0 msgr2=0x7f4138004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41380044e0 0x7f4138004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 -- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4138003730 msgr2=0x7f4138003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4138003730 0x7f4138003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 -- 172.21.15.47:0/3150887960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4138003240 con 0x7f4138004e50 2020-07-18T03:25:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 -- 172.21.15.47:0/3150887960 reap_dead start 2020-07-18T03:25:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4156955700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138004e50 0x7f413800b730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f4148001760 tx=0x7f4148007a50).ready entity=mon.1 client_cookie=ea2aa2d101a7c5bf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4148005b80 con 0x7f4138004e50 2020-07-18T03:25:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148005ce0 con 0x7f4138004e50 2020-07-18T03:25:44.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f4155152700 1 -- 172.21.15.47:0/3150887960 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41480072c0 con 0x7f4138004e50 2020-07-18T03:25:44.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 -- 172.21.15.47:0/3150887960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138004e50 msgr2=0x7f413800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138004e50 0x7f413800b730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f4148001760 tx=0x7f4148007a50).stop 2020-07-18T03:25:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 -- 172.21.15.47:0/3150887960 shutdown_connections 2020-07-18T03:25:44.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 --2- 172.21.15.47:0/3150887960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4138004e50 0x7f413800b730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 -- 172.21.15.47:0/3150887960 >> 172.21.15.47:0/3150887960 conn(0x7f415005f5a0 msgr2=0x7f415005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:44.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 -- 172.21.15.47:0/3150887960 shutdown_connections 2020-07-18T03:25:44.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.678+0000 7f41583b8700 1 -- 172.21.15.47:0/3150887960 wait complete. 2020-07-18T03:25:44.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.679+0000 7f41583b8700 1 Processor -- start 2020-07-18T03:25:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.679+0000 7f41583b8700 1 -- start start 2020-07-18T03:25:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f41583b8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41501471d0 0x7f4150066300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f41583b8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f41500667f0 0x7f4150062e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f41583b8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4150066ca0 con 0x7f41501581f0 2020-07-18T03:25:44.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f41583b8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4150158ce0 con 0x7f41500667f0 2020-07-18T03:25:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f41583b8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4150146240 con 0x7f41501471d0 2020-07-18T03:25:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f4155953700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41501471d0 0x7f4150066300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f4156955700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f41500667f0 0x7f4150062e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f4156154700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f41501581f0 0x7f41501585d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58112/0 (socket says 172.21.15.47:58112) 2020-07-18T03:25:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.680+0000 7f4156154700 1 -- 172.21.15.47:0/3257288563 learned_addr learned my addr 172.21.15.47:0/3257288563 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f4156955700 1 -- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41501471d0 msgr2=0x7f4150066300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f4156955700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41501471d0 0x7f4150066300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f4156955700 1 -- 172.21.15.47:0/3257288563 >> v1:172.21.15.18:6789/0 conn(0x7f41501581f0 legacy=0x7f41501585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067376697 0 0) 0x7f414400b380 con 0x7f41501581f0 2020-07-18T03:25:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f4156955700 1 -- 172.21.15.47:0/3257288563 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41501464e0 con 0x7f41500667f0 2020-07-18T03:25:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f4155953700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41501471d0 0x7f4150066300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f4156955700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41500667f0 0x7f4150062e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4148001760 tx=0x7f4148015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4148005b80 con 0x7f41500667f0 2020-07-18T03:25:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148007800 con 0x7f41500667f0 2020-07-18T03:25:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4150146d30 con 0x7f41500667f0 2020-07-18T03:25:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4148020630 con 0x7f41500667f0 2020-07-18T03:25:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.681+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4150143640 con 0x7f41500667f0 2020-07-18T03:25:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.682+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415005b120 con 0x7f41500667f0 2020-07-18T03:25:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.682+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4148026020 con 0x7f41500667f0 2020-07-18T03:25:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.683+0000 7f41427fc700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f412804fd50 0x7f4128052210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:44.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.683+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f4148069610 con 0x7f41500667f0 2020-07-18T03:25:44.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.683+0000 7f4156154700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f412804fd50 0x7f4128052210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.683+0000 7f4156154700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f412804fd50 0x7f4128052210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4144007000 tx=0x7f4144008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.687+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4148041cf0 con 0x7f41500667f0 2020-07-18T03:25:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:44.844+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f41500690d0 con 0x7f41500667f0 2020-07-18T03:25:45.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.532+0000 7f41427fc700 1 -- 172.21.15.47:0/3257288563 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 166+0+0 (secure 0 0 0) 0x7f4148045310 con 0x7f41500667f0 2020-07-18T03:25:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f412804fd50 msgr2=0x7f4128052210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f412804fd50 0x7f4128052210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4144007000 tx=0x7f4144008040).stop 2020-07-18T03:25:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41500667f0 msgr2=0x7f4150062e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41500667f0 0x7f4150062e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4148001760 tx=0x7f4148015000).stop 2020-07-18T03:25:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 shutdown_connections 2020-07-18T03:25:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f412804fd50 0x7f4128052210 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0x7f4144007000 tx=0x7f4144008040).stop 2020-07-18T03:25:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41500667f0 0x7f4150062e80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 --2- 172.21.15.47:0/3257288563 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41501471d0 0x7f4150066300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 >> 172.21.15.47:0/3257288563 conn(0x7f415005f5a0 msgr2=0x7f4150061c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:45.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 shutdown_connections 2020-07-18T03:25:45.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.536+0000 7f41583b8700 1 -- 172.21.15.47:0/3257288563 wait complete. 2020-07-18T03:25:45.539 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 allow_ec_overwrites to true 2020-07-18T03:25:45.552 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-07-18T03:25:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.680+0000 7fca693b3700 1 Processor -- start 2020-07-18T03:25:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.680+0000 7fca693b3700 1 -- start start 2020-07-18T03:25:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.680+0000 7fca693b3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fca640ad330 0x7fca64143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.680+0000 7fca693b3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca640b59b0 0x7fca640b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.680+0000 7fca693b3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca64147de0 con 0x7fca640b6300 2020-07-18T03:25:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca640ae000 con 0x7fca640ad330 2020-07-18T03:25:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca64148010 con 0x7fca640b59b0 2020-07-18T03:25:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca640b59b0 0x7fca640b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca640b59b0 0x7fca640b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58266/0 (socket says 172.21.15.47:58266) 2020-07-18T03:25:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 -- 172.21.15.47:0/3924111708 learned_addr learned my addr 172.21.15.47:0/3924111708 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca62ffd700 1 --2- 172.21.15.47:0/3924111708 >> v2:172.21.15.18:3300/0 conn(0x7fca640ad330 0x7fca64143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 -- 172.21.15.47:0/3924111708 >> v2:172.21.15.18:3300/0 conn(0x7fca640ad330 msgr2=0x7fca64143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 --2- 172.21.15.47:0/3924111708 >> v2:172.21.15.18:3300/0 conn(0x7fca640ad330 0x7fca64143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 -- 172.21.15.47:0/3924111708 >> v1:172.21.15.18:6789/0 conn(0x7fca640b6300 legacy=0x7fca640b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:45.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 -- 172.21.15.47:0/3924111708 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640ad710 con 0x7fca640b59b0 2020-07-18T03:25:45.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca627fc700 1 --2- 172.21.15.47:0/3924111708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca640b59b0 0x7fca640b5e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fca640ae4e0 tx=0x7fca54006e90).ready entity=mon.1 client_cookie=644f135775c30cee server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:45.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca61ffb700 1 -- 172.21.15.47:0/3924111708 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca54009930 con 0x7fca640b59b0 2020-07-18T03:25:45.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca61ffb700 1 -- 172.21.15.47:0/3924111708 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca54009a90 con 0x7fca640b59b0 2020-07-18T03:25:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca61ffb700 1 -- 172.21.15.47:0/3924111708 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca54009e30 con 0x7fca640b59b0 2020-07-18T03:25:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- 172.21.15.47:0/3924111708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca640b59b0 msgr2=0x7fca640b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 --2- 172.21.15.47:0/3924111708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca640b59b0 0x7fca640b5e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fca640ae4e0 tx=0x7fca54006e90).stop 2020-07-18T03:25:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- 172.21.15.47:0/3924111708 shutdown_connections 2020-07-18T03:25:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 --2- 172.21.15.47:0/3924111708 >> v2:172.21.15.18:3300/0 conn(0x7fca640ad330 0x7fca64143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 --2- 172.21.15.47:0/3924111708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca640b59b0 0x7fca640b5e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- 172.21.15.47:0/3924111708 >> 172.21.15.47:0/3924111708 conn(0x7fca6405f5a0 msgr2=0x7fca6405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- 172.21.15.47:0/3924111708 shutdown_connections 2020-07-18T03:25:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.681+0000 7fca693b3700 1 -- 172.21.15.47:0/3924111708 wait complete. 2020-07-18T03:25:45.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.683+0000 7fca693b3700 1 Processor -- start 2020-07-18T03:25:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.683+0000 7fca693b3700 1 -- start start 2020-07-18T03:25:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fca640b6300 0x7fca64062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca641531c0 0x7fca64153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca64146e20 con 0x7fca640ad330 2020-07-18T03:25:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca64112a90 con 0x7fca640b6300 2020-07-18T03:25:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca64112350 con 0x7fca641531c0 2020-07-18T03:25:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca641531c0 0x7fca64153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca641531c0 0x7fca64153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58276/0 (socket says 172.21.15.47:58276) 2020-07-18T03:25:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 -- 172.21.15.47:0/2975077208 learned_addr learned my addr 172.21.15.47:0/2975077208 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca627fc700 1 --2- 172.21.15.47:0/2975077208 >> v2:172.21.15.18:3300/0 conn(0x7fca640b6300 0x7fca64062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 -- 172.21.15.47:0/2975077208 >> v2:172.21.15.18:3300/0 conn(0x7fca640b6300 msgr2=0x7fca64062e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 --2- 172.21.15.47:0/2975077208 >> v2:172.21.15.18:3300/0 conn(0x7fca640b6300 0x7fca64062e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 -- 172.21.15.47:0/2975077208 >> v1:172.21.15.18:6789/0 conn(0x7fca640ad330 legacy=0x7fca64061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 -- 172.21.15.47:0/2975077208 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca641125f0 con 0x7fca641531c0 2020-07-18T03:25:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca637fe700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca641531c0 0x7fca64153610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fca64065780 tx=0x7fca5c007f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca5c00a770 con 0x7fca641531c0 2020-07-18T03:25:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c00a8d0 con 0x7fca641531c0 2020-07-18T03:25:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca5c00aa40 con 0x7fca641531c0 2020-07-18T03:25:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca64146aa0 con 0x7fca641531c0 2020-07-18T03:25:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.684+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca640663d0 con 0x7fca641531c0 2020-07-18T03:25:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.685+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6405b120 con 0x7fca641531c0 2020-07-18T03:25:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.686+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fca5c010070 con 0x7fca641531c0 2020-07-18T03:25:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.686+0000 7fca4affd700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fca4004ffc0 0x7fca40052480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.686+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7fca5c067800 con 0x7fca641531c0 2020-07-18T03:25:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.686+0000 7fca62ffd700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fca4004ffc0 0x7fca40052480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.686+0000 7fca62ffd700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fca4004ffc0 0x7fca40052480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fca5800a000 tx=0x7fca58008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.689+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fca5c040000 con 0x7fca641531c0 2020-07-18T03:25:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.897+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fca64146c90 con 0x7fca641531c0 2020-07-18T03:25:45.899 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:25:45.897+0000 7f96cf9a3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:25:45.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.902+0000 7fca4affd700 1 -- 172.21.15.47:0/2975077208 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v2) v1 ==== 187+0+0 (secure 0 0 0) 0x7fca5c043620 con 0x7fca641531c0 2020-07-18T03:25:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fca4004ffc0 msgr2=0x7fca40052480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fca4004ffc0 0x7fca40052480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fca5800a000 tx=0x7fca58008040).stop 2020-07-18T03:25:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca641531c0 msgr2=0x7fca64153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca641531c0 0x7fca64153610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fca64065780 tx=0x7fca5c007f00).stop 2020-07-18T03:25:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 shutdown_connections 2020-07-18T03:25:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fca4004ffc0 0x7fca40052480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 --2- 172.21.15.47:0/2975077208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca641531c0 0x7fca64153610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 --2- 172.21.15.47:0/2975077208 >> v2:172.21.15.18:3300/0 conn(0x7fca640b6300 0x7fca64062e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 >> 172.21.15.47:0/2975077208 conn(0x7fca6405f5a0 msgr2=0x7fca64142950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 shutdown_connections 2020-07-18T03:25:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:45.904+0000 7fca693b3700 1 -- 172.21.15.47:0/2975077208 wait complete. 2020-07-18T03:25:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 3 and data pool 4 2020-07-18T03:25:45.915 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.028+0000 7f7089bf6700 1 Processor -- start 2020-07-18T03:25:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.028+0000 7f7089bf6700 1 -- start start 2020-07-18T03:25:46.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7089bf6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 0x7f7084143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7089bf6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084147330 con 0x7f7084147a00 2020-07-18T03:25:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7089bf6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840b57e0 con 0x7f70840ad100 2020-07-18T03:25:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7089bf6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840ae530 con 0x7f7084147e40 2020-07-18T03:25:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f70837fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f70840ad100 0x7f70840b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39368/0 (socket says 172.21.15.47:39368) 2020-07-18T03:25:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f70837fe700 1 -- 172.21.15.47:0/568069418 learned_addr learned my addr 172.21.15.47:0/568069418 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7083fff700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 0x7f7084143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433544007 0 0) 0x7f70840b57e0 con 0x7f70840ad100 2020-07-18T03:25:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7064003240 con 0x7f70840ad100 2020-07-18T03:25:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3096751406 0 0) 0x7f7064003240 con 0x7f70840ad100 2020-07-18T03:25:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 msgr2=0x7f7084143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 0x7f7084143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 >> v1:172.21.15.18:6789/0 conn(0x7f7084147a00 legacy=0x7f70841433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840b5a80 con 0x7f70840ad100 2020-07-18T03:25:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f70740016d0 con 0x7f70840ad100 2020-07-18T03:25:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 >> v1:172.21.15.47:6789/0 conn(0x7f70840ad100 legacy=0x7f70840b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 0x7f7064003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064003c40 0x7f70640044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70640049d0 0x7f7064004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840ae530 con 0x7f7064003c40 2020-07-18T03:25:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70640041d0 con 0x7f70640049d0 2020-07-18T03:25:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.029+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706400b730 con 0x7f7084147e40 2020-07-18T03:25:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f70837fe700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 0x7f7064003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f707bfff700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064003c40 0x7f70640044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70640049d0 0x7f7064004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 -- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 msgr2=0x7f7064003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:46.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7084147e40 0x7f7064003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 -- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064003c40 msgr2=0x7f70640044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:46.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064003c40 0x7f70640044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 -- 172.21.15.47:0/568069418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064003240 con 0x7f70640049d0 2020-07-18T03:25:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70640049d0 0x7f7064004e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f707c00c000 tx=0x7f707c0091d0).ready entity=mon.1 client_cookie=24992cb2f6272d95 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f707c00eb20 con 0x7f70640049d0 2020-07-18T03:25:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c00ec80 con 0x7f70640049d0 2020-07-18T03:25:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7082ffd700 1 -- 172.21.15.47:0/568069418 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f707c0123f0 con 0x7f70640049d0 2020-07-18T03:25:46.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7089bf6700 1 -- 172.21.15.47:0/568069418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70640049d0 msgr2=0x7f7064004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7089bf6700 1 --2- 172.21.15.47:0/568069418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70640049d0 0x7f7064004e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f707c00c000 tx=0x7f707c0091d0).stop 2020-07-18T03:25:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7083fff700 1 -- 172.21.15.47:0/568069418 reap_dead start 2020-07-18T03:25:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7089bf6700 1 -- 172.21.15.47:0/568069418 shutdown_connections 2020-07-18T03:25:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7089bf6700 1 -- 172.21.15.47:0/568069418 >> 172.21.15.47:0/568069418 conn(0x7f70840b9340 msgr2=0x7f70840b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:46.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7089bf6700 1 -- 172.21.15.47:0/568069418 shutdown_connections 2020-07-18T03:25:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.030+0000 7f7089bf6700 1 -- 172.21.15.47:0/568069418 wait complete. 2020-07-18T03:25:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 Processor -- start 2020-07-18T03:25:46.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 -- start start 2020-07-18T03:25:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70840b1920 0x7f7084145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7084145e80 0x7f708415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840b7bb0 con 0x7f70841517a0 2020-07-18T03:25:46.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70840b79d0 con 0x7f7084145e80 2020-07-18T03:25:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7089bf6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840b7e50 con 0x7f70840b1920 2020-07-18T03:25:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f70837fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f70841517a0 0x7f70840b7230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39380/0 (socket says 172.21.15.47:39380) 2020-07-18T03:25:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f70837fe700 1 -- 172.21.15.47:0/2278441041 learned_addr learned my addr 172.21.15.47:0/2278441041 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f7083fff700 1 --2- 172.21.15.47:0/2278441041 >> v2:172.21.15.47:3300/0 conn(0x7f7084145e80 0x7f708415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.032+0000 7f707bfff700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70840b1920 0x7f7084145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610057438 0 0) 0x7f70840b7bb0 con 0x7f70841517a0 2020-07-18T03:25:46.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7060003240 con 0x7f70841517a0 2020-07-18T03:25:46.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7083fff700 1 -- 172.21.15.47:0/2278441041 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70840b1920 msgr2=0x7f7084145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7083fff700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70840b1920 0x7f7084145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7083fff700 1 -- 172.21.15.47:0/2278441041 >> v1:172.21.15.47:6789/0 conn(0x7f70841517a0 legacy=0x7f70840b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7083fff700 1 -- 172.21.15.47:0/2278441041 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840b80f0 con 0x7f7084145e80 2020-07-18T03:25:46.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f707bfff700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70840b1920 0x7f7084145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:46.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7083fff700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7084145e80 0x7f708415a360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f70841462d0 tx=0x7f707c01a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f707c012870 con 0x7f7084145e80 2020-07-18T03:25:46.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c0129d0 con 0x7f7084145e80 2020-07-18T03:25:46.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f707c012d30 con 0x7f7084145e80 2020-07-18T03:25:46.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7084150fd0 con 0x7f7084145e80 2020-07-18T03:25:46.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.033+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708415a850 con 0x7f7084145e80 2020-07-18T03:25:46.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.034+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7084057450 con 0x7f7084145e80 2020-07-18T03:25:46.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.034+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f707c02a020 con 0x7f7084145e80 2020-07-18T03:25:46.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.035+0000 7f707b7fe700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f70600503b0 0x7f7060052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.035+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f707c0699e0 con 0x7f7084145e80 2020-07-18T03:25:46.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.035+0000 7f70837fe700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f70600503b0 0x7f7060052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.035+0000 7f70837fe700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f70600503b0 0x7f7060052870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f707400b000 tx=0x7f70840b03e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.038+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f707c0421b0 con 0x7f7084145e80 2020-07-18T03:25:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.193+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f708406a100 con 0x7f7084145e80 2020-07-18T03:25:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.194+0000 7f707b7fe700 1 -- 172.21.15.47:0/2278441041 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f707c045790 con 0x7f7084145e80 2020-07-18T03:25:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:46.196 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f70600503b0 msgr2=0x7f7060052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f70600503b0 0x7f7060052870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f707400b000 tx=0x7f70840b03e0).stop 2020-07-18T03:25:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7084145e80 msgr2=0x7f708415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7084145e80 0x7f708415a360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f70841462d0 tx=0x7f707c01a000).stop 2020-07-18T03:25:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 shutdown_connections 2020-07-18T03:25:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7084145e80 0x7f708415a360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f70600503b0 0x7f7060052870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 --2- 172.21.15.47:0/2278441041 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70840b1920 0x7f7084145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.195+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 >> 172.21.15.47:0/2278441041 conn(0x7f70840b9340 msgr2=0x7f708405f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.196+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 shutdown_connections 2020-07-18T03:25:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.196+0000 7f7089bf6700 1 -- 172.21.15.47:0/2278441041 wait complete. 2020-07-18T03:25:46.207 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.322+0000 7fec68939700 1 Processor -- start 2020-07-18T03:25:46.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.322+0000 7fec68939700 1 -- start start 2020-07-18T03:25:46.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec68939700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec64147a00 0x7fec64147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec68939700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec64147330 con 0x7fec641433e0 2020-07-18T03:25:46.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec68939700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec640b57e0 con 0x7fec640ad100 2020-07-18T03:25:46.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec68939700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec640ae530 con 0x7fec64147a00 2020-07-18T03:25:46.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec6259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fec640ad100 0x7fec640b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39388/0 (socket says 172.21.15.47:39388) 2020-07-18T03:25:46.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec6259c700 1 -- 172.21.15.47:0/3837518468 learned_addr learned my addr 172.21.15.47:0/3837518468 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:46.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec61d9b700 1 --2- 172.21.15.47:0/3837518468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec64147a00 0x7fec64147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876947510 0 0) 0x7fec640b57e0 con 0x7fec640ad100 2020-07-18T03:25:46.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.323+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec48003240 con 0x7fec640ad100 2020-07-18T03:25:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356922009 0 0) 0x7fec64147330 con 0x7fec641433e0 2020-07-18T03:25:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec640b57e0 con 0x7fec641433e0 2020-07-18T03:25:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3282676649 0 0) 0x7fec48003240 con 0x7fec640ad100 2020-07-18T03:25:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec61d9b700 1 -- 172.21.15.47:0/3837518468 >> v1:172.21.15.47:6789/0 conn(0x7fec640ad100 legacy=0x7fec640b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec61d9b700 1 -- 172.21.15.47:0/3837518468 >> v1:172.21.15.18:6789/0 conn(0x7fec641433e0 legacy=0x7fec64143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec61d9b700 1 -- 172.21.15.47:0/3837518468 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640b5a80 con 0x7fec64147a00 2020-07-18T03:25:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec61d9b700 1 --2- 172.21.15.47:0/3837518468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec64147a00 0x7fec64147e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fec580010d0 tx=0x7fec540091d0).ready entity=mon.2 client_cookie=cd9f61ccff218faa server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec580016d0 con 0x7fec64147a00 2020-07-18T03:25:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5400b880 con 0x7fec64147a00 2020-07-18T03:25:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec6159a700 1 -- 172.21.15.47:0/3837518468 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec5400b9f0 con 0x7fec64147a00 2020-07-18T03:25:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 -- 172.21.15.47:0/3837518468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec64147a00 msgr2=0x7fec64147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 --2- 172.21.15.47:0/3837518468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec64147a00 0x7fec64147e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fec580010d0 tx=0x7fec540091d0).stop 2020-07-18T03:25:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 -- 172.21.15.47:0/3837518468 shutdown_connections 2020-07-18T03:25:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 --2- 172.21.15.47:0/3837518468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec64147a00 0x7fec64147e40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 -- 172.21.15.47:0/3837518468 >> 172.21.15.47:0/3837518468 conn(0x7fec640b9340 msgr2=0x7fec640b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 -- 172.21.15.47:0/3837518468 shutdown_connections 2020-07-18T03:25:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.324+0000 7fec68939700 1 -- 172.21.15.47:0/3837518468 wait complete. 2020-07-18T03:25:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec68939700 1 Processor -- start 2020-07-18T03:25:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec68939700 1 -- start start 2020-07-18T03:25:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec68939700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec641433e0 0x7fec640b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec68939700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec64145f60 con 0x7fec64140600 2020-07-18T03:25:46.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec68939700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec640b1640 con 0x7fec640ad100 2020-07-18T03:25:46.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec68939700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec64146190 con 0x7fec641433e0 2020-07-18T03:25:46.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec6259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fec640ad100 0x7fec64155c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39394/0 (socket says 172.21.15.47:39394) 2020-07-18T03:25:46.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.326+0000 7fec6259c700 1 -- 172.21.15.47:0/3001511558 learned_addr learned my addr 172.21.15.47:0/3001511558 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec61d9b700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec641433e0 0x7fec640b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756348110 0 0) 0x7fec640b1640 con 0x7fec640ad100 2020-07-18T03:25:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec38003240 con 0x7fec640ad100 2020-07-18T03:25:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784874617 0 0) 0x7fec64145f60 con 0x7fec64140600 2020-07-18T03:25:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec640b1640 con 0x7fec64140600 2020-07-18T03:25:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec61d9b700 1 -- 172.21.15.47:0/3001511558 >> v1:172.21.15.47:6789/0 conn(0x7fec640ad100 legacy=0x7fec64155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec61d9b700 1 -- 172.21.15.47:0/3001511558 >> v1:172.21.15.18:6789/0 conn(0x7fec64140600 legacy=0x7fec64065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec61d9b700 1 -- 172.21.15.47:0/3001511558 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64146430 con 0x7fec641433e0 2020-07-18T03:25:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec61d9b700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec641433e0 0x7fec640b0660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fec580010d0 tx=0x7fec54016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec54011e40 con 0x7fec641433e0 2020-07-18T03:25:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec540105b0 con 0x7fec641433e0 2020-07-18T03:25:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec54010910 con 0x7fec641433e0 2020-07-18T03:25:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec640665f0 con 0x7fec641433e0 2020-07-18T03:25:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.327+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64140a90 con 0x7fec641433e0 2020-07-18T03:25:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.328+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64144a10 con 0x7fec641433e0 2020-07-18T03:25:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.328+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fec54027030 con 0x7fec641433e0 2020-07-18T03:25:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.329+0000 7fec527fc700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec380504c0 0x7fec38052980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.329+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fec54068bf0 con 0x7fec641433e0 2020-07-18T03:25:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.329+0000 7fec6259c700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec380504c0 0x7fec38052980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.329+0000 7fec6259c700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec380504c0 0x7fec38052980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fec58001110 tx=0x7fec58001440).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.333+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fec64144a10 con 0x7fec641433e0 2020-07-18T03:25:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.492+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec641449f0 con 0x7fec641433e0 2020-07-18T03:25:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.493+0000 7fec527fc700 1 -- 172.21.15.47:0/3001511558 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7fec54044d10 con 0x7fec641433e0 2020-07-18T03:25:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:46.494 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec380504c0 msgr2=0x7fec38052980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec380504c0 0x7fec38052980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fec58001110 tx=0x7fec58001440).stop 2020-07-18T03:25:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec641433e0 msgr2=0x7fec640b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec641433e0 0x7fec640b0660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fec580010d0 tx=0x7fec54016000).stop 2020-07-18T03:25:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 shutdown_connections 2020-07-18T03:25:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec380504c0 0x7fec38052980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.494+0000 7fec68939700 1 --2- 172.21.15.47:0/3001511558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec641433e0 0x7fec640b0660 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.495+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 >> 172.21.15.47:0/3001511558 conn(0x7fec640b9340 msgr2=0x7fec640b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.495+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 shutdown_connections 2020-07-18T03:25:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.495+0000 7fec68939700 1 -- 172.21.15.47:0/3001511558 wait complete. 2020-07-18T03:25:46.506 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-07-18T03:25:46.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.623+0000 7f0fa8542700 1 Processor -- start 2020-07-18T03:25:46.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.623+0000 7f0fa8542700 1 -- start start 2020-07-18T03:25:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa8542700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0fa0142a20 0x7f0fa0142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa8542700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0fa00b3af0 0x7f0fa00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa8542700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa00af4b0 con 0x7f0fa0061c60 2020-07-18T03:25:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa8542700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa00b6d80 con 0x7f0fa00b3af0 2020-07-18T03:25:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa8542700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa00af710 con 0x7f0fa0142a20 2020-07-18T03:25:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0fa00b3af0 0x7f0fa00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0fa00b3af0 0x7f0fa00b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58318/0 (socket says 172.21.15.47:58318) 2020-07-18T03:25:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 -- 172.21.15.47:0/2154044970 learned_addr learned my addr 172.21.15.47:0/2154044970 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa62de700 1 --2- 172.21.15.47:0/2154044970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0fa0142a20 0x7f0fa0142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 -- 172.21.15.47:0/2154044970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0fa0142a20 msgr2=0x7f0fa0142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 --2- 172.21.15.47:0/2154044970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0fa0142a20 0x7f0fa0142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 -- 172.21.15.47:0/2154044970 >> v1:172.21.15.18:6789/0 conn(0x7f0fa0061c60 legacy=0x7f0fa00620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.624+0000 7f0fa6adf700 1 -- 172.21.15.47:0/2154044970 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00af9b0 con 0x7f0fa00b3af0 2020-07-18T03:25:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa6adf700 1 --2- 172.21.15.47:0/2154044970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa00b3af0 0x7f0fa00b3f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f0f980032b0 tx=0x7f0f98009270).ready entity=mon.1 client_cookie=a15b95ac82133c11 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa52dc700 1 -- 172.21.15.47:0/2154044970 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f98009e60 con 0x7f0fa00b3af0 2020-07-18T03:25:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa52dc700 1 -- 172.21.15.47:0/2154044970 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9800bc00 con 0x7f0fa00b3af0 2020-07-18T03:25:46.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa52dc700 1 -- 172.21.15.47:0/2154044970 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f9800bd70 con 0x7f0fa00b3af0 2020-07-18T03:25:46.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 -- 172.21.15.47:0/2154044970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa00b3af0 msgr2=0x7f0fa00b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 --2- 172.21.15.47:0/2154044970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa00b3af0 0x7f0fa00b3f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f0f980032b0 tx=0x7f0f98009270).stop 2020-07-18T03:25:46.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 -- 172.21.15.47:0/2154044970 shutdown_connections 2020-07-18T03:25:46.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 --2- 172.21.15.47:0/2154044970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa00b3af0 0x7f0fa00b3f60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 --2- 172.21.15.47:0/2154044970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0fa0142a20 0x7f0fa0142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:46.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 -- 172.21.15.47:0/2154044970 >> 172.21.15.47:0/2154044970 conn(0x7f0fa005f5a0 msgr2=0x7f0fa005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:46.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 -- 172.21.15.47:0/2154044970 shutdown_connections 2020-07-18T03:25:46.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.625+0000 7f0fa8542700 1 -- 172.21.15.47:0/2154044970 wait complete. 2020-07-18T03:25:46.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.626+0000 7f0fa8542700 1 Processor -- start 2020-07-18T03:25:46.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.626+0000 7f0fa8542700 1 -- start start 2020-07-18T03:25:46.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa8542700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0fa01417d0 0x7f0fa013df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa8542700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa01562d0 con 0x7f0fa0061c60 2020-07-18T03:25:46.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa8542700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa0066440 con 0x7f0fa0142a20 2020-07-18T03:25:46.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa8542700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa0063930 con 0x7f0fa01417d0 2020-07-18T03:25:46.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa5add700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0fa0142a20 0x7f0fa0065d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39410/0 (socket says 172.21.15.47:39410) 2020-07-18T03:25:46.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa6adf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0fa01417d0 0x7f0fa013df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa5add700 1 -- 172.21.15.47:0/1008753834 learned_addr learned my addr 172.21.15.47:0/1008753834 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:46.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa6adf700 1 -- 172.21.15.47:0/1008753834 >> v1:172.21.15.47:6789/0 conn(0x7f0fa0142a20 legacy=0x7f0fa0065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa6adf700 1 -- 172.21.15.47:0/1008753834 >> v1:172.21.15.18:6789/0 conn(0x7f0fa0061c60 legacy=0x7f0fa0145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:46.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413543667 0 0) 0x7f0fa0066440 con 0x7f0fa0142a20 2020-07-18T03:25:46.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.627+0000 7f0fa6adf700 1 -- 172.21.15.47:0/1008753834 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa0063bd0 con 0x7f0fa01417d0 2020-07-18T03:25:46.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.628+0000 7f0fa6adf700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa01417d0 0x7f0fa013df20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0fa0145da0 tx=0x7f0f98016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.628+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f980108c0 con 0x7f0fa01417d0 2020-07-18T03:25:46.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.628+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98010a20 con 0x7f0fa01417d0 2020-07-18T03:25:46.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.628+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa01459b0 con 0x7f0fa01417d0 2020-07-18T03:25:46.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.628+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f98010d80 con 0x7f0fa01417d0 2020-07-18T03:25:46.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.628+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa01413d0 con 0x7f0fa01417d0 2020-07-18T03:25:46.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.629+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa005b120 con 0x7f0fa01417d0 2020-07-18T03:25:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.629+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0f9800f070 con 0x7f0fa01417d0 2020-07-18T03:25:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.629+0000 7f0f8e7fc700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0f7804ff50 0x7f0f78052410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.629+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0f98069210 con 0x7f0fa01417d0 2020-07-18T03:25:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.629+0000 7f0fa62de700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0f7804ff50 0x7f0f78052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.630+0000 7f0fa62de700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0f7804ff50 0x7f0f78052410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0f9000c000 tx=0x7f0f90006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.633+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0f98041a50 con 0x7f0fa01417d0 2020-07-18T03:25:46.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:46.845+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0fa00690d0 con 0x7f0fa01417d0 2020-07-18T03:25:47.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.576+0000 7f0f8e7fc700 1 -- 172.21.15.47:0/1008753834 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v3) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0f98015020 con 0x7f0fa01417d0 2020-07-18T03:25:47.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0f7804ff50 msgr2=0x7f0f78052410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0f7804ff50 0x7f0f78052410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0f9000c000 tx=0x7f0f90006dd0).stop 2020-07-18T03:25:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa01417d0 msgr2=0x7f0fa013df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa01417d0 0x7f0fa013df20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0fa0145da0 tx=0x7f0f98016000).stop 2020-07-18T03:25:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 shutdown_connections 2020-07-18T03:25:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa01417d0 0x7f0fa013df20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 --2- 172.21.15.47:0/1008753834 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0f7804ff50 0x7f0f78052410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 >> 172.21.15.47:0/1008753834 conn(0x7f0fa005f5a0 msgr2=0x7f0fa0145f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:47.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 shutdown_connections 2020-07-18T03:25:47.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.578+0000 7f0fa8542700 1 -- 172.21.15.47:0/1008753834 wait complete. 2020-07-18T03:25:47.593 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:25:47.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.715+0000 7f50b7f51700 1 Processor -- start 2020-07-18T03:25:47.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.715+0000 7f50b7f51700 1 -- start start 2020-07-18T03:25:47.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b7f51700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f50b00ad100 0x7f50b00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:47.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b7f51700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50b01443f0 0x7f50b0147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:47.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b7f51700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b0147330 con 0x7f50b0147a00 2020-07-18T03:25:47.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b7f51700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b00ae530 con 0x7f50b00ad100 2020-07-18T03:25:47.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b7f51700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b00b57e0 con 0x7f50b01443f0 2020-07-18T03:25:47.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b54ec700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f50b0147a00 0x7f50b005f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39416/0 (socket says 172.21.15.47:39416) 2020-07-18T03:25:47.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b64ee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50b01443f0 0x7f50b0147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b54ec700 1 -- 172.21.15.47:0/3417054961 learned_addr learned my addr 172.21.15.47:0/3417054961 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b5ced700 1 --2- 172.21.15.47:0/3417054961 >> v2:172.21.15.18:3300/0 conn(0x7f50b00ad100 0x7f50b00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b64ee700 1 -- 172.21.15.47:0/3417054961 >> v2:172.21.15.18:3300/0 conn(0x7f50b00ad100 msgr2=0x7f50b00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b64ee700 1 --2- 172.21.15.47:0/3417054961 >> v2:172.21.15.18:3300/0 conn(0x7f50b00ad100 0x7f50b00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b64ee700 1 -- 172.21.15.47:0/3417054961 >> v1:172.21.15.47:6789/0 conn(0x7f50b0147a00 legacy=0x7f50b005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b64ee700 1 -- 172.21.15.47:0/3417054961 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00b5a80 con 0x7f50b01443f0 2020-07-18T03:25:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b4ceb700 1 -- 172.21.15.47:0/3417054961 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084017475 0 0) 0x7f50b0147330 con 0x7f50b0147a00 2020-07-18T03:25:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.716+0000 7f50b64ee700 1 --2- 172.21.15.47:0/3417054961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b01443f0 0x7f50b0147e40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f50a80032b0 tx=0x7f50a80091d0).ready entity=mon.1 client_cookie=f1181487837b0f1b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b4ceb700 1 -- 172.21.15.47:0/3417054961 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a800ba50 con 0x7f50b01443f0 2020-07-18T03:25:47.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b4ceb700 1 -- 172.21.15.47:0/3417054961 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a800bbb0 con 0x7f50b01443f0 2020-07-18T03:25:47.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b4ceb700 1 -- 172.21.15.47:0/3417054961 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a800bd20 con 0x7f50b01443f0 2020-07-18T03:25:47.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 -- 172.21.15.47:0/3417054961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b01443f0 msgr2=0x7f50b0147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 --2- 172.21.15.47:0/3417054961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b01443f0 0x7f50b0147e40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f50a80032b0 tx=0x7f50a80091d0).stop 2020-07-18T03:25:47.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 -- 172.21.15.47:0/3417054961 shutdown_connections 2020-07-18T03:25:47.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 --2- 172.21.15.47:0/3417054961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b01443f0 0x7f50b0147e40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 --2- 172.21.15.47:0/3417054961 >> v2:172.21.15.18:3300/0 conn(0x7f50b00ad100 0x7f50b00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 -- 172.21.15.47:0/3417054961 >> 172.21.15.47:0/3417054961 conn(0x7f50b00b9340 msgr2=0x7f50b00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:47.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 -- 172.21.15.47:0/3417054961 shutdown_connections 2020-07-18T03:25:47.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.717+0000 7f50b7f51700 1 -- 172.21.15.47:0/3417054961 wait complete. 2020-07-18T03:25:47.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.718+0000 7f50b7f51700 1 Processor -- start 2020-07-18T03:25:47.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.718+0000 7f50b7f51700 1 -- start start 2020-07-18T03:25:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b7f51700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50b00ad100 0x7f50b014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b7f51700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50b014aea0 0x7f50b0156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b7f51700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b00b1680 con 0x7f50b0147a00 2020-07-18T03:25:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b7f51700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b014d640 con 0x7f50b014aea0 2020-07-18T03:25:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b7f51700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00b3070 con 0x7f50b00ad100 2020-07-18T03:25:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50b014aea0 0x7f50b0156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50b014aea0 0x7f50b0156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58338/0 (socket says 172.21.15.47:58338) 2020-07-18T03:25:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 -- 172.21.15.47:0/2274648884 learned_addr learned my addr 172.21.15.47:0/2274648884 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b5ced700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50b00ad100 0x7f50b014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 -- 172.21.15.47:0/2274648884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50b00ad100 msgr2=0x7f50b014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50b00ad100 0x7f50b014c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 -- 172.21.15.47:0/2274648884 >> v1:172.21.15.18:6789/0 conn(0x7f50b0147a00 legacy=0x7f50b014cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 -- 172.21.15.47:0/2274648884 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00b1920 con 0x7f50b014aea0 2020-07-18T03:25:47.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.719+0000 7f50b64ee700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b014aea0 0x7f50b0156880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f50a80032b0 tx=0x7f50a8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.720+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a80108c0 con 0x7f50b014aea0 2020-07-18T03:25:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.720+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8010a20 con 0x7f50b014aea0 2020-07-18T03:25:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.720+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b00b1bc0 con 0x7f50b014aea0 2020-07-18T03:25:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.720+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a8010d80 con 0x7f50b014aea0 2020-07-18T03:25:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.720+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b0156d70 con 0x7f50b014aea0 2020-07-18T03:25:47.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.721+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b0057450 con 0x7f50b014aea0 2020-07-18T03:25:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.721+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f50a8027030 con 0x7f50b014aea0 2020-07-18T03:25:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.721+0000 7f509dffb700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f508804ffe0 0x7f50880524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.721+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f50a80690f0 con 0x7f50b014aea0 2020-07-18T03:25:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.721+0000 7f50b5ced700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f508804ffe0 0x7f50880524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.722+0000 7f50b5ced700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f508804ffe0 0x7f50880524a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f50a000c000 tx=0x7f50a0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.725+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f50a80452b0 con 0x7f50b014aea0 2020-07-18T03:25:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.934+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50b006a100 con 0x7f50b014aea0 2020-07-18T03:25:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.934+0000 7f509dffb700 1 -- 172.21.15.47:0/2274648884 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1416 (secure 0 0 0) 0x7f50a8044ea0 con 0x7f50b014aea0 2020-07-18T03:25:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f508804ffe0 msgr2=0x7f50880524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f508804ffe0 0x7f50880524a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f50a000c000 tx=0x7f50a0008040).stop 2020-07-18T03:25:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b014aea0 msgr2=0x7f50b0156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b014aea0 0x7f50b0156880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f50a80032b0 tx=0x7f50a8016000).stop 2020-07-18T03:25:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 shutdown_connections 2020-07-18T03:25:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f508804ffe0 0x7f50880524a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50b014aea0 0x7f50b0156880 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 --2- 172.21.15.47:0/2274648884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50b00ad100 0x7f50b014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 >> 172.21.15.47:0/2274648884 conn(0x7f50b00b9340 msgr2=0x7f50b00ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:47.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 shutdown_connections 2020-07-18T03:25:47.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:47.936+0000 7f50b7f51700 1 -- 172.21.15.47:0/2274648884 wait complete. 2020-07-18T03:25:47.945 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2020-07-18T03:25:47.945 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:47.945 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":3,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:47.571812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:25:47.947 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.060+0000 7fc87b2d8700 1 Processor -- start 2020-07-18T03:25:48.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc87b2d8700 1 -- start start 2020-07-18T03:25:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc87b2d8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8740ad100 0x7fc8740b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc87b2d8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc874147a00 0x7fc874147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc87b2d8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874147330 con 0x7fc87405f5a0 2020-07-18T03:25:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc87b2d8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740ae530 con 0x7fc874147a00 2020-07-18T03:25:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc87b2d8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740b57e0 con 0x7fc8740ad100 2020-07-18T03:25:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.061+0000 7fc879074700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8740ad100 0x7fc8740b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8740ad100 0x7fc8740b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58342/0 (socket says 172.21.15.47:58342) 2020-07-18T03:25:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 -- 172.21.15.47:0/3309033088 learned_addr learned my addr 172.21.15.47:0/3309033088 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879875700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc87405f5a0 0x7fc87405f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39430/0 (socket says 172.21.15.47:39430) 2020-07-18T03:25:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc878873700 1 --2- 172.21.15.47:0/3309033088 >> v2:172.21.15.18:3300/0 conn(0x7fc874147a00 0x7fc874147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 -- 172.21.15.47:0/3309033088 >> v2:172.21.15.18:3300/0 conn(0x7fc874147a00 msgr2=0x7fc874147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 --2- 172.21.15.47:0/3309033088 >> v2:172.21.15.18:3300/0 conn(0x7fc874147a00 0x7fc874147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 -- 172.21.15.47:0/3309033088 >> v1:172.21.15.47:6789/0 conn(0x7fc87405f5a0 legacy=0x7fc87405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 -- 172.21.15.47:0/3309033088 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740b5a80 con 0x7fc8740ad100 2020-07-18T03:25:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc867fff700 1 -- 172.21.15.47:0/3309033088 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888414363 0 0) 0x7fc874147330 con 0x7fc87405f5a0 2020-07-18T03:25:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc879074700 1 --2- 172.21.15.47:0/3309033088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8740ad100 0x7fc8740b4c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc868002f30 tx=0x7fc868006e90).ready entity=mon.1 client_cookie=5f1cfa74528b6b56 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc867fff700 1 -- 172.21.15.47:0/3309033088 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc868009850 con 0x7fc8740ad100 2020-07-18T03:25:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc867fff700 1 -- 172.21.15.47:0/3309033088 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8680099b0 con 0x7fc8740ad100 2020-07-18T03:25:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc867fff700 1 -- 172.21.15.47:0/3309033088 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc868009d10 con 0x7fc8740ad100 2020-07-18T03:25:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 -- 172.21.15.47:0/3309033088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8740ad100 msgr2=0x7fc8740b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 --2- 172.21.15.47:0/3309033088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8740ad100 0x7fc8740b4c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc868002f30 tx=0x7fc868006e90).stop 2020-07-18T03:25:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 -- 172.21.15.47:0/3309033088 shutdown_connections 2020-07-18T03:25:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 --2- 172.21.15.47:0/3309033088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8740ad100 0x7fc8740b4c10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 --2- 172.21.15.47:0/3309033088 >> v2:172.21.15.18:3300/0 conn(0x7fc874147a00 0x7fc874147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 -- 172.21.15.47:0/3309033088 >> 172.21.15.47:0/3309033088 conn(0x7fc8740b9340 msgr2=0x7fc8740b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 -- 172.21.15.47:0/3309033088 shutdown_connections 2020-07-18T03:25:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.062+0000 7fc87b2d8700 1 -- 172.21.15.47:0/3309033088 wait complete. 2020-07-18T03:25:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.064+0000 7fc87b2d8700 1 Processor -- start 2020-07-18T03:25:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.064+0000 7fc87b2d8700 1 -- start start 2020-07-18T03:25:48.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.064+0000 7fc87b2d8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc87405f5a0 0x7fc874138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.064+0000 7fc87b2d8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 0x7fc8740b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc87b2d8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874062c60 con 0x7fc8740635b0 2020-07-18T03:25:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc87b2d8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc874133460 con 0x7fc87405f5a0 2020-07-18T03:25:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc87b2d8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741393c0 con 0x7fc874147a00 2020-07-18T03:25:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc879875700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc8740635b0 0x7fc874061ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39440/0 (socket says 172.21.15.47:39440) 2020-07-18T03:25:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc879875700 1 -- 172.21.15.47:0/1153373841 learned_addr learned my addr 172.21.15.47:0/1153373841 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:48.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc879074700 1 --2- 172.21.15.47:0/1153373841 >> v2:172.21.15.18:3300/0 conn(0x7fc87405f5a0 0x7fc874138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc878873700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 0x7fc8740b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140205846 0 0) 0x7fc874062c60 con 0x7fc8740635b0 2020-07-18T03:25:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc84c003240 con 0x7fc8740635b0 2020-07-18T03:25:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2271877906 0 0) 0x7fc84c003240 con 0x7fc8740635b0 2020-07-18T03:25:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc874062c60 con 0x7fc8740635b0 2020-07-18T03:25:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fc86c001cb0 con 0x7fc8740635b0 2020-07-18T03:25:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 msgr2=0x7fc8740b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 0x7fc8740b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 >> v2:172.21.15.18:3300/0 conn(0x7fc87405f5a0 msgr2=0x7fc874138ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 --2- 172.21.15.47:0/1153373841 >> v2:172.21.15.18:3300/0 conn(0x7fc87405f5a0 0x7fc874138ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 >> v1:172.21.15.47:6789/0 conn(0x7fc8740635b0 legacy=0x7fc874061ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 0x7fc84c0038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc84c003de0 0x7fc84c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c0049d0 0x7fc84c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741393c0 con 0x7fc84c0049d0 2020-07-18T03:25:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c005340 con 0x7fc84c003de0 2020-07-18T03:25:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c00b730 con 0x7fc874147a00 2020-07-18T03:25:48.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc879074700 1 --2- 172.21.15.47:0/1153373841 >> v2:172.21.15.18:3300/0 conn(0x7fc87405f5a0 0x7fc874138ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:48.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc878873700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc84c003de0 0x7fc84c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc879875700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c0049d0 0x7fc84c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.065+0000 7fc879074700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 0x7fc84c0038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc878873700 1 -- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 msgr2=0x7fc84c0038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc878873700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc874147a00 0x7fc84c0038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc878873700 1 -- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c0049d0 msgr2=0x7fc84c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc878873700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c0049d0 0x7fc84c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc878873700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874063dd0 con 0x7fc84c003de0 2020-07-18T03:25:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874064070 con 0x7fc84c003de0 2020-07-18T03:25:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc874066750 con 0x7fc84c003de0 2020-07-18T03:25:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc878873700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc84c003de0 0x7fc84c0044e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc87000b000 tx=0x7fc870006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc870018070 con 0x7fc84c003de0 2020-07-18T03:25:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000f9e0 con 0x7fc84c003de0 2020-07-18T03:25:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.066+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc87000fd40 con 0x7fc84c003de0 2020-07-18T03:25:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.067+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874057450 con 0x7fc84c003de0 2020-07-18T03:25:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.067+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc870014440 con 0x7fc84c003de0 2020-07-18T03:25:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.067+0000 7fc8657fa700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc84c058440 0x7fc84c05a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.067+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc8700684d0 con 0x7fc84c003de0 2020-07-18T03:25:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.067+0000 7fc879074700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc84c058440 0x7fc84c05a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.068+0000 7fc879074700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc84c058440 0x7fc84c05a900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc868016000 tx=0x7fc868015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.071+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc870040b80 con 0x7fc84c003de0 2020-07-18T03:25:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.226+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc874148110 con 0x7fc84c003de0 2020-07-18T03:25:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.227+0000 7fc8657fa700 1 -- 172.21.15.47:0/1153373841 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7fc8700441a0 con 0x7fc84c003de0 2020-07-18T03:25:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:48.229 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.228+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc84c058440 msgr2=0x7fc84c05a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.228+0000 7fc87b2d8700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc84c058440 0x7fc84c05a900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc868016000 tx=0x7fc868015000).stop 2020-07-18T03:25:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.228+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc84c003de0 msgr2=0x7fc84c0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.228+0000 7fc87b2d8700 1 --2- 172.21.15.47:0/1153373841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc84c003de0 0x7fc84c0044e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc87000b000 tx=0x7fc870006dd0).stop 2020-07-18T03:25:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.228+0000 7fc879875700 1 -- 172.21.15.47:0/1153373841 reap_dead start 2020-07-18T03:25:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.228+0000 7fc879875700 1 -- 172.21.15.47:0/1153373841 reap_dead start 2020-07-18T03:25:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.229+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 shutdown_connections 2020-07-18T03:25:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.229+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 >> 172.21.15.47:0/1153373841 conn(0x7fc8740b9340 msgr2=0x7fc8740601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.229+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 shutdown_connections 2020-07-18T03:25:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.229+0000 7fc87b2d8700 1 -- 172.21.15.47:0/1153373841 wait complete. 2020-07-18T03:25:48.240 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-07-18T03:25:48.241 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:25:48.241 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:25:48.243 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:25:48.244 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:25:48.244 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:25:48.247 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:25:48.247 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:48.248 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-07-18T03:25:48.324 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:25:48.336 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:25:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.364+0000 7f1e60cd6700 1 Processor -- start 2020-07-18T03:25:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.364+0000 7f1e60cd6700 1 -- start start 2020-07-18T03:25:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e60cd6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e5c0b8510 0x7f1e5c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e60cd6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0b95a0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e60cd6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0b5c80 con 0x7f1e5c0b7420 2020-07-18T03:25:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e60cd6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b5a90 con 0x7f1e5c0b8510 2020-07-18T03:25:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e53fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1e5c0b7420 0x7f1e5c0b7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39460/0 (socket says 172.21.15.47:39460) 2020-07-18T03:25:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e53fff700 1 -- 172.21.15.47:0/121075560 learned_addr learned my addr 172.21.15.47:0/121075560 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e5ad9d700 1 --2- 172.21.15.47:0/121075560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e5c0b8510 0x7f1e5c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491568584 0 0) 0x7f1e5c0b5c80 con 0x7f1e5c0b7420 2020-07-18T03:25:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e3c003240 con 0x7f1e5c0b7420 2020-07-18T03:25:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039660923 0 0) 0x7f1e5c0b95a0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e5c0b5c80 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e5ad9d700 1 -- 172.21.15.47:0/121075560 >> v1:172.21.15.47:6789/0 conn(0x7f1e5c0b7420 legacy=0x7f1e5c0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e5ad9d700 1 -- 172.21.15.47:0/121075560 >> v1:172.21.15.18:6789/0 conn(0x7f1e5c0b42a0 legacy=0x7f1e5c0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.365+0000 7f1e5ad9d700 1 -- 172.21.15.47:0/121075560 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0b5f20 con 0x7f1e5c0b8510 2020-07-18T03:25:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e5ad9d700 1 --2- 172.21.15.47:0/121075560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e5c0b8510 0x7f1e5c151bc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1e540032b0 tx=0x7f1e540091d0).ready entity=mon.2 client_cookie=8f7cc67553a1dd31 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5400b920 con 0x7f1e5c0b8510 2020-07-18T03:25:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5400ba80 con 0x7f1e5c0b8510 2020-07-18T03:25:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e59d9b700 1 -- 172.21.15.47:0/121075560 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5400bbf0 con 0x7f1e5c0b8510 2020-07-18T03:25:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 -- 172.21.15.47:0/121075560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e5c0b8510 msgr2=0x7f1e5c151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 --2- 172.21.15.47:0/121075560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e5c0b8510 0x7f1e5c151bc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1e540032b0 tx=0x7f1e540091d0).stop 2020-07-18T03:25:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 -- 172.21.15.47:0/121075560 shutdown_connections 2020-07-18T03:25:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 --2- 172.21.15.47:0/121075560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1e5c0b8510 0x7f1e5c151bc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 -- 172.21.15.47:0/121075560 >> 172.21.15.47:0/121075560 conn(0x7f1e5c05f5a0 msgr2=0x7f1e5c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 -- 172.21.15.47:0/121075560 shutdown_connections 2020-07-18T03:25:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.366+0000 7f1e60cd6700 1 -- 172.21.15.47:0/121075560 wait complete. 2020-07-18T03:25:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.367+0000 7f1e60cd6700 1 Processor -- start 2020-07-18T03:25:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.367+0000 7f1e60cd6700 1 -- start start 2020-07-18T03:25:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e60cd6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1e5c0b42a0 0x7f1e5c13b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e60cd6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c1488d0 con 0x7f1e5c0b7420 2020-07-18T03:25:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e60cd6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c148b70 con 0x7f1e5c135e60 2020-07-18T03:25:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e60cd6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c148e10 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e5ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1e5c135e60 0x7f1e5c1492b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39468/0 (socket says 172.21.15.47:39468) 2020-07-18T03:25:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e5ad9d700 1 -- 172.21.15.47:0/4273733296 learned_addr learned my addr 172.21.15.47:0/4273733296 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e5a59c700 1 --2- 172.21.15.47:0/4273733296 >> v2:172.21.15.18:3300/0 conn(0x7f1e5c0b42a0 0x7f1e5c13b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116737958 0 0) 0x7f1e5c148b70 con 0x7f1e5c135e60 2020-07-18T03:25:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.368+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e38003240 con 0x7f1e5c135e60 2020-07-18T03:25:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e5a59c700 1 -- 172.21.15.47:0/4273733296 >> v1:172.21.15.47:6789/0 conn(0x7f1e5c135e60 legacy=0x7f1e5c1492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264719693 0 0) 0x7f1e5c1488d0 con 0x7f1e5c0b7420 2020-07-18T03:25:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e5a59c700 1 -- 172.21.15.47:0/4273733296 >> v1:172.21.15.18:6789/0 conn(0x7f1e5c0b7420 legacy=0x7f1e5c139640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e5a59c700 1 -- 172.21.15.47:0/4273733296 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c13bde0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e5a59c700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e5c0b42a0 0x7f1e5c13b8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1e54009ae0 tx=0x7f1e4c007950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5400b920 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c013410 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c137350 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e4c013580 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.369+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c14a7a0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.370+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c139de0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.370+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1e4c010030 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.370+0000 7f1e527fc700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1e380502b0 0x7f1e38052770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.370+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1e4c067490 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.370+0000 7f1e53fff700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1e380502b0 0x7f1e38052770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.371+0000 7f1e53fff700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1e380502b0 0x7f1e38052770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1e44001090 tx=0x7f1e44007860).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.374+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1e4c03fbc0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.530+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e5c0bab20 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.531+0000 7f1e527fc700 1 -- 172.21.15.47:0/4273733296 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f1e4c0431e0 con 0x7f1e5c0b42a0 2020-07-18T03:25:48.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:48.533 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1e380502b0 msgr2=0x7f1e38052770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1e380502b0 0x7f1e38052770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1e44001090 tx=0x7f1e44007860).stop 2020-07-18T03:25:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e5c0b42a0 msgr2=0x7f1e5c13b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e5c0b42a0 0x7f1e5c13b8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1e54009ae0 tx=0x7f1e4c007950).stop 2020-07-18T03:25:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 shutdown_connections 2020-07-18T03:25:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1e380502b0 0x7f1e38052770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 --2- 172.21.15.47:0/4273733296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e5c0b42a0 0x7f1e5c13b8f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 >> 172.21.15.47:0/4273733296 conn(0x7f1e5c05f5a0 msgr2=0x7f1e5c05d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 shutdown_connections 2020-07-18T03:25:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.533+0000 7f1e60cd6700 1 -- 172.21.15.47:0/4273733296 wait complete. 2020-07-18T03:25:48.544 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:25:48.545 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:25:48.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.667+0000 7f2672eff700 1 Processor -- start 2020-07-18T03:25:48.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.667+0000 7f2672eff700 1 -- start start 2020-07-18T03:25:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f2672eff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 0x7f266c0b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f2672eff700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c14b520 con 0x7f266c147d50 2020-07-18T03:25:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f2672eff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c1471b0 con 0x7f266c1478f0 2020-07-18T03:25:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f2672eff700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c147450 con 0x7f266c0ad020 2020-07-18T03:25:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f266c1478f0 0x7f266c144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39484/0 (socket says 172.21.15.47:39484) 2020-07-18T03:25:48.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266bfff700 1 -- 172.21.15.47:0/3175108505 learned_addr learned my addr 172.21.15.47:0/3175108505 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:48.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f2670c9b700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 0x7f266c0b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537096624 0 0) 0x7f266c1471b0 con 0x7f266c1478f0 2020-07-18T03:25:48.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2650003240 con 0x7f266c1478f0 2020-07-18T03:25:48.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3739869709 0 0) 0x7f2650003240 con 0x7f266c1478f0 2020-07-18T03:25:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 msgr2=0x7f266c0b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 0x7f266c0b9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 >> v1:172.21.15.18:6789/0 conn(0x7f266c147d50 legacy=0x7f266c0616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0ba390 con 0x7f266c1478f0 2020-07-18T03:25:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f2658001850 con 0x7f266c1478f0 2020-07-18T03:25:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 >> v1:172.21.15.47:6789/0 conn(0x7f266c1478f0 legacy=0x7f266c144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 0x7f2650003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2650003c90 0x7f26500044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26500049d0 0x7f2650004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c147450 con 0x7f2650003c90 2020-07-18T03:25:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2650004220 con 0x7f26500049d0 2020-07-18T03:25:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265000b730 con 0x7f266c0ad020 2020-07-18T03:25:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.668+0000 7f267149c700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26500049d0 0x7f2650004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2670c9b700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 0x7f2650003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f266bfff700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2650003c90 0x7f26500044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 -- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 msgr2=0x7f2650003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0ad020 0x7f2650003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 -- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2650003c90 msgr2=0x7f26500044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2650003c90 0x7f26500044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 -- 172.21.15.47:0/3175108505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650003240 con 0x7f26500049d0 2020-07-18T03:25:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26500049d0 0x7f2650004e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f266c1477d0 tx=0x7f2664007e60).ready entity=mon.1 client_cookie=c9e23eb9a1826db1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266400c900 con 0x7f26500049d0 2020-07-18T03:25:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266400ca60 con 0x7f26500049d0 2020-07-18T03:25:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f266b7fe700 1 -- 172.21.15.47:0/3175108505 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266400cdc0 con 0x7f26500049d0 2020-07-18T03:25:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2672eff700 1 -- 172.21.15.47:0/3175108505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26500049d0 msgr2=0x7f2650004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2672eff700 1 --2- 172.21.15.47:0/3175108505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26500049d0 0x7f2650004e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f266c1477d0 tx=0x7f2664007e60).stop 2020-07-18T03:25:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f267149c700 1 -- 172.21.15.47:0/3175108505 reap_dead start 2020-07-18T03:25:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2672eff700 1 -- 172.21.15.47:0/3175108505 shutdown_connections 2020-07-18T03:25:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2672eff700 1 -- 172.21.15.47:0/3175108505 >> 172.21.15.47:0/3175108505 conn(0x7f266c05f5a0 msgr2=0x7f266c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2672eff700 1 -- 172.21.15.47:0/3175108505 shutdown_connections 2020-07-18T03:25:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.669+0000 7f2672eff700 1 -- 172.21.15.47:0/3175108505 wait complete. 2020-07-18T03:25:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 Processor -- start 2020-07-18T03:25:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 -- start start 2020-07-18T03:25:48.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f266c0b0880 0x7f266c1469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0b0c60 0x7f266c144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f266c13c6e0 0x7f266c13cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c141d30 con 0x7f266c13c6e0 2020-07-18T03:25:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c141fd0 con 0x7f266c0b0880 2020-07-18T03:25:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.671+0000 7f2672eff700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c13e160 con 0x7f266c0b0c60 2020-07-18T03:25:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f266c0b0880 0x7f266c1469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f266c0b0880 0x7f266c1469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58408/0 (socket says 172.21.15.47:58408) 2020-07-18T03:25:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 -- 172.21.15.47:0/791695726 learned_addr learned my addr 172.21.15.47:0/791695726 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f266bfff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0b0c60 0x7f266c144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f267149c700 1 --2- 172.21.15.47:0/791695726 >> v2:172.21.15.18:3300/0 conn(0x7f266c13c6e0 0x7f266c13cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 -- 172.21.15.47:0/791695726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0b0c60 msgr2=0x7f266c144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0b0c60 0x7f266c144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 -- 172.21.15.47:0/791695726 >> v2:172.21.15.18:3300/0 conn(0x7f266c13c6e0 msgr2=0x7f266c13cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 --2- 172.21.15.47:0/791695726 >> v2:172.21.15.18:3300/0 conn(0x7f266c13c6e0 0x7f266c13cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 -- 172.21.15.47:0/791695726 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c1426e0 con 0x7f266c0b0880 2020-07-18T03:25:48.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2670c9b700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f266c0b0880 0x7f266c1469d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f2660008000 tx=0x7f2660008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266000f830 con 0x7f266c0b0880 2020-07-18T03:25:48.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266000f990 con 0x7f266c0b0880 2020-07-18T03:25:48.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f266000fb00 con 0x7f266c0b0880 2020-07-18T03:25:48.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c141960 con 0x7f266c0b0880 2020-07-18T03:25:48.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.672+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c13e7c0 con 0x7f266c0b0880 2020-07-18T03:25:48.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.673+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c0b9ea0 con 0x7f266c0b0880 2020-07-18T03:25:48.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.673+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f266000fcf0 con 0x7f266c0b0880 2020-07-18T03:25:48.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.674+0000 7f2668ff9700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f264404ffa0 0x7f2644052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.674+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2660067ea0 con 0x7f266c0b0880 2020-07-18T03:25:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.674+0000 7f266bfff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f264404ffa0 0x7f2644052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.674+0000 7f266bfff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f264404ffa0 0x7f2644052460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2658009000 tx=0x7f2658009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.677+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2660040630 con 0x7f266c0b0880 2020-07-18T03:25:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.834+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f266c142870 con 0x7f266c0b0880 2020-07-18T03:25:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.834+0000 7f2668ff9700 1 -- 172.21.15.47:0/791695726 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2660043c50 con 0x7f266c0b0880 2020-07-18T03:25:48.836 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:25:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f264404ffa0 msgr2=0x7f2644052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f264404ffa0 0x7f2644052460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2658009000 tx=0x7f2658009040).stop 2020-07-18T03:25:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f266c0b0880 msgr2=0x7f266c1469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f266c0b0880 0x7f266c1469d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f2660008000 tx=0x7f2660008040).stop 2020-07-18T03:25:48.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 shutdown_connections 2020-07-18T03:25:48.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f264404ffa0 0x7f2644052460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 --2- 172.21.15.47:0/791695726 >> v2:172.21.15.18:3300/0 conn(0x7f266c13c6e0 0x7f266c13cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f266c0b0880 0x7f266c1469d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 --2- 172.21.15.47:0/791695726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f266c0b0c60 0x7f266c144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 >> 172.21.15.47:0/791695726 conn(0x7f266c05f5a0 msgr2=0x7f266c062050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 shutdown_connections 2020-07-18T03:25:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.836+0000 7f2672eff700 1 -- 172.21.15.47:0/791695726 wait complete. 2020-07-18T03:25:48.847 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:25:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.963+0000 7fba20ab1700 1 Processor -- start 2020-07-18T03:25:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.963+0000 7fba20ab1700 1 -- start start 2020-07-18T03:25:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba20ab1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba1c144e70 0x7fba1c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba20ab1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c149260 con 0x7fba1c142430 2020-07-18T03:25:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba20ab1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c061db0 con 0x7fba1c141fb0 2020-07-18T03:25:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba20ab1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c0b14c0 con 0x7fba1c144e70 2020-07-18T03:25:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba1a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba1c144e70 0x7fba1c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba19d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fba1c141fb0 0x7fba1c148440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39508/0 (socket says 172.21.15.47:39508) 2020-07-18T03:25:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba19d9b700 1 -- 172.21.15.47:0/2454484025 learned_addr learned my addr 172.21.15.47:0/2454484025 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba1a59c700 1 -- 172.21.15.47:0/2454484025 >> v1:172.21.15.47:6789/0 conn(0x7fba1c141fb0 legacy=0x7fba1c148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba1a59c700 1 -- 172.21.15.47:0/2454484025 >> v1:172.21.15.18:6789/0 conn(0x7fba1c142430 legacy=0x7fba1c148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.964+0000 7fba1a59c700 1 -- 172.21.15.47:0/2454484025 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c062050 con 0x7fba1c144e70 2020-07-18T03:25:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba1a59c700 1 --2- 172.21.15.47:0/2454484025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c144e70 0x7fba1c0b5f60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fba1c062670 tx=0x7fba10006dd0).ready entity=mon.1 client_cookie=e74062d4cb330e44 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba1959a700 1 -- 172.21.15.47:0/2454484025 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba100097b0 con 0x7fba1c144e70 2020-07-18T03:25:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba1959a700 1 -- 172.21.15.47:0/2454484025 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10009910 con 0x7fba1c144e70 2020-07-18T03:25:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba1959a700 1 -- 172.21.15.47:0/2454484025 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba10009c70 con 0x7fba1c144e70 2020-07-18T03:25:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 -- 172.21.15.47:0/2454484025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c144e70 msgr2=0x7fba1c0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 --2- 172.21.15.47:0/2454484025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c144e70 0x7fba1c0b5f60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fba1c062670 tx=0x7fba10006dd0).stop 2020-07-18T03:25:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 -- 172.21.15.47:0/2454484025 shutdown_connections 2020-07-18T03:25:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 --2- 172.21.15.47:0/2454484025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c144e70 0x7fba1c0b5f60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 -- 172.21.15.47:0/2454484025 >> 172.21.15.47:0/2454484025 conn(0x7fba1c05f5a0 msgr2=0x7fba1c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 -- 172.21.15.47:0/2454484025 shutdown_connections 2020-07-18T03:25:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.965+0000 7fba20ab1700 1 -- 172.21.15.47:0/2454484025 wait complete. 2020-07-18T03:25:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.966+0000 7fba20ab1700 1 Processor -- start 2020-07-18T03:25:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.966+0000 7fba20ab1700 1 -- start start 2020-07-18T03:25:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba20ab1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba1c141fb0 0x7fba1c0b9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba20ab1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fba1c142430 0x7fba1c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba20ab1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba1c155580 0x7fba1c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba20ab1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c155ad0 con 0x7fba1c142430 2020-07-18T03:25:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba20ab1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c155d70 con 0x7fba1c141fb0 2020-07-18T03:25:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba20ab1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c156010 con 0x7fba1c155580 2020-07-18T03:25:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba1c141fb0 0x7fba1c0b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba1c141fb0 0x7fba1c0b9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58422/0 (socket says 172.21.15.47:58422) 2020-07-18T03:25:48.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 -- 172.21.15.47:0/3311808558 learned_addr learned my addr 172.21.15.47:0/3311808558 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:48.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba19d9b700 1 --2- 172.21.15.47:0/3311808558 >> v2:172.21.15.18:3300/0 conn(0x7fba1c142430 0x7fba1c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1ad9d700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba1c155580 0x7fba1c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 -- 172.21.15.47:0/3311808558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba1c155580 msgr2=0x7fba1c159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba1c155580 0x7fba1c159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 -- 172.21.15.47:0/3311808558 >> v2:172.21.15.18:3300/0 conn(0x7fba1c142430 msgr2=0x7fba1c158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:48.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 --2- 172.21.15.47:0/3311808558 >> v2:172.21.15.18:3300/0 conn(0x7fba1c142430 0x7fba1c158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:48.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.967+0000 7fba1a59c700 1 -- 172.21.15.47:0/3311808558 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c0b9df0 con 0x7fba1c141fb0 2020-07-18T03:25:48.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.968+0000 7fba1a59c700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c141fb0 0x7fba1c0b9900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fba1c062670 tx=0x7fba10015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.968+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba1001d5d0 con 0x7fba1c141fb0 2020-07-18T03:25:48.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.968+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1001d730 con 0x7fba1c141fb0 2020-07-18T03:25:48.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.968+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c159810 con 0x7fba1c141fb0 2020-07-18T03:25:48.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.968+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba1001da90 con 0x7fba1c141fb0 2020-07-18T03:25:48.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.968+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c147e70 con 0x7fba1c141fb0 2020-07-18T03:25:48.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.969+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c141d60 con 0x7fba1c141fb0 2020-07-18T03:25:48.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.969+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fba10025030 con 0x7fba1c141fb0 2020-07-18T03:25:48.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.969+0000 7fba0a7fc700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb9f004ffd0 0x7fb9f0052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:48.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.969+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fba10069620 con 0x7fba1c141fb0 2020-07-18T03:25:48.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.969+0000 7fba19d9b700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb9f004ffd0 0x7fb9f0052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:48.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.969+0000 7fba19d9b700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb9f004ffd0 0x7fb9f0052490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fba0c00b000 tx=0x7fba0c009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:48.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:48.972+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fba10041db0 con 0x7fba1c141fb0 2020-07-18T03:25:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.130+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba1c156180 con 0x7fba1c141fb0 2020-07-18T03:25:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.130+0000 7fba0a7fc700 1 -- 172.21.15.47:0/3311808558 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba100230a0 con 0x7fba1c141fb0 2020-07-18T03:25:49.132 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb9f004ffd0 msgr2=0x7fb9f0052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb9f004ffd0 0x7fb9f0052490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fba0c00b000 tx=0x7fba0c009040).stop 2020-07-18T03:25:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c141fb0 msgr2=0x7fba1c0b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c141fb0 0x7fba1c0b9900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fba1c062670 tx=0x7fba10015000).stop 2020-07-18T03:25:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 shutdown_connections 2020-07-18T03:25:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb9f004ffd0 0x7fb9f0052490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fba1c155580 0x7fba1c159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 --2- 172.21.15.47:0/3311808558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1c141fb0 0x7fba1c0b9900 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 --2- 172.21.15.47:0/3311808558 >> v2:172.21.15.18:3300/0 conn(0x7fba1c142430 0x7fba1c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 >> 172.21.15.47:0/3311808558 conn(0x7fba1c05f5a0 msgr2=0x7fba1c12cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 shutdown_connections 2020-07-18T03:25:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.132+0000 7fba20ab1700 1 -- 172.21.15.47:0/3311808558 wait complete. 2020-07-18T03:25:49.143 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:25:49.144 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:25:49.146 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:25:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.262+0000 7f76e30ca700 1 Processor -- start 2020-07-18T03:25:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e30ca700 1 -- start start 2020-07-18T03:25:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e30ca700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76dc142a20 0x7f76dc142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e30ca700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f76dc061c60 0x7f76dc0620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e30ca700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc0af4b0 con 0x7f76dc0b5dc0 2020-07-18T03:25:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e30ca700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc0b6d80 con 0x7f76dc061c60 2020-07-18T03:25:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e30ca700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc0af710 con 0x7f76dc142a20 2020-07-18T03:25:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e0e66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76dc142a20 0x7f76dc142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e0e66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76dc142a20 0x7f76dc142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58430/0 (socket says 172.21.15.47:58430) 2020-07-18T03:25:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.263+0000 7f76e0e66700 1 -- 172.21.15.47:0/2648380435 learned_addr learned my addr 172.21.15.47:0/2648380435 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76dbfff700 1 --2- 172.21.15.47:0/2648380435 >> v2:172.21.15.18:3300/0 conn(0x7f76dc061c60 0x7f76dc0620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e0e66700 1 -- 172.21.15.47:0/2648380435 >> v2:172.21.15.18:3300/0 conn(0x7f76dc061c60 msgr2=0x7f76dc0620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e0e66700 1 --2- 172.21.15.47:0/2648380435 >> v2:172.21.15.18:3300/0 conn(0x7f76dc061c60 0x7f76dc0620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e0e66700 1 -- 172.21.15.47:0/2648380435 >> v1:172.21.15.18:6789/0 conn(0x7f76dc0b5dc0 legacy=0x7f76dc146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e0e66700 1 -- 172.21.15.47:0/2648380435 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc0af9b0 con 0x7f76dc142a20 2020-07-18T03:25:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e0e66700 1 --2- 172.21.15.47:0/2648380435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc142a20 0x7f76dc142e00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f76dc05d1a0 tx=0x7f76cc006e90).ready entity=mon.1 client_cookie=bf21e250969dafa9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76db7fe700 1 -- 172.21.15.47:0/2648380435 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76cc009930 con 0x7f76dc142a20 2020-07-18T03:25:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76db7fe700 1 -- 172.21.15.47:0/2648380435 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc009a90 con 0x7f76dc142a20 2020-07-18T03:25:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76db7fe700 1 -- 172.21.15.47:0/2648380435 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76cc009e30 con 0x7f76dc142a20 2020-07-18T03:25:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 -- 172.21.15.47:0/2648380435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc142a20 msgr2=0x7f76dc142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 --2- 172.21.15.47:0/2648380435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc142a20 0x7f76dc142e00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f76dc05d1a0 tx=0x7f76cc006e90).stop 2020-07-18T03:25:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 -- 172.21.15.47:0/2648380435 shutdown_connections 2020-07-18T03:25:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 --2- 172.21.15.47:0/2648380435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc142a20 0x7f76dc142e00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 --2- 172.21.15.47:0/2648380435 >> v2:172.21.15.18:3300/0 conn(0x7f76dc061c60 0x7f76dc0620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 -- 172.21.15.47:0/2648380435 >> 172.21.15.47:0/2648380435 conn(0x7f76dc05f5a0 msgr2=0x7f76dc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 -- 172.21.15.47:0/2648380435 shutdown_connections 2020-07-18T03:25:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.264+0000 7f76e30ca700 1 -- 172.21.15.47:0/2648380435 wait complete. 2020-07-18T03:25:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 Processor -- start 2020-07-18T03:25:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 -- start start 2020-07-18T03:25:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76dc061c60 0x7f76dc0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f76dc0b5dc0 0x7f76dc0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76dc144300 0x7f76dc1447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc143950 con 0x7f76dc0b5dc0 2020-07-18T03:25:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc143bc0 con 0x7f76dc061c60 2020-07-18T03:25:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.266+0000 7f76e30ca700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc143e60 con 0x7f76dc144300 2020-07-18T03:25:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76dc061c60 0x7f76dc0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76dc061c60 0x7f76dc0666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58438/0 (socket says 172.21.15.47:58438) 2020-07-18T03:25:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 -- 172.21.15.47:0/1945307782 learned_addr learned my addr 172.21.15.47:0/1945307782 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76dbfff700 1 --2- 172.21.15.47:0/1945307782 >> v2:172.21.15.18:3300/0 conn(0x7f76dc0b5dc0 0x7f76dc0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e1667700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76dc144300 0x7f76dc1447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 -- 172.21.15.47:0/1945307782 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76dc144300 msgr2=0x7f76dc1447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76dc144300 0x7f76dc1447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 -- 172.21.15.47:0/1945307782 >> v2:172.21.15.18:3300/0 conn(0x7f76dc0b5dc0 msgr2=0x7f76dc0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 --2- 172.21.15.47:0/1945307782 >> v2:172.21.15.18:3300/0 conn(0x7f76dc0b5dc0 0x7f76dc0b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 -- 172.21.15.47:0/1945307782 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc066ba0 con 0x7f76dc061c60 2020-07-18T03:25:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e0e66700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc061c60 0x7f76dc0666b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f76dc05d1a0 tx=0x7f76cc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76cc01e460 con 0x7f76dc061c60 2020-07-18T03:25:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc01e5c0 con 0x7f76dc061c60 2020-07-18T03:25:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc0b2d70 con 0x7f76dc061c60 2020-07-18T03:25:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76cc01e920 con 0x7f76dc061c60 2020-07-18T03:25:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.267+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc063a20 con 0x7f76dc061c60 2020-07-18T03:25:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.268+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc143fd0 con 0x7f76dc061c60 2020-07-18T03:25:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.268+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f76cc025030 con 0x7f76dc061c60 2020-07-18T03:25:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.269+0000 7f76d8ff9700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f76b404ffd0 0x7f76b4052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.269+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f76cc069470 con 0x7f76dc061c60 2020-07-18T03:25:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.269+0000 7f76dbfff700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f76b404ffd0 0x7f76b4052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.269+0000 7f76dbfff700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f76b404ffd0 0x7f76b4052490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f76d000c000 tx=0x7f76d0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.272+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f76cc041c00 con 0x7f76dc061c60 2020-07-18T03:25:49.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.433+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76dc05b120 con 0x7f76dc061c60 2020-07-18T03:25:49.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.434+0000 7f76d8ff9700 1 -- 172.21.15.47:0/1945307782 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f76cc045220 con 0x7f76dc061c60 2020-07-18T03:25:49.435 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:49.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f76b404ffd0 msgr2=0x7f76b4052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f76b404ffd0 0x7f76b4052490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f76d000c000 tx=0x7f76d0008040).stop 2020-07-18T03:25:49.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc061c60 msgr2=0x7f76dc0666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc061c60 0x7f76dc0666b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f76dc05d1a0 tx=0x7f76cc015000).stop 2020-07-18T03:25:49.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 shutdown_connections 2020-07-18T03:25:49.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f76b404ffd0 0x7f76b4052490 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76dc144300 0x7f76dc1447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 --2- 172.21.15.47:0/1945307782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76dc061c60 0x7f76dc0666b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 --2- 172.21.15.47:0/1945307782 >> v2:172.21.15.18:3300/0 conn(0x7f76dc0b5dc0 0x7f76dc0b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 >> 172.21.15.47:0/1945307782 conn(0x7f76dc05f5a0 msgr2=0x7f76dc05f980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 shutdown_connections 2020-07-18T03:25:49.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.435+0000 7f76e30ca700 1 -- 172.21.15.47:0/1945307782 wait complete. 2020-07-18T03:25:49.446 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:25:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 Processor -- start 2020-07-18T03:25:49.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 -- start start 2020-07-18T03:25:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80ad330 0x7fbde8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde80b59b0 0x7fbde80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbde80b6300 0x7fbde80b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde8147de0 con 0x7fbde80b6300 2020-07-18T03:25:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde80ae000 con 0x7fbde80b59b0 2020-07-18T03:25:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.563+0000 7fbded868700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8148010 con 0x7fbde80ad330 2020-07-18T03:25:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde80b59b0 0x7fbde80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde80b59b0 0x7fbde80b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58446/0 (socket says 172.21.15.47:58446) 2020-07-18T03:25:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 -- 172.21.15.47:0/224746783 learned_addr learned my addr 172.21.15.47:0/224746783 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde77fe700 1 --2- 172.21.15.47:0/224746783 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b6300 0x7fbde80b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde6ffd700 1 --2- 172.21.15.47:0/224746783 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80ad330 0x7fbde8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 -- 172.21.15.47:0/224746783 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80ad330 msgr2=0x7fbde8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 --2- 172.21.15.47:0/224746783 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80ad330 0x7fbde8143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 -- 172.21.15.47:0/224746783 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b6300 msgr2=0x7fbde80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 --2- 172.21.15.47:0/224746783 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b6300 0x7fbde80b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 -- 172.21.15.47:0/224746783 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80ad710 con 0x7fbde80b59b0 2020-07-18T03:25:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde67fc700 1 --2- 172.21.15.47:0/224746783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80b59b0 0x7fbde80b5e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbddc002f30 tx=0x7fbddc006e90).ready entity=mon.1 client_cookie=47a59e56f5e79a3d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde5ffb700 1 -- 172.21.15.47:0/224746783 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbddc009730 con 0x7fbde80b59b0 2020-07-18T03:25:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde5ffb700 1 -- 172.21.15.47:0/224746783 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc009890 con 0x7fbde80b59b0 2020-07-18T03:25:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbde5ffb700 1 -- 172.21.15.47:0/224746783 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbddc009bf0 con 0x7fbde80b59b0 2020-07-18T03:25:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 -- 172.21.15.47:0/224746783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80b59b0 msgr2=0x7fbde80b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 --2- 172.21.15.47:0/224746783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80b59b0 0x7fbde80b5e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbddc002f30 tx=0x7fbddc006e90).stop 2020-07-18T03:25:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 -- 172.21.15.47:0/224746783 shutdown_connections 2020-07-18T03:25:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 --2- 172.21.15.47:0/224746783 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b6300 0x7fbde80b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 --2- 172.21.15.47:0/224746783 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80ad330 0x7fbde8143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 --2- 172.21.15.47:0/224746783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80b59b0 0x7fbde80b5e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 -- 172.21.15.47:0/224746783 >> 172.21.15.47:0/224746783 conn(0x7fbde805f5a0 msgr2=0x7fbde805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 -- 172.21.15.47:0/224746783 shutdown_connections 2020-07-18T03:25:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.564+0000 7fbded868700 1 -- 172.21.15.47:0/224746783 wait complete. 2020-07-18T03:25:49.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 Processor -- start 2020-07-18T03:25:49.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 -- start start 2020-07-18T03:25:49.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde80ad330 0x7fbde813a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80b6300 0x7fbde8142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbde80b3210 0x7fbde813afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde813abd0 con 0x7fbde80b3210 2020-07-18T03:25:49.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde814a3c0 con 0x7fbde80ad330 2020-07-18T03:25:49.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.566+0000 7fbded868700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde814a660 con 0x7fbde80b6300 2020-07-18T03:25:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde80ad330 0x7fbde813a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde80ad330 0x7fbde813a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58450/0 (socket says 172.21.15.47:58450) 2020-07-18T03:25:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 -- 172.21.15.47:0/1246752386 learned_addr learned my addr 172.21.15.47:0/1246752386 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde67fc700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80b6300 0x7fbde8142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde77fe700 1 --2- 172.21.15.47:0/1246752386 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b3210 0x7fbde813afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 -- 172.21.15.47:0/1246752386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80b6300 msgr2=0x7fbde8142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80b6300 0x7fbde8142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 -- 172.21.15.47:0/1246752386 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b3210 msgr2=0x7fbde813afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 --2- 172.21.15.47:0/1246752386 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b3210 0x7fbde813afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 -- 172.21.15.47:0/1246752386 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde8142cf0 con 0x7fbde80ad330 2020-07-18T03:25:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbde6ffd700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80ad330 0x7fbde813a6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fbde000c000 tx=0x7fbde00091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde00095d0 con 0x7fbde80ad330 2020-07-18T03:25:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde000cbb0 con 0x7fbde80ad330 2020-07-18T03:25:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde813b4c0 con 0x7fbde80ad330 2020-07-18T03:25:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde00123f0 con 0x7fbde80ad330 2020-07-18T03:25:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.567+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde814acc0 con 0x7fbde80ad330 2020-07-18T03:25:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.568+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbde0022070 con 0x7fbde80ad330 2020-07-18T03:25:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.568+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde814e0c0 con 0x7fbde80ad330 2020-07-18T03:25:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.569+0000 7fbdceffd700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdc404ff80 0x7fbdc4052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.569+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbde00683e0 con 0x7fbde80ad330 2020-07-18T03:25:49.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.569+0000 7fbde67fc700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdc404ff80 0x7fbdc4052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.569+0000 7fbde67fc700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdc404ff80 0x7fbdc4052440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fbddc01c000 tx=0x7fbddc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.572+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbde0040af0 con 0x7fbde80ad330 2020-07-18T03:25:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.728+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbde805b120 con 0x7fbde80ad330 2020-07-18T03:25:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.728+0000 7fbdceffd700 1 -- 172.21.15.47:0/1246752386 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbde0044110 con 0x7fbde80ad330 2020-07-18T03:25:49.729 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.729+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdc404ff80 msgr2=0x7fbdc4052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.729+0000 7fbded868700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdc404ff80 0x7fbdc4052440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fbddc01c000 tx=0x7fbddc015000).stop 2020-07-18T03:25:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.729+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80ad330 msgr2=0x7fbde813a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.729+0000 7fbded868700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80ad330 0x7fbde813a6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fbde000c000 tx=0x7fbde00091d0).stop 2020-07-18T03:25:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 shutdown_connections 2020-07-18T03:25:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdc404ff80 0x7fbdc4052440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 --2- 172.21.15.47:0/1246752386 >> v2:172.21.15.18:3300/0 conn(0x7fbde80b3210 0x7fbde813afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde80ad330 0x7fbde813a6e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 --2- 172.21.15.47:0/1246752386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbde80b6300 0x7fbde8142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 >> 172.21.15.47:0/1246752386 conn(0x7fbde805f5a0 msgr2=0x7fbde814d610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 shutdown_connections 2020-07-18T03:25:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.730+0000 7fbded868700 1 -- 172.21.15.47:0/1246752386 wait complete. 2020-07-18T03:25:49.741 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-07-18T03:25:49.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-07-18T03:25:49.742 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-07-18T03:25:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.855+0000 7feac4a3e700 1 Processor -- start 2020-07-18T03:25:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.855+0000 7feac4a3e700 1 -- start start 2020-07-18T03:25:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feac4a3e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac01456c0 0x7feac00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feac4a3e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00b85a0 con 0x7feac0145b70 2020-07-18T03:25:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feac4a3e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00b8840 con 0x7feac00b7f00 2020-07-18T03:25:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feac4a3e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac0145330 con 0x7feac01456c0 2020-07-18T03:25:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac01456c0 0x7feac00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabe59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feac00b7f00 0x7feac0063ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39546/0 (socket says 172.21.15.47:39546) 2020-07-18T03:25:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabe59c700 1 -- 172.21.15.47:0/2251425453 learned_addr learned my addr 172.21.15.47:0/2251425453 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabdd9b700 1 -- 172.21.15.47:0/2251425453 >> v1:172.21.15.47:6789/0 conn(0x7feac00b7f00 legacy=0x7feac0063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabdd9b700 1 -- 172.21.15.47:0/2251425453 >> v1:172.21.15.18:6789/0 conn(0x7feac0145b70 legacy=0x7feac00aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabdd9b700 1 -- 172.21.15.47:0/2251425453 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00b8ae0 con 0x7feac01456c0 2020-07-18T03:25:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.856+0000 7feabdd9b700 1 --2- 172.21.15.47:0/2251425453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac01456c0 0x7feac00ae520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7feab00032b0 tx=0x7feab00091d0).ready entity=mon.1 client_cookie=b695b9236507b8f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feabd59a700 1 -- 172.21.15.47:0/2251425453 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab000ba50 con 0x7feac01456c0 2020-07-18T03:25:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feabd59a700 1 -- 172.21.15.47:0/2251425453 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab000bbb0 con 0x7feac01456c0 2020-07-18T03:25:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feabd59a700 1 -- 172.21.15.47:0/2251425453 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab000bd20 con 0x7feac01456c0 2020-07-18T03:25:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 -- 172.21.15.47:0/2251425453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac01456c0 msgr2=0x7feac00ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2251425453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac01456c0 0x7feac00ae520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7feab00032b0 tx=0x7feab00091d0).stop 2020-07-18T03:25:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 -- 172.21.15.47:0/2251425453 shutdown_connections 2020-07-18T03:25:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2251425453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac01456c0 0x7feac00ae520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 -- 172.21.15.47:0/2251425453 >> 172.21.15.47:0/2251425453 conn(0x7feac005f5a0 msgr2=0x7feac005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 -- 172.21.15.47:0/2251425453 shutdown_connections 2020-07-18T03:25:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.857+0000 7feac4a3e700 1 -- 172.21.15.47:0/2251425453 wait complete. 2020-07-18T03:25:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.858+0000 7feac4a3e700 1 Processor -- start 2020-07-18T03:25:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.858+0000 7feac4a3e700 1 -- start start 2020-07-18T03:25:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feac4a3e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feac00b7f00 0x7feac00b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feac4a3e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac0145b70 0x7feac00ba470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feac4a3e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac0141690 con 0x7feac01537b0 2020-07-18T03:25:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feac4a3e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac0141930 con 0x7feac00b7f00 2020-07-18T03:25:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feac4a3e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac0141bd0 con 0x7feac0145b70 2020-07-18T03:25:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac0145b70 0x7feac00ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac0145b70 0x7feac00ba470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58464/0 (socket says 172.21.15.47:58464) 2020-07-18T03:25:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 -- 172.21.15.47:0/2793775293 learned_addr learned my addr 172.21.15.47:0/2793775293 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabe59c700 1 --2- 172.21.15.47:0/2793775293 >> v2:172.21.15.18:3300/0 conn(0x7feac00b7f00 0x7feac00b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 -- 172.21.15.47:0/2793775293 >> v2:172.21.15.18:3300/0 conn(0x7feac00b7f00 msgr2=0x7feac00b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 --2- 172.21.15.47:0/2793775293 >> v2:172.21.15.18:3300/0 conn(0x7feac00b7f00 0x7feac00b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 -- 172.21.15.47:0/2793775293 >> v1:172.21.15.18:6789/0 conn(0x7feac01537b0 legacy=0x7feac0142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabdd9b700 1 -- 172.21.15.47:0/2793775293 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0141380 con 0x7feac0145b70 2020-07-18T03:25:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.859+0000 7feabe59c700 1 --2- 172.21.15.47:0/2793775293 >> v2:172.21.15.18:3300/0 conn(0x7feac00b7f00 0x7feac00b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.860+0000 7feabdd9b700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0145b70 0x7feac00ba470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7feac01407e0 tx=0x7feab0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.860+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab0010a10 con 0x7feac0145b70 2020-07-18T03:25:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.860+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab0010b70 con 0x7feac0145b70 2020-07-18T03:25:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.860+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac00b0290 con 0x7feac0145b70 2020-07-18T03:25:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.860+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab0020400 con 0x7feac0145b70 2020-07-18T03:25:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.860+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac0142780 con 0x7feac0145b70 2020-07-18T03:25:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.861+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac0057450 con 0x7feac0145b70 2020-07-18T03:25:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.861+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7feab0027030 con 0x7feac0145b70 2020-07-18T03:25:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.861+0000 7feaae7fc700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9404fef0 0x7fea940523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.861+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feab0069370 con 0x7feac0145b70 2020-07-18T03:25:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.861+0000 7feabe59c700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9404fef0 0x7fea940523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.862+0000 7feabe59c700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9404fef0 0x7fea940523b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7feab400b000 tx=0x7feab4009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:49.865+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7feab0041a80 con 0x7feac0145b70 2020-07-18T03:25:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.060+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7feac0144bd0 con 0x7feac0145b70 2020-07-18T03:25:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.063+0000 7feaae7fc700 1 -- 172.21.15.47:0/2793775293 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+124398 (secure 0 0 0) 0x7feab00450a0 con 0x7feac0145b70 2020-07-18T03:25:50.068 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9404fef0 msgr2=0x7fea940523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9404fef0 0x7fea940523b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7feab400b000 tx=0x7feab4009040).stop 2020-07-18T03:25:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0145b70 msgr2=0x7feac00ba470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0145b70 0x7feac00ba470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7feac01407e0 tx=0x7feab0016000).stop 2020-07-18T03:25:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 shutdown_connections 2020-07-18T03:25:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9404fef0 0x7fea940523b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2793775293 >> v2:172.21.15.18:3300/0 conn(0x7feac00b7f00 0x7feac00b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 --2- 172.21.15.47:0/2793775293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0145b70 0x7feac00ba470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 >> 172.21.15.47:0/2793775293 conn(0x7feac005f5a0 msgr2=0x7feac005f980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 shutdown_connections 2020-07-18T03:25:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.065+0000 7feac4a3e700 1 -- 172.21.15.47:0/2793775293 wait complete. 2020-07-18T03:25:50.076 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":23227},{"type":"v1","addr":"172.21.15.47:6801","nonce":23227}]},"active_addr":"172.21.15.47:6801/23227","active_change":"2020-07-18T03:25:27.678962+0000","active_mgr_features":4540138292837744639,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138292837744639,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":822685497}]},{"addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2882666964}]},{"addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2653063485}]}]}} 2020-07-18T03:25:50.083 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-07-18T03:25:50.084 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-07-18T03:25:50.084 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:50.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.199+0000 7f5151359700 1 Processor -- start 2020-07-18T03:25:50.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.199+0000 7f5151359700 1 -- start start 2020-07-18T03:25:50.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.199+0000 7f5151359700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 0x7f514c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.199+0000 7f5151359700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c147330 con 0x7f514c0ad100 2020-07-18T03:25:50.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.199+0000 7f5151359700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c0b57e0 con 0x7f514c1433e0 2020-07-18T03:25:50.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.199+0000 7f5151359700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0ae530 con 0x7f514c147a00 2020-07-18T03:25:50.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f514b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f514c1433e0 0x7f514c143850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39562/0 (socket says 172.21.15.47:39562) 2020-07-18T03:25:50.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f514b7fe700 1 -- 172.21.15.47:0/3562175707 learned_addr learned my addr 172.21.15.47:0/3562175707 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f514a7fc700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 0x7f514c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378561174 0 0) 0x7f514c0b57e0 con 0x7f514c1433e0 2020-07-18T03:25:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5138003240 con 0x7f514c1433e0 2020-07-18T03:25:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558376091 0 0) 0x7f514c147330 con 0x7f514c0ad100 2020-07-18T03:25:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f514c0b57e0 con 0x7f514c0ad100 2020-07-18T03:25:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4147405752 0 0) 0x7f5138003240 con 0x7f514c1433e0 2020-07-18T03:25:50.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 msgr2=0x7f514c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 0x7f514c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 >> v1:172.21.15.18:6789/0 conn(0x7f514c0ad100 legacy=0x7f514c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0b5a80 con 0x7f514c1433e0 2020-07-18T03:25:50.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5144001a50 con 0x7f514c1433e0 2020-07-18T03:25:50.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 >> v1:172.21.15.47:6789/0 conn(0x7f514c1433e0 legacy=0x7f514c143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51380038c0 0x7f5138003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 0x7f51380044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51380049d0 0x7f5138004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0ae530 con 0x7f51380049d0 2020-07-18T03:25:50.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138005340 con 0x7f51380038c0 2020-07-18T03:25:50.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513800b760 con 0x7f514c147a00 2020-07-18T03:25:50.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f514affd700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51380038c0 0x7f5138003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f514a7fc700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 0x7f51380044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.200+0000 7f514b7fe700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51380049d0 0x7f5138004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514affd700 1 -- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 msgr2=0x7f51380044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514affd700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f514c147a00 0x7f51380044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514affd700 1 -- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51380049d0 msgr2=0x7f5138004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514affd700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51380049d0 0x7f5138004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514affd700 1 -- 172.21.15.47:0/3562175707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138003240 con 0x7f51380038c0 2020-07-18T03:25:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514affd700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51380038c0 0x7f5138003cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5138004450 tx=0x7f5140007ae0).ready entity=mon.1 client_cookie=4cb690139880f56c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514000c830 con 0x7f51380038c0 2020-07-18T03:25:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000c990 con 0x7f51380038c0 2020-07-18T03:25:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5149ffb700 1 -- 172.21.15.47:0/3562175707 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514000ccf0 con 0x7f51380038c0 2020-07-18T03:25:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5151359700 1 -- 172.21.15.47:0/3562175707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51380038c0 msgr2=0x7f5138003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5151359700 1 --2- 172.21.15.47:0/3562175707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51380038c0 0x7f5138003cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5138004450 tx=0x7f5140007ae0).stop 2020-07-18T03:25:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f514b7fe700 1 -- 172.21.15.47:0/3562175707 reap_dead start 2020-07-18T03:25:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5151359700 1 -- 172.21.15.47:0/3562175707 shutdown_connections 2020-07-18T03:25:50.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5151359700 1 -- 172.21.15.47:0/3562175707 >> 172.21.15.47:0/3562175707 conn(0x7f514c0b9340 msgr2=0x7f514c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5151359700 1 -- 172.21.15.47:0/3562175707 shutdown_connections 2020-07-18T03:25:50.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.201+0000 7f5151359700 1 -- 172.21.15.47:0/3562175707 wait complete. 2020-07-18T03:25:50.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f5151359700 1 Processor -- start 2020-07-18T03:25:50.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f5151359700 1 -- start start 2020-07-18T03:25:50.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f5151359700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f514c13bda0 0x7f514c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f5151359700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c13d600 con 0x7f514c13b920 2020-07-18T03:25:50.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f5151359700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c13d8a0 con 0x7f514c1589b0 2020-07-18T03:25:50.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f5151359700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c155100 con 0x7f514c13bda0 2020-07-18T03:25:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f514b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f514c13bda0 0x7f514c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f514affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f514c1589b0 0x7f514c158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39570/0 (socket says 172.21.15.47:39570) 2020-07-18T03:25:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.203+0000 7f514affd700 1 -- 172.21.15.47:0/3823882385 learned_addr learned my addr 172.21.15.47:0/3823882385 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f514b7fe700 1 -- 172.21.15.47:0/3823882385 >> v1:172.21.15.47:6789/0 conn(0x7f514c1589b0 legacy=0x7f514c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f514b7fe700 1 -- 172.21.15.47:0/3823882385 >> v1:172.21.15.18:6789/0 conn(0x7f514c13b920 legacy=0x7f514c065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964436749 0 0) 0x7f514c13d8a0 con 0x7f514c1589b0 2020-07-18T03:25:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f514b7fe700 1 -- 172.21.15.47:0/3823882385 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c066620 con 0x7f514c13bda0 2020-07-18T03:25:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f514b7fe700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f514c13bda0 0x7f514c1555a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5144000b20 tx=0x7f514c0b03e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514400fb60 con 0x7f514c13bda0 2020-07-18T03:25:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514400fcc0 con 0x7f514c13bda0 2020-07-18T03:25:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c146630 con 0x7f514c13bda0 2020-07-18T03:25:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5144012350 con 0x7f514c13bda0 2020-07-18T03:25:50.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.204+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c13d390 con 0x7f514c13bda0 2020-07-18T03:25:50.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.205+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c1467c0 con 0x7f514c13bda0 2020-07-18T03:25:50.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.205+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f514401d440 con 0x7f514c13bda0 2020-07-18T03:25:50.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.206+0000 7f5132ffd700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f512804fe40 0x7f5128052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.206+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5144067310 con 0x7f514c13bda0 2020-07-18T03:25:50.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.206+0000 7f514affd700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f512804fe40 0x7f5128052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.206+0000 7f514affd700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f512804fe40 0x7f5128052300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f514000c3c0 tx=0x7f5140017000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.209+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f514403fa40 con 0x7f514c13bda0 2020-07-18T03:25:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.365+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514c057450 con 0x7f514c13bda0 2020-07-18T03:25:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.366+0000 7f5132ffd700 1 -- 172.21.15.47:0/3823882385 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f5144019090 con 0x7f514c13bda0 2020-07-18T03:25:50.368 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:50.368 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f512804fe40 msgr2=0x7f5128052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f512804fe40 0x7f5128052300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f514000c3c0 tx=0x7f5140017000).stop 2020-07-18T03:25:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f514c13bda0 msgr2=0x7f514c1555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f514c13bda0 0x7f514c1555a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5144000b20 tx=0x7f514c0b03e0).stop 2020-07-18T03:25:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 shutdown_connections 2020-07-18T03:25:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f514c13bda0 0x7f514c1555a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 --2- 172.21.15.47:0/3823882385 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f512804fe40 0x7f5128052300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 >> 172.21.15.47:0/3823882385 conn(0x7f514c0b9340 msgr2=0x7f514c0b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 shutdown_connections 2020-07-18T03:25:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.368+0000 7f5151359700 1 -- 172.21.15.47:0/3823882385 wait complete. 2020-07-18T03:25:50.379 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-07-18T03:25:50.380 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.493+0000 7f010337b700 1 Processor -- start 2020-07-18T03:25:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.493+0000 7f010337b700 1 -- start start 2020-07-18T03:25:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f010337b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f00fc147e40 0x7f00fc143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f010337b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc147330 con 0x7f00fc147a00 2020-07-18T03:25:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f010337b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc0b57e0 con 0x7f00fc0ad100 2020-07-18T03:25:50.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f010337b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc0ae530 con 0x7f00fc147e40 2020-07-18T03:25:50.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101117700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00fc0ad100 0x7f00fc0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39578/0 (socket says 172.21.15.47:39578) 2020-07-18T03:25:50.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101117700 1 -- 172.21.15.47:0/1955821672 learned_addr learned my addr 172.21.15.47:0/1955821672 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101918700 1 --2- 172.21.15.47:0/1955821672 >> v2:172.21.15.18:3300/0 conn(0x7f00fc147e40 0x7f00fc143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f00fbfff700 1 -- 172.21.15.47:0/1955821672 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351924059 0 0) 0x7f00fc0b57e0 con 0x7f00fc0ad100 2020-07-18T03:25:50.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f00fbfff700 1 -- 172.21.15.47:0/1955821672 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e8003240 con 0x7f00fc0ad100 2020-07-18T03:25:50.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101918700 1 -- 172.21.15.47:0/1955821672 >> v1:172.21.15.47:6789/0 conn(0x7f00fc0ad100 legacy=0x7f00fc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101918700 1 -- 172.21.15.47:0/1955821672 >> v1:172.21.15.18:6789/0 conn(0x7f00fc147a00 legacy=0x7f00fc1433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f00fbfff700 1 -- 172.21.15.47:0/1955821672 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772875188 0 0) 0x7f00fc147330 con 0x7f00fc147a00 2020-07-18T03:25:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101918700 1 -- 172.21.15.47:0/1955821672 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0b5a80 con 0x7f00fc147e40 2020-07-18T03:25:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f0101918700 1 --2- 172.21.15.47:0/1955821672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00fc147e40 0x7f00fc143af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f00ec002f30 tx=0x7f00ec006dd0).ready entity=mon.0 client_cookie=c57b11f08fc49c13 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.494+0000 7f00fbfff700 1 -- 172.21.15.47:0/1955821672 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00ec00fa60 con 0x7f00fc147e40 2020-07-18T03:25:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f00fbfff700 1 -- 172.21.15.47:0/1955821672 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec00fbc0 con 0x7f00fc147e40 2020-07-18T03:25:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f00fbfff700 1 -- 172.21.15.47:0/1955821672 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00ec010470 con 0x7f00fc147e40 2020-07-18T03:25:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 -- 172.21.15.47:0/1955821672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00fc147e40 msgr2=0x7f00fc143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 --2- 172.21.15.47:0/1955821672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00fc147e40 0x7f00fc143af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f00ec002f30 tx=0x7f00ec006dd0).stop 2020-07-18T03:25:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 -- 172.21.15.47:0/1955821672 shutdown_connections 2020-07-18T03:25:50.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 --2- 172.21.15.47:0/1955821672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00fc147e40 0x7f00fc143af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 -- 172.21.15.47:0/1955821672 >> 172.21.15.47:0/1955821672 conn(0x7f00fc0b9340 msgr2=0x7f00fc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 -- 172.21.15.47:0/1955821672 shutdown_connections 2020-07-18T03:25:50.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.495+0000 7f010337b700 1 -- 172.21.15.47:0/1955821672 wait complete. 2020-07-18T03:25:50.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.496+0000 7f010337b700 1 Processor -- start 2020-07-18T03:25:50.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.496+0000 7f010337b700 1 -- start start 2020-07-18T03:25:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f010337b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f00fc0ad100 0x7f00fc065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f010337b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc155500 con 0x7f00fc155ee0 2020-07-18T03:25:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f010337b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc1557a0 con 0x7f00fc147a00 2020-07-18T03:25:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f010337b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc155a40 con 0x7f00fc0ad100 2020-07-18T03:25:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f0100916700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00fc147a00 0x7f00fc158b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39586/0 (socket says 172.21.15.47:39586) 2020-07-18T03:25:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f0100916700 1 -- 172.21.15.47:0/678454630 learned_addr learned my addr 172.21.15.47:0/678454630 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f0101117700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f00fc0ad100 0x7f00fc065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792257911 0 0) 0x7f00fc1557a0 con 0x7f00fc147a00 2020-07-18T03:25:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d4003240 con 0x7f00fc147a00 2020-07-18T03:25:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051832230 0 0) 0x7f00fc155500 con 0x7f00fc155ee0 2020-07-18T03:25:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.497+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00fc1557a0 con 0x7f00fc155ee0 2020-07-18T03:25:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1157554068 0 0) 0x7f00d4003240 con 0x7f00fc147a00 2020-07-18T03:25:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f0101117700 1 -- 172.21.15.47:0/678454630 >> v1:172.21.15.47:6789/0 conn(0x7f00fc147a00 legacy=0x7f00fc158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f0101117700 1 -- 172.21.15.47:0/678454630 >> v1:172.21.15.18:6789/0 conn(0x7f00fc155ee0 legacy=0x7f00fc115810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f0101117700 1 -- 172.21.15.47:0/678454630 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0b3040 con 0x7f00fc0ad100 2020-07-18T03:25:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f0101117700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f00fc0ad100 0x7f00fc065ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f00f0001b00 tx=0x7f00f4007cc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00f00019a0 con 0x7f00fc0ad100 2020-07-18T03:25:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f4005b80 con 0x7f00fc0ad100 2020-07-18T03:25:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc155140 con 0x7f00fc0ad100 2020-07-18T03:25:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00f4005cf0 con 0x7f00fc0ad100 2020-07-18T03:25:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.498+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc066590 con 0x7f00fc0ad100 2020-07-18T03:25:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.499+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc057450 con 0x7f00fc0ad100 2020-07-18T03:25:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.499+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f00f4010030 con 0x7f00fc0ad100 2020-07-18T03:25:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.499+0000 7f00f97fa700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f00d4050420 0x7f00d40528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.499+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f00f4067590 con 0x7f00fc0ad100 2020-07-18T03:25:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.499+0000 7f0100916700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f00d4050420 0x7f00d40528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.500+0000 7f0100916700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f00d4050420 0x7f00d40528e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f00f0000ff0 tx=0x7f00f0007640).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.504+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f00f403fba0 con 0x7f00fc0ad100 2020-07-18T03:25:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.661+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00fc155b80 con 0x7f00fc0ad100 2020-07-18T03:25:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.662+0000 7f00f97fa700 1 -- 172.21.15.47:0/678454630 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f00f40431c0 con 0x7f00fc0ad100 2020-07-18T03:25:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f00d4050420 msgr2=0x7f00d40528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f00d4050420 0x7f00d40528e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f00f0000ff0 tx=0x7f00f0007640).stop 2020-07-18T03:25:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f00fc0ad100 msgr2=0x7f00fc065ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f00fc0ad100 0x7f00fc065ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f00f0001b00 tx=0x7f00f4007cc0).stop 2020-07-18T03:25:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 shutdown_connections 2020-07-18T03:25:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f00fc0ad100 0x7f00fc065ee0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 --2- 172.21.15.47:0/678454630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f00d4050420 0x7f00d40528e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 >> 172.21.15.47:0/678454630 conn(0x7f00fc0b9340 msgr2=0x7f00fc1469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 shutdown_connections 2020-07-18T03:25:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.664+0000 7f010337b700 1 -- 172.21.15.47:0/678454630 wait complete. 2020-07-18T03:25:50.667 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:50.668 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:50.675 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-07-18T03:25:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.789+0000 7f837d5af700 1 Processor -- start 2020-07-18T03:25:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.789+0000 7f837d5af700 1 -- start start 2020-07-18T03:25:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f837d5af700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f83780ad020 0x7f83780b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f837d5af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f837813dbc0 0x7f837813e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f837d5af700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8378147640 0x7f8378147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f837d5af700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8378144700 con 0x7f83780ad020 2020-07-18T03:25:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f837d5af700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83781449a0 con 0x7f837813dbc0 2020-07-18T03:25:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f837d5af700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837813e510 con 0x7f8378147640 2020-07-18T03:25:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f83767fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f837813dbc0 0x7f837813e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f83767fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f837813dbc0 0x7f837813e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58506/0 (socket says 172.21.15.47:58506) 2020-07-18T03:25:50.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f83767fc700 1 -- 172.21.15.47:0/1705151446 learned_addr learned my addr 172.21.15.47:0/1705151446 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:50.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f83777fe700 1 --2- 172.21.15.47:0/1705151446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8378147640 0x7f8378147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.790+0000 7f8376ffd700 1 --2- 172.21.15.47:0/1705151446 >> v2:172.21.15.18:3300/0 conn(0x7f83780ad020 0x7f83780b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f83767fc700 1 -- 172.21.15.47:0/1705151446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8378147640 msgr2=0x7f8378147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f83767fc700 1 --2- 172.21.15.47:0/1705151446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8378147640 0x7f8378147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f83767fc700 1 -- 172.21.15.47:0/1705151446 >> v2:172.21.15.18:3300/0 conn(0x7f83780ad020 msgr2=0x7f83780b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f83767fc700 1 --2- 172.21.15.47:0/1705151446 >> v2:172.21.15.18:3300/0 conn(0x7f83780ad020 0x7f83780b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f83767fc700 1 -- 172.21.15.47:0/1705151446 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780b59e0 con 0x7f837813dbc0 2020-07-18T03:25:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f83767fc700 1 --2- 172.21.15.47:0/1705151446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837813dbc0 0x7f837813e020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8368002f30 tx=0x7f8368006e90).ready entity=mon.1 client_cookie=28a73000cde4cea4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f8375ffb700 1 -- 172.21.15.47:0/1705151446 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8368009730 con 0x7f837813dbc0 2020-07-18T03:25:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f8375ffb700 1 -- 172.21.15.47:0/1705151446 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368009890 con 0x7f837813dbc0 2020-07-18T03:25:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f8375ffb700 1 -- 172.21.15.47:0/1705151446 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8368009bf0 con 0x7f837813dbc0 2020-07-18T03:25:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 -- 172.21.15.47:0/1705151446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837813dbc0 msgr2=0x7f837813e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 --2- 172.21.15.47:0/1705151446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837813dbc0 0x7f837813e020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8368002f30 tx=0x7f8368006e90).stop 2020-07-18T03:25:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 -- 172.21.15.47:0/1705151446 shutdown_connections 2020-07-18T03:25:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 --2- 172.21.15.47:0/1705151446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8378147640 0x7f8378147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 --2- 172.21.15.47:0/1705151446 >> v2:172.21.15.18:3300/0 conn(0x7f83780ad020 0x7f83780b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 --2- 172.21.15.47:0/1705151446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f837813dbc0 0x7f837813e020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 -- 172.21.15.47:0/1705151446 >> 172.21.15.47:0/1705151446 conn(0x7f83780b93c0 msgr2=0x7f83780b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 -- 172.21.15.47:0/1705151446 shutdown_connections 2020-07-18T03:25:50.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.791+0000 7f837d5af700 1 -- 172.21.15.47:0/1705151446 wait complete. 2020-07-18T03:25:50.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.793+0000 7f837d5af700 1 Processor -- start 2020-07-18T03:25:50.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.793+0000 7f837d5af700 1 -- start start 2020-07-18T03:25:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.793+0000 7f837d5af700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83780b1750 0x7f837813d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.793+0000 7f837d5af700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780b37c0 con 0x7f83780ad020 2020-07-18T03:25:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.793+0000 7f837d5af700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837813e910 con 0x7f8378147640 2020-07-18T03:25:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.793+0000 7f837d5af700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83780b2120 con 0x7f83780b1750 2020-07-18T03:25:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83767fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8378147640 0x7f83780b3c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39600/0 (socket says 172.21.15.47:39600) 2020-07-18T03:25:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83767fc700 1 -- 172.21.15.47:0/2652026680 learned_addr learned my addr 172.21.15.47:0/2652026680 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83777fe700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83780b1750 0x7f837813d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353424635 0 0) 0x7f837813e910 con 0x7f8378147640 2020-07-18T03:25:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8354003240 con 0x7f8378147640 2020-07-18T03:25:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877947914 0 0) 0x7f83780b37c0 con 0x7f83780ad020 2020-07-18T03:25:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f837813e910 con 0x7f83780ad020 2020-07-18T03:25:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83777fe700 1 -- 172.21.15.47:0/2652026680 >> v1:172.21.15.47:6789/0 conn(0x7f8378147640 legacy=0x7f83780b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83777fe700 1 -- 172.21.15.47:0/2652026680 >> v1:172.21.15.18:6789/0 conn(0x7f83780ad020 legacy=0x7f83780662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83777fe700 1 -- 172.21.15.47:0/2652026680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780b1e80 con 0x7f83780b1750 2020-07-18T03:25:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f83777fe700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83780b1750 0x7f837813d320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f837000c000 tx=0x7f83700091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837000ca60 con 0x7f83780b1750 2020-07-18T03:25:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837000cbc0 con 0x7f83780b1750 2020-07-18T03:25:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83780b4330 con 0x7f83780b1750 2020-07-18T03:25:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83700123f0 con 0x7f83780b1750 2020-07-18T03:25:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.794+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83780b0810 con 0x7f83780b1750 2020-07-18T03:25:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.795+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8378057450 con 0x7f83780b1750 2020-07-18T03:25:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.795+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8370022070 con 0x7f83780b1750 2020-07-18T03:25:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f836effd700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83540504a0 0x7f8354052960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f83700681c0 con 0x7f83780b1750 2020-07-18T03:25:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f8376ffd700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83540504a0 0x7f8354052960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f836effd700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] conn(0x7f83540575d0 0x7f83540599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 --> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f835405a040 con 0x7f83540575d0 2020-07-18T03:25:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f837006a760 con 0x7f83780b1750 2020-07-18T03:25:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f8376ffd700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83540504a0 0x7f8354052960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f836000a000 tx=0x7f8360006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f83767fc700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] conn(0x7f83540575d0 0x7f83540599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.796+0000 7f83767fc700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] conn(0x7f83540575d0 0x7f83540599f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.797+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== osd.0 v2:172.21.15.47:6820/23865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7f835405a040 con 0x7f83540575d0 2020-07-18T03:25:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.815+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 --> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83780b3950 con 0x7f83540575d0 2020-07-18T03:25:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.815+0000 7f836effd700 1 -- 172.21.15.47:0/2652026680 <== osd.0 v2:172.21.15.47:6820/23865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f83780b3950 con 0x7f83540575d0 2020-07-18T03:25:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] conn(0x7f83540575d0 msgr2=0x7f83540599f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6820/23865,v1:172.21.15.47:6821/23865] conn(0x7f83540575d0 0x7f83540599f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83540504a0 msgr2=0x7f8354052960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83540504a0 0x7f8354052960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f836000a000 tx=0x7f8360006dd0).stop 2020-07-18T03:25:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83780b1750 msgr2=0x7f837813d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 --2- 172.21.15.47:0/2652026680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83780b1750 0x7f837813d320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f837000c000 tx=0x7f83700091d0).stop 2020-07-18T03:25:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f83777fe700 1 -- 172.21.15.47:0/2652026680 reap_dead start 2020-07-18T03:25:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 shutdown_connections 2020-07-18T03:25:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 >> 172.21.15.47:0/2652026680 conn(0x7f83780b93c0 msgr2=0x7f83780b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 shutdown_connections 2020-07-18T03:25:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.816+0000 7f837d5af700 1 -- 172.21.15.47:0/2652026680 wait complete. 2020-07-18T03:25:50.826 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:50.827 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-07-18T03:25:50.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.941+0000 7f7bda5e6700 1 Processor -- start 2020-07-18T03:25:50.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.941+0000 7f7bda5e6700 1 -- start start 2020-07-18T03:25:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.942+0000 7f7bda5e6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7bd41466f0 0x7f7bd4146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.942+0000 7f7bda5e6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd40607b0 con 0x7f7bd40666e0 2020-07-18T03:25:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.942+0000 7f7bda5e6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd405fdd0 con 0x7f7bd4065350 2020-07-18T03:25:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.942+0000 7f7bda5e6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd4147070 con 0x7f7bd41466f0 2020-07-18T03:25:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.942+0000 7f7bd8b83700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7bd4065350 0x7f7bd4065790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39612/0 (socket says 172.21.15.47:39612) 2020-07-18T03:25:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.942+0000 7f7bd8b83700 1 -- 172.21.15.47:0/1736521705 learned_addr learned my addr 172.21.15.47:0/1736521705 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694496586 0 0) 0x7f7bd405fdd0 con 0x7f7bd4065350 2020-07-18T03:25:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd37fe700 1 --2- 172.21.15.47:0/1736521705 >> v2:172.21.15.18:3300/0 conn(0x7f7bd41466f0 0x7f7bd4146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc0003240 con 0x7f7bd4065350 2020-07-18T03:25:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 477576391 0 0) 0x7f7bc0003240 con 0x7f7bd4065350 2020-07-18T03:25:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 >> v2:172.21.15.18:3300/0 conn(0x7f7bd41466f0 msgr2=0x7f7bd4146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 --2- 172.21.15.47:0/1736521705 >> v2:172.21.15.18:3300/0 conn(0x7f7bd41466f0 0x7f7bd4146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 >> v1:172.21.15.18:6789/0 conn(0x7f7bd40666e0 legacy=0x7f7bd40b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd4060070 con 0x7f7bd4065350 2020-07-18T03:25:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7bcc001a50 con 0x7f7bd4065350 2020-07-18T03:25:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 >> v1:172.21.15.47:6789/0 conn(0x7f7bd4065350 legacy=0x7f7bd4065790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc0003730 0x7f7bc0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7bc00044e0 0x7f7bc0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7bc0004e50 0x7f7bc000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0004320 con 0x7f7bc0004e50 2020-07-18T03:25:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc000bc20 con 0x7f7bc0003730 2020-07-18T03:25:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc000bd90 con 0x7f7bc00044e0 2020-07-18T03:25:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc0003730 0x7f7bc0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd37fe700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7bc00044e0 0x7f7bc0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd8b83700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7bc0004e50 0x7f7bc000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 -- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7bc00044e0 msgr2=0x7f7bc0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7bc00044e0 0x7f7bc0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 -- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7bc0004e50 msgr2=0x7f7bc000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:50.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7bc0004e50 0x7f7bc000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 -- 172.21.15.47:0/1736521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0003240 con 0x7f7bc0003730 2020-07-18T03:25:50.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd8b83700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7bc0004e50 0x7f7bc000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd8b83700 1 -- 172.21.15.47:0/1736521705 reap_dead start 2020-07-18T03:25:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.943+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc0003730 0x7f7bc0003b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f7bd40689a0 tx=0x7f7bc4007b70).ready entity=mon.1 client_cookie=69a551c3a2a2619f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bc400c660 con 0x7f7bc0003730 2020-07-18T03:25:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc400c7c0 con 0x7f7bc0003730 2020-07-18T03:25:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bd2ffd700 1 -- 172.21.15.47:0/1736521705 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bc400cb20 con 0x7f7bc0003730 2020-07-18T03:25:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1736521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc0003730 msgr2=0x7f7bc0003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc0003730 0x7f7bc0003b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f7bd40689a0 tx=0x7f7bc4007b70).stop 2020-07-18T03:25:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1736521705 shutdown_connections 2020-07-18T03:25:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 --2- 172.21.15.47:0/1736521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bc0003730 0x7f7bc0003b50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1736521705 >> 172.21.15.47:0/1736521705 conn(0x7f7bd40bc0a0 msgr2=0x7f7bd40b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1736521705 shutdown_connections 2020-07-18T03:25:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.944+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1736521705 wait complete. 2020-07-18T03:25:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.945+0000 7f7bda5e6700 1 Processor -- start 2020-07-18T03:25:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.945+0000 7f7bda5e6700 1 -- start start 2020-07-18T03:25:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bda5e6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7bd4158a30 0x7f7bd414a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bda5e6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd40b7530 con 0x7f7bd4158650 2020-07-18T03:25:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bda5e6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd40b77d0 con 0x7f7bd414ab50 2020-07-18T03:25:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bda5e6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd40b7290 con 0x7f7bd4158a30 2020-07-18T03:25:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7bd4158a30 0x7f7bd414a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd8b83700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7bd414ab50 0x7f7bd40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39624/0 (socket says 172.21.15.47:39624) 2020-07-18T03:25:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd8b83700 1 -- 172.21.15.47:0/1919813450 learned_addr learned my addr 172.21.15.47:0/1919813450 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd37fe700 1 -- 172.21.15.47:0/1919813450 >> v1:172.21.15.47:6789/0 conn(0x7f7bd414ab50 legacy=0x7f7bd40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd37fe700 1 -- 172.21.15.47:0/1919813450 >> v1:172.21.15.18:6789/0 conn(0x7f7bd4158650 legacy=0x7f7bd40b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd37fe700 1 -- 172.21.15.47:0/1919813450 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd4158fc0 con 0x7f7bd4158a30 2020-07-18T03:25:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663030012 0 0) 0x7f7bd40b77d0 con 0x7f7bd414ab50 2020-07-18T03:25:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.946+0000 7f7bd37fe700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bd4158a30 0x7f7bd414a660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f7bc8008000 tx=0x7f7bc8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.947+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bc800f910 con 0x7f7bd4158a30 2020-07-18T03:25:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.947+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800fa70 con 0x7f7bd4158a30 2020-07-18T03:25:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.947+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd40b2fb0 con 0x7f7bd4158a30 2020-07-18T03:25:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.947+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bc800fdd0 con 0x7f7bd4158a30 2020-07-18T03:25:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.947+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd40b3480 con 0x7f7bd4158a30 2020-07-18T03:25:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.947+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7bd40b7960 con 0x7f7bd4158a30 2020-07-18T03:25:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7bc8014440 con 0x7f7bd4158a30 2020-07-18T03:25:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bb3fff700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7bac04fdc0 0x7f7bac052280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7bc8068690 con 0x7f7bd4158a30 2020-07-18T03:25:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7bac04fdc0 0x7f7bac052280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bb3fff700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] conn(0x7f7bac056ff0 0x7f7bac059410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 --> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bac059a60 con 0x7f7bac056ff0 2020-07-18T03:25:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc806ad00 con 0x7f7bd4158a30 2020-07-18T03:25:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bd8b83700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] conn(0x7f7bac056ff0 0x7f7bac059410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.948+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7bac04fdc0 0x7f7bac052280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7bd40b6130 tx=0x7f7bc4016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.949+0000 7f7bd8b83700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] conn(0x7f7bac056ff0 0x7f7bac059410 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.950+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== osd.1 v2:172.21.15.47:6802/23853 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7f7bac059a60 con 0x7f7bac056ff0 2020-07-18T03:25:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 --> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7bd4057450 con 0x7f7bac056ff0 2020-07-18T03:25:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bb3fff700 1 -- 172.21.15.47:0/1919813450 <== osd.1 v2:172.21.15.47:6802/23853 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7bd4057450 con 0x7f7bac056ff0 2020-07-18T03:25:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] conn(0x7f7bac056ff0 msgr2=0x7f7bac059410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6802/23853,v1:172.21.15.47:6803/23853] conn(0x7f7bac056ff0 0x7f7bac059410 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7bac04fdc0 msgr2=0x7f7bac052280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7bac04fdc0 0x7f7bac052280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7bd40b6130 tx=0x7f7bc4016000).stop 2020-07-18T03:25:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bd4158a30 msgr2=0x7f7bd414a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bda5e6700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bd4158a30 0x7f7bd414a660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f7bc8008000 tx=0x7f7bc8008040).stop 2020-07-18T03:25:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.968+0000 7f7bd8b83700 1 -- 172.21.15.47:0/1919813450 reap_dead start 2020-07-18T03:25:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.969+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 shutdown_connections 2020-07-18T03:25:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.969+0000 7f7bda5e6700 1 --2- 172.21.15.47:0/1919813450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bd4158a30 0x7f7bd414a660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.969+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 >> 172.21.15.47:0/1919813450 conn(0x7f7bd40bc0a0 msgr2=0x7f7bd40baa30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.969+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 shutdown_connections 2020-07-18T03:25:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:50.969+0000 7f7bda5e6700 1 -- 172.21.15.47:0/1919813450 wait complete. 2020-07-18T03:25:50.979 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:50.980 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-07-18T03:25:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.094+0000 7f6d19e49700 1 Processor -- start 2020-07-18T03:25:51.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.094+0000 7f6d19e49700 1 -- start start 2020-07-18T03:25:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d19e49700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d14149040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d19e49700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6d14062c80 0x7f6d140b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d19e49700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d140b6880 0x7f6d141447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d19e49700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d14148900 con 0x7f6d14062c80 2020-07-18T03:25:51.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d19e49700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d14148ba0 con 0x7f6d140b6880 2020-07-18T03:25:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d19e49700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14063260 con 0x7f6d140628a0 2020-07-18T03:25:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d13fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d140b6880 0x7f6d141447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d13fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d140b6880 0x7f6d141447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58546/0 (socket says 172.21.15.47:58546) 2020-07-18T03:25:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.095+0000 7f6d13fff700 1 -- 172.21.15.47:0/1942168493 learned_addr learned my addr 172.21.15.47:0/1942168493 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d137fe700 1 --2- 172.21.15.47:0/1942168493 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d14149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d12ffd700 1 --2- 172.21.15.47:0/1942168493 >> v2:172.21.15.18:3300/0 conn(0x7f6d14062c80 0x7f6d140b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d13fff700 1 -- 172.21.15.47:0/1942168493 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 msgr2=0x7f6d14149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d13fff700 1 --2- 172.21.15.47:0/1942168493 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d14149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d13fff700 1 -- 172.21.15.47:0/1942168493 >> v2:172.21.15.18:3300/0 conn(0x7f6d14062c80 msgr2=0x7f6d140b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d13fff700 1 --2- 172.21.15.47:0/1942168493 >> v2:172.21.15.18:3300/0 conn(0x7f6d14062c80 0x7f6d140b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d13fff700 1 -- 172.21.15.47:0/1942168493 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140b6cf0 con 0x7f6d140b6880 2020-07-18T03:25:51.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d13fff700 1 --2- 172.21.15.47:0/1942168493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d140b6880 0x7f6d141447c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6d0c0032b0 tx=0x7f6d0c009270).ready entity=mon.1 client_cookie=f85056bae39e93e6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d127fc700 1 -- 172.21.15.47:0/1942168493 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d0c00bc00 con 0x7f6d140b6880 2020-07-18T03:25:51.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d127fc700 1 -- 172.21.15.47:0/1942168493 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c00bd60 con 0x7f6d140b6880 2020-07-18T03:25:51.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d127fc700 1 -- 172.21.15.47:0/1942168493 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d0c011400 con 0x7f6d140b6880 2020-07-18T03:25:51.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 -- 172.21.15.47:0/1942168493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d140b6880 msgr2=0x7f6d141447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 --2- 172.21.15.47:0/1942168493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d140b6880 0x7f6d141447c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6d0c0032b0 tx=0x7f6d0c009270).stop 2020-07-18T03:25:51.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 -- 172.21.15.47:0/1942168493 shutdown_connections 2020-07-18T03:25:51.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 --2- 172.21.15.47:0/1942168493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d140b6880 0x7f6d141447c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 --2- 172.21.15.47:0/1942168493 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d14149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 --2- 172.21.15.47:0/1942168493 >> v2:172.21.15.18:3300/0 conn(0x7f6d14062c80 0x7f6d140b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 -- 172.21.15.47:0/1942168493 >> 172.21.15.47:0/1942168493 conn(0x7f6d1405f5a0 msgr2=0x7f6d1405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 -- 172.21.15.47:0/1942168493 shutdown_connections 2020-07-18T03:25:51.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.096+0000 7f6d19e49700 1 -- 172.21.15.47:0/1942168493 wait complete. 2020-07-18T03:25:51.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.098+0000 7f6d19e49700 1 Processor -- start 2020-07-18T03:25:51.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.098+0000 7f6d19e49700 1 -- start start 2020-07-18T03:25:51.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.098+0000 7f6d19e49700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d140b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d14062c80 0x7f6d14135980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6d14135e70 0x7f6d140b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140b02e0 con 0x7f6d14135e70 2020-07-18T03:25:51.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140b0580 con 0x7f6d14062c80 2020-07-18T03:25:51.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d140b0820 con 0x7f6d140628a0 2020-07-18T03:25:51.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d14062c80 0x7f6d14135980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d14062c80 0x7f6d14135980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58550/0 (socket says 172.21.15.47:58550) 2020-07-18T03:25:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 -- 172.21.15.47:0/2860088788 learned_addr learned my addr 172.21.15.47:0/2860088788 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d13fff700 1 --2- 172.21.15.47:0/2860088788 >> v2:172.21.15.18:3300/0 conn(0x7f6d14135e70 0x7f6d140b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d137fe700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d140b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 -- 172.21.15.47:0/2860088788 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 msgr2=0x7f6d140b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d140628a0 0x7f6d140b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 -- 172.21.15.47:0/2860088788 >> v2:172.21.15.18:3300/0 conn(0x7f6d14135e70 msgr2=0x7f6d140b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 --2- 172.21.15.47:0/2860088788 >> v2:172.21.15.18:3300/0 conn(0x7f6d14135e70 0x7f6d140b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 -- 172.21.15.47:0/2860088788 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d141362e0 con 0x7f6d14062c80 2020-07-18T03:25:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d13fff700 1 --2- 172.21.15.47:0/2860088788 >> v2:172.21.15.18:3300/0 conn(0x7f6d14135e70 0x7f6d140b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:25:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d12ffd700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d14062c80 0x7f6d14135980 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6d08003070 tx=0x7f6d08008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d08009980 con 0x7f6d14062c80 2020-07-18T03:25:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08009ae0 con 0x7f6d14062c80 2020-07-18T03:25:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d141420a0 con 0x7f6d14062c80 2020-07-18T03:25:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d08009e40 con 0x7f6d14062c80 2020-07-18T03:25:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.099+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d140b22f0 con 0x7f6d14062c80 2020-07-18T03:25:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.100+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d14057450 con 0x7f6d14062c80 2020-07-18T03:25:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.100+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6d08013480 con 0x7f6d14062c80 2020-07-18T03:25:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6cf37fe700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6cec04ffd0 0x7f6cec052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6d080681c0 con 0x7f6d14062c80 2020-07-18T03:25:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6d137fe700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6cec04ffd0 0x7f6cec052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6cf37fe700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] conn(0x7f6cec057380 0x7f6cec0597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 --> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cec059df0 con 0x7f6cec057380 2020-07-18T03:25:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d0806a7d0 con 0x7f6d14062c80 2020-07-18T03:25:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6d13fff700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] conn(0x7f6cec057380 0x7f6cec0597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6d137fe700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6cec04ffd0 0x7f6cec052490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6d04008000 tx=0x7f6d04008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.101+0000 7f6d13fff700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] conn(0x7f6cec057380 0x7f6cec0597a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.102+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== osd.2 v2:172.21.15.47:6818/23872 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7f6cec059df0 con 0x7f6cec057380 2020-07-18T03:25:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.120+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 --> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d140b2790 con 0x7f6cec057380 2020-07-18T03:25:51.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.120+0000 7f6cf37fe700 1 -- 172.21.15.47:0/2860088788 <== osd.2 v2:172.21.15.47:6818/23872 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6d140b2790 con 0x7f6cec057380 2020-07-18T03:25:51.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.120+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] conn(0x7f6cec057380 msgr2=0x7f6cec0597a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.120+0000 7f6d19e49700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6818/23872,v1:172.21.15.47:6819/23872] conn(0x7f6cec057380 0x7f6cec0597a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6cec04ffd0 msgr2=0x7f6cec052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6cec04ffd0 0x7f6cec052490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6d04008000 tx=0x7f6d04008040).stop 2020-07-18T03:25:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d14062c80 msgr2=0x7f6d14135980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d14062c80 0x7f6d14135980 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6d08003070 tx=0x7f6d08008040).stop 2020-07-18T03:25:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d13fff700 1 -- 172.21.15.47:0/2860088788 reap_dead start 2020-07-18T03:25:51.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 shutdown_connections 2020-07-18T03:25:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 --2- 172.21.15.47:0/2860088788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d14062c80 0x7f6d14135980 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 >> 172.21.15.47:0/2860088788 conn(0x7f6d1405f5a0 msgr2=0x7f6d1405d340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 shutdown_connections 2020-07-18T03:25:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.121+0000 7f6d19e49700 1 -- 172.21.15.47:0/2860088788 wait complete. 2020-07-18T03:25:51.140 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:51.141 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-07-18T03:25:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.267+0000 7f0d3d1ad700 1 Processor -- start 2020-07-18T03:25:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.267+0000 7f0d3d1ad700 1 -- start start 2020-07-18T03:25:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d3d1ad700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d380666e0 0x7f0d380b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d3d1ad700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d380607b0 con 0x7f0d381466f0 2020-07-18T03:25:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d3d1ad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d380b2250 con 0x7f0d38146b50 2020-07-18T03:25:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d3d1ad700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3805fdd0 con 0x7f0d380666e0 2020-07-18T03:25:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d3759e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d38146b50 0x7f0d380b1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39646/0 (socket says 172.21.15.47:39646) 2020-07-18T03:25:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d3759e700 1 -- 172.21.15.47:0/2557324396 learned_addr learned my addr 172.21.15.47:0/2557324396 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d36d9d700 1 --2- 172.21.15.47:0/2557324396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d380666e0 0x7f0d380b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.268+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220275882 0 0) 0x7f0d380b2250 con 0x7f0d38146b50 2020-07-18T03:25:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d18003240 con 0x7f0d38146b50 2020-07-18T03:25:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829367312 0 0) 0x7f0d380607b0 con 0x7f0d381466f0 2020-07-18T03:25:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d380b2250 con 0x7f0d381466f0 2020-07-18T03:25:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d36d9d700 1 -- 172.21.15.47:0/2557324396 >> v1:172.21.15.47:6789/0 conn(0x7f0d38146b50 legacy=0x7f0d380b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1956514300 0 0) 0x7f0d18003240 con 0x7f0d38146b50 2020-07-18T03:25:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d36d9d700 1 -- 172.21.15.47:0/2557324396 >> v1:172.21.15.18:6789/0 conn(0x7f0d381466f0 legacy=0x7f0d380b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d36d9d700 1 -- 172.21.15.47:0/2557324396 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d38060070 con 0x7f0d380666e0 2020-07-18T03:25:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d36d9d700 1 --2- 172.21.15.47:0/2557324396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d380666e0 0x7f0d380b8940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f0d28001450 tx=0x7f0d30006dd0).ready entity=mon.2 client_cookie=9678d61c5480dfa1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d28001a50 con 0x7f0d380666e0 2020-07-18T03:25:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3000d5c0 con 0x7f0d380666e0 2020-07-18T03:25:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d35d9b700 1 -- 172.21.15.47:0/2557324396 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d3000d730 con 0x7f0d380666e0 2020-07-18T03:25:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/2557324396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d380666e0 msgr2=0x7f0d380b8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 --2- 172.21.15.47:0/2557324396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d380666e0 0x7f0d380b8940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f0d28001450 tx=0x7f0d30006dd0).stop 2020-07-18T03:25:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/2557324396 shutdown_connections 2020-07-18T03:25:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 --2- 172.21.15.47:0/2557324396 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0d380666e0 0x7f0d380b8940 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/2557324396 >> 172.21.15.47:0/2557324396 conn(0x7f0d380bc0a0 msgr2=0x7f0d380b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/2557324396 shutdown_connections 2020-07-18T03:25:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.269+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/2557324396 wait complete. 2020-07-18T03:25:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 Processor -- start 2020-07-18T03:25:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 -- start start 2020-07-18T03:25:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0d381466f0 0x7f0d380b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d38158be0 0x7f0d38159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d38159840 con 0x7f0d38146b50 2020-07-18T03:25:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380b7360 con 0x7f0d381466f0 2020-07-18T03:25:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.271+0000 7f0d3d1ad700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380b6e30 con 0x7f0d38158be0 2020-07-18T03:25:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d38158be0 0x7f0d38159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d38146b50 0x7f0d38147f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39652/0 (socket says 172.21.15.47:39652) 2020-07-18T03:25:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d38158be0 0x7f0d38159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58566/0 (socket says 172.21.15.47:58566) 2020-07-18T03:25:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3659c700 1 -- 172.21.15.47:0/326323391 learned_addr learned my addr 172.21.15.47:0/326323391 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d36d9d700 1 --2- 172.21.15.47:0/326323391 >> v2:172.21.15.18:3300/0 conn(0x7f0d381466f0 0x7f0d380b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204243404 0 0) 0x7f0d38159840 con 0x7f0d38146b50 2020-07-18T03:25:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d14003240 con 0x7f0d38146b50 2020-07-18T03:25:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 -- 172.21.15.47:0/326323391 >> v2:172.21.15.18:3300/0 conn(0x7f0d381466f0 msgr2=0x7f0d380b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 --2- 172.21.15.47:0/326323391 >> v2:172.21.15.18:3300/0 conn(0x7f0d381466f0 0x7f0d380b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 -- 172.21.15.47:0/326323391 >> v1:172.21.15.47:6789/0 conn(0x7f0d38146b50 legacy=0x7f0d38147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 -- 172.21.15.47:0/326323391 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d380b6b80 con 0x7f0d38158be0 2020-07-18T03:25:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3759e700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d38158be0 0x7f0d38159030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0d28001320 tx=0x7f0d28002140).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2800cea0 con 0x7f0d38158be0 2020-07-18T03:25:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d38159520 con 0x7f0d38158be0 2020-07-18T03:25:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d38148650 con 0x7f0d38158be0 2020-07-18T03:25:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28004a90 con 0x7f0d38158be0 2020-07-18T03:25:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.272+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d28004c40 con 0x7f0d38158be0 2020-07-18T03:25:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.273+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d38057450 con 0x7f0d38158be0 2020-07-18T03:25:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.273+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0d28010070 con 0x7f0d38158be0 2020-07-18T03:25:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d2effd700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0d14050310 0x7f0d140527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0d28067860 con 0x7f0d38158be0 2020-07-18T03:25:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d36d9d700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0d14050310 0x7f0d140527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d2effd700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] conn(0x7f0d140573b0 0x7f0d140597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 --> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d14059e20 con 0x7f0d140573b0 2020-07-18T03:25:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d2806c220 con 0x7f0d38158be0 2020-07-18T03:25:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d3659c700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] conn(0x7f0d140573b0 0x7f0d140597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d36d9d700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0d14050310 0x7f0d140527d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0d3001c000 tx=0x7f0d30015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.274+0000 7f0d3659c700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] conn(0x7f0d140573b0 0x7f0d140597d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.275+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== osd.3 v2:172.21.15.47:6810/23854 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7f0d20012b80 con 0x7f0d140573b0 2020-07-18T03:25:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.293+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 --> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0d3805cb70 con 0x7f0d140573b0 2020-07-18T03:25:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.293+0000 7f0d2effd700 1 -- 172.21.15.47:0/326323391 <== osd.3 v2:172.21.15.47:6810/23854 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0d3805cb70 con 0x7f0d140573b0 2020-07-18T03:25:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] conn(0x7f0d140573b0 msgr2=0x7f0d140597d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6810/23854,v1:172.21.15.47:6811/23854] conn(0x7f0d140573b0 0x7f0d140597d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0d14050310 msgr2=0x7f0d140527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0d14050310 0x7f0d140527d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0d3001c000 tx=0x7f0d30015000).stop 2020-07-18T03:25:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d38158be0 msgr2=0x7f0d38159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d38158be0 0x7f0d38159030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0d28001320 tx=0x7f0d28002140).stop 2020-07-18T03:25:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3759e700 1 -- 172.21.15.47:0/326323391 reap_dead start 2020-07-18T03:25:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 shutdown_connections 2020-07-18T03:25:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 --2- 172.21.15.47:0/326323391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d38158be0 0x7f0d38159030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 >> 172.21.15.47:0/326323391 conn(0x7f0d380bc0a0 msgr2=0x7f0d380652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 shutdown_connections 2020-07-18T03:25:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.294+0000 7f0d3d1ad700 1 -- 172.21.15.47:0/326323391 wait complete. 2020-07-18T03:25:51.304 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:51.305 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-07-18T03:25:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.420+0000 7f07ec0ef700 1 Processor -- start 2020-07-18T03:25:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.420+0000 7f07ec0ef700 1 -- start start 2020-07-18T03:25:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07ec0ef700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 0x7f07e40625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07ec0ef700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40b5830 con 0x7f07e41471d0 2020-07-18T03:25:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07ec0ef700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e4146da0 con 0x7f07e40aea60 2020-07-18T03:25:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07ec0ef700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4147d70 con 0x7f07e405ffb0 2020-07-18T03:25:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07e968a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f07e40aea60 0x7f07e4142e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39662/0 (socket says 172.21.15.47:39662) 2020-07-18T03:25:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07e968a700 1 -- 172.21.15.47:0/2813369759 learned_addr learned my addr 172.21.15.47:0/2813369759 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.421+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 0x7f07e40625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003719984 0 0) 0x7f07e4146da0 con 0x7f07e40aea60 2020-07-18T03:25:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07cc003240 con 0x7f07e40aea60 2020-07-18T03:25:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639914801 0 0) 0x7f07e40b5830 con 0x7f07e41471d0 2020-07-18T03:25:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e4146da0 con 0x7f07e41471d0 2020-07-18T03:25:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1526896005 0 0) 0x7f07cc003240 con 0x7f07e40aea60 2020-07-18T03:25:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 msgr2=0x7f07e40625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 0x7f07e40625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 >> v1:172.21.15.18:6789/0 conn(0x7f07e41471d0 legacy=0x7f07e4147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40b5a60 con 0x7f07e40aea60 2020-07-18T03:25:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f07e0001710 con 0x7f07e40aea60 2020-07-18T03:25:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 >> v1:172.21.15.47:6789/0 conn(0x7f07e40aea60 legacy=0x7f07e4142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07cc0038c0 0x7f07cc003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07cc0044e0 0x7f07cc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 0x7f07cc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4147d70 con 0x7f07cc0044e0 2020-07-18T03:25:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc005340 con 0x7f07cc0038c0 2020-07-18T03:25:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00b760 con 0x7f07e405ffb0 2020-07-18T03:25:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07cc0038c0 0x7f07cc003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e968a700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07cc0044e0 0x7f07cc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07ea68c700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 0x7f07cc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 -- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 msgr2=0x7f07cc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e405ffb0 0x7f07cc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 -- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07cc0044e0 msgr2=0x7f07cc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07cc0044e0 0x7f07cc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 -- 172.21.15.47:0/2813369759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc003240 con 0x7f07cc0038c0 2020-07-18T03:25:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e968a700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07cc0044e0 0x7f07cc004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.422+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07cc0038c0 0x7f07cc003cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f07cc004450 tx=0x7f07d4008040).ready entity=mon.1 client_cookie=75908eb474d15e83 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d400e970 con 0x7f07cc0038c0 2020-07-18T03:25:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d400ead0 con 0x7f07cc0038c0 2020-07-18T03:25:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07e8e89700 1 -- 172.21.15.47:0/2813369759 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d400ee30 con 0x7f07cc0038c0 2020-07-18T03:25:51.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ec0ef700 1 -- 172.21.15.47:0/2813369759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07cc0038c0 msgr2=0x7f07cc003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ec0ef700 1 --2- 172.21.15.47:0/2813369759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07cc0038c0 0x7f07cc003cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f07cc004450 tx=0x7f07d4008040).stop 2020-07-18T03:25:51.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ea68c700 1 -- 172.21.15.47:0/2813369759 reap_dead start 2020-07-18T03:25:51.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ec0ef700 1 -- 172.21.15.47:0/2813369759 shutdown_connections 2020-07-18T03:25:51.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ec0ef700 1 -- 172.21.15.47:0/2813369759 >> 172.21.15.47:0/2813369759 conn(0x7f07e40b93c0 msgr2=0x7f07e40b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ec0ef700 1 -- 172.21.15.47:0/2813369759 shutdown_connections 2020-07-18T03:25:51.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.423+0000 7f07ec0ef700 1 -- 172.21.15.47:0/2813369759 wait complete. 2020-07-18T03:25:51.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.424+0000 7f07ec0ef700 1 Processor -- start 2020-07-18T03:25:51.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.424+0000 7f07ec0ef700 1 -- start start 2020-07-18T03:25:51.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07e4146460 0x7f07e4145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f07e4146840 0x7f07e4065950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e4143520 0x7f07e4065e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e4145020 con 0x7f07e4146840 2020-07-18T03:25:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e41452c0 con 0x7f07e4146460 2020-07-18T03:25:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4145560 con 0x7f07e4143520 2020-07-18T03:25:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07e4146460 0x7f07e4145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07e4146460 0x7f07e4145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58584/0 (socket says 172.21.15.47:58584) 2020-07-18T03:25:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 -- 172.21.15.47:0/3488551696 learned_addr learned my addr 172.21.15.47:0/3488551696 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ea68c700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e4143520 0x7f07e4065e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e968a700 1 --2- 172.21.15.47:0/3488551696 >> v2:172.21.15.18:3300/0 conn(0x7f07e4146840 0x7f07e4065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 -- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e4143520 msgr2=0x7f07e4065e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f07e4143520 0x7f07e4065e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 -- 172.21.15.47:0/3488551696 >> v2:172.21.15.18:3300/0 conn(0x7f07e4146840 msgr2=0x7f07e4065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/3488551696 >> v2:172.21.15.18:3300/0 conn(0x7f07e4146840 0x7f07e4065950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 -- 172.21.15.47:0/3488551696 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4066330 con 0x7f07e4146460 2020-07-18T03:25:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e968a700 1 --2- 172.21.15.47:0/3488551696 >> v2:172.21.15.18:3300/0 conn(0x7f07e4146840 0x7f07e4065950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07e9e8b700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07e4146460 0x7f07e4145a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f07cc004450 tx=0x7f07d401a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d40129c0 con 0x7f07e4146460 2020-07-18T03:25:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d4012b20 con 0x7f07e4146460 2020-07-18T03:25:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e4145ef0 con 0x7f07e4146460 2020-07-18T03:25:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.425+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d4012e80 con 0x7f07e4146460 2020-07-18T03:25:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.426+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e4156750 con 0x7f07e4146460 2020-07-18T03:25:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.426+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07e4057450 con 0x7f07e4146460 2020-07-18T03:25:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f07d402a020 con 0x7f07e4146460 2020-07-18T03:25:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07d1ffb700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f07bc04ffd0 0x7f07bc052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f07d4069a90 con 0x7f07e4146460 2020-07-18T03:25:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07e968a700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f07bc04ffd0 0x7f07bc052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07d1ffb700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] conn(0x7f07bc057380 0x7f07bc0597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 --> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07bc059df0 con 0x7f07bc057380 2020-07-18T03:25:51.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07d406e220 con 0x7f07e4146460 2020-07-18T03:25:51.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07e968a700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f07bc04ffd0 0x7f07bc052490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f07e000d000 tx=0x7f07e0009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.427+0000 7f07ea68c700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] conn(0x7f07bc057380 0x7f07bc0597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.428+0000 7f07ea68c700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] conn(0x7f07bc057380 0x7f07bc0597a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.429+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== osd.4 v2:172.21.15.18:6808/24092 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7f07bc059df0 con 0x7f07bc057380 2020-07-18T03:25:51.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.447+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 --> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f07e40b8940 con 0x7f07bc057380 2020-07-18T03:25:51.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07d1ffb700 1 -- 172.21.15.47:0/3488551696 <== osd.4 v2:172.21.15.18:6808/24092 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07e40b8940 con 0x7f07bc057380 2020-07-18T03:25:51.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] conn(0x7f07bc057380 msgr2=0x7f07bc0597a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.18:6808/24092,v1:172.21.15.18:6809/24092] conn(0x7f07bc057380 0x7f07bc0597a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f07bc04ffd0 msgr2=0x7f07bc052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f07bc04ffd0 0x7f07bc052490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f07e000d000 tx=0x7f07e0009040).stop 2020-07-18T03:25:51.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07e4146460 msgr2=0x7f07e4145a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07e4146460 0x7f07e4145a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f07cc004450 tx=0x7f07d401a000).stop 2020-07-18T03:25:51.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ea68c700 1 -- 172.21.15.47:0/3488551696 reap_dead start 2020-07-18T03:25:51.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 shutdown_connections 2020-07-18T03:25:51.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 --2- 172.21.15.47:0/3488551696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07e4146460 0x7f07e4145a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 >> 172.21.15.47:0/3488551696 conn(0x7f07e40b93c0 msgr2=0x7f07e405f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 shutdown_connections 2020-07-18T03:25:51.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.448+0000 7f07ec0ef700 1 -- 172.21.15.47:0/3488551696 wait complete. 2020-07-18T03:25:51.461 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:51.462 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-07-18T03:25:51.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.590+0000 7fd13a219700 1 Processor -- start 2020-07-18T03:25:51.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.590+0000 7fd13a219700 1 -- start start 2020-07-18T03:25:51.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd13a219700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd13405ffb0 0x7fd1340625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd13a219700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340b5830 con 0x7fd1340aea60 2020-07-18T03:25:51.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd13a219700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd134146da0 con 0x7fd1341471d0 2020-07-18T03:25:51.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd13a219700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd134147d30 con 0x7fd13405ffb0 2020-07-18T03:25:51.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd133fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd1341471d0 0x7fd134147620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39686/0 (socket says 172.21.15.47:39686) 2020-07-18T03:25:51.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd133fff700 1 -- 172.21.15.47:0/3688855224 learned_addr learned my addr 172.21.15.47:0/3688855224 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877782771 0 0) 0x7fd134146da0 con 0x7fd1341471d0 2020-07-18T03:25:51.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd114003240 con 0x7fd1341471d0 2020-07-18T03:25:51.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.591+0000 7fd1337fe700 1 --2- 172.21.15.47:0/3688855224 >> v2:172.21.15.18:3300/0 conn(0x7fd13405ffb0 0x7fd1340625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3034998142 0 0) 0x7fd114003240 con 0x7fd1341471d0 2020-07-18T03:25:51.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 >> v2:172.21.15.18:3300/0 conn(0x7fd13405ffb0 msgr2=0x7fd1340625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 --2- 172.21.15.47:0/3688855224 >> v2:172.21.15.18:3300/0 conn(0x7fd13405ffb0 0x7fd1340625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 >> v1:172.21.15.18:6789/0 conn(0x7fd1340aea60 legacy=0x7fd134142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340b5a60 con 0x7fd1341471d0 2020-07-18T03:25:51.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fd1240016d0 con 0x7fd1341471d0 2020-07-18T03:25:51.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 >> v1:172.21.15.47:6789/0 conn(0x7fd1341471d0 legacy=0x7fd134147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd114003730 0x7fd114003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1140044e0 0x7fd114004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd114004e50 0x7fd11400c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134147d30 con 0x7fd1140044e0 2020-07-18T03:25:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114004350 con 0x7fd114003730 2020-07-18T03:25:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd114005700 con 0x7fd114004e50 2020-07-18T03:25:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd114003730 0x7fd114003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd132ffd700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1140044e0 0x7fd114004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd133fff700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd114004e50 0x7fd11400c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 -- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd114004e50 msgr2=0x7fd11400c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd114004e50 0x7fd11400c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 -- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1140044e0 msgr2=0x7fd114004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1140044e0 0x7fd114004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 -- 172.21.15.47:0/3688855224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd114003240 con 0x7fd114003730 2020-07-18T03:25:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd133fff700 1 -- 172.21.15.47:0/3688855224 reap_dead start 2020-07-18T03:25:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1337fe700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd114003730 0x7fd114003b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd12c00c000 tx=0x7fd12c0091d0).ready entity=mon.1 client_cookie=cf53bdb1c84a509c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd12c00ebd0 con 0x7fd114003730 2020-07-18T03:25:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c00ed30 con 0x7fd114003730 2020-07-18T03:25:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd1327fc700 1 -- 172.21.15.47:0/3688855224 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd12c00eea0 con 0x7fd114003730 2020-07-18T03:25:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd13a219700 1 -- 172.21.15.47:0/3688855224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd114003730 msgr2=0x7fd114003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd13a219700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd114003730 0x7fd114003b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd12c00c000 tx=0x7fd12c0091d0).stop 2020-07-18T03:25:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd13a219700 1 -- 172.21.15.47:0/3688855224 shutdown_connections 2020-07-18T03:25:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd13a219700 1 --2- 172.21.15.47:0/3688855224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd114003730 0x7fd114003b50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.592+0000 7fd13a219700 1 -- 172.21.15.47:0/3688855224 >> 172.21.15.47:0/3688855224 conn(0x7fd1340b93c0 msgr2=0x7fd1340b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.593+0000 7fd13a219700 1 -- 172.21.15.47:0/3688855224 shutdown_connections 2020-07-18T03:25:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.593+0000 7fd13a219700 1 -- 172.21.15.47:0/3688855224 wait complete. 2020-07-18T03:25:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.594+0000 7fd13a219700 1 Processor -- start 2020-07-18T03:25:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.594+0000 7fd13a219700 1 -- start start 2020-07-18T03:25:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd13a219700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd13413a560 0x7fd13413a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd13a219700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13413aec0 con 0x7fd13413fe20 2020-07-18T03:25:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd13a219700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340b3150 con 0x7fd134158a20 2020-07-18T03:25:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd13a219700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1340b1020 con 0x7fd13413a560 2020-07-18T03:25:51.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd1337fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd134158a20 0x7fd134158e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39694/0 (socket says 172.21.15.47:39694) 2020-07-18T03:25:51.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd1337fe700 1 -- 172.21.15.47:0/1133269075 learned_addr learned my addr 172.21.15.47:0/1133269075 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd133fff700 1 --2- 172.21.15.47:0/1133269075 >> v2:172.21.15.47:3300/0 conn(0x7fd13413a560 0x7fd13413a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd133fff700 1 -- 172.21.15.47:0/1133269075 >> v1:172.21.15.47:6789/0 conn(0x7fd134158a20 legacy=0x7fd134158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd133fff700 1 -- 172.21.15.47:0/1133269075 >> v1:172.21.15.18:6789/0 conn(0x7fd13413fe20 legacy=0x7fd1340b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424890350 0 0) 0x7fd1340b3150 con 0x7fd134158a20 2020-07-18T03:25:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd133fff700 1 -- 172.21.15.47:0/1133269075 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340b2dd0 con 0x7fd13413a560 2020-07-18T03:25:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd133fff700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd13413a560 0x7fd13413a9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd1340b2140 tx=0x7fd1240076d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd12400fa90 con 0x7fd13413a560 2020-07-18T03:25:51.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12400fbf0 con 0x7fd13413a560 2020-07-18T03:25:51.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1340b1bd0 con 0x7fd13413a560 2020-07-18T03:25:51.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.595+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd124012350 con 0x7fd13413a560 2020-07-18T03:25:51.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.596+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd134155480 con 0x7fd13413a560 2020-07-18T03:25:51.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.596+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd13405fd90 con 0x7fd13413a560 2020-07-18T03:25:51.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.596+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd1240124b0 con 0x7fd13413a560 2020-07-18T03:25:51.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd12b7fe700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd11004fe40 0x7fd110052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd124067200 con 0x7fd13413a560 2020-07-18T03:25:51.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd1337fe700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd11004fe40 0x7fd110052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd12b7fe700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] conn(0x7fd110057070 0x7fd110059490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 --> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd110059ae0 con 0x7fd110057070 2020-07-18T03:25:51.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd124069870 con 0x7fd13413a560 2020-07-18T03:25:51.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd1337fe700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd11004fe40 0x7fd110052300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd12c000f00 tx=0x7fd12c01a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.597+0000 7fd132ffd700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] conn(0x7fd110057070 0x7fd110059490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.598+0000 7fd132ffd700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] conn(0x7fd110057070 0x7fd110059490 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.599+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== osd.5 v2:172.21.15.18:6800/24091 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7fd110059ae0 con 0x7fd110057070 2020-07-18T03:25:51.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.616+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 --> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1340afa30 con 0x7fd110057070 2020-07-18T03:25:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.617+0000 7fd12b7fe700 1 -- 172.21.15.47:0/1133269075 <== osd.5 v2:172.21.15.18:6800/24091 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd1340afa30 con 0x7fd110057070 2020-07-18T03:25:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 >> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] conn(0x7fd110057070 msgr2=0x7fd110059490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.18:6800/24091,v1:172.21.15.18:6801/24091] conn(0x7fd110057070 0x7fd110059490 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd11004fe40 msgr2=0x7fd110052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd11004fe40 0x7fd110052300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd12c000f00 tx=0x7fd12c01a000).stop 2020-07-18T03:25:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd13413a560 msgr2=0x7fd13413a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd13413a560 0x7fd13413a9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd1340b2140 tx=0x7fd1240076d0).stop 2020-07-18T03:25:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd133fff700 1 -- 172.21.15.47:0/1133269075 reap_dead start 2020-07-18T03:25:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 shutdown_connections 2020-07-18T03:25:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 --2- 172.21.15.47:0/1133269075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd13413a560 0x7fd13413a9d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 >> 172.21.15.47:0/1133269075 conn(0x7fd1340b93c0 msgr2=0x7fd13413d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 shutdown_connections 2020-07-18T03:25:51.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.618+0000 7fd13a219700 1 -- 172.21.15.47:0/1133269075 wait complete. 2020-07-18T03:25:51.630 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2020-07-18T03:25:51.631 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-07-18T03:25:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.748+0000 7ff6fda4c700 1 Processor -- start 2020-07-18T03:25:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.748+0000 7ff6fda4c700 1 -- start start 2020-07-18T03:25:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6fda4c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff6f813dbc0 0x7ff6f813e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6fda4c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6f80af320 0x7ff6f80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6fda4c700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f8144700 con 0x7ff6f80ad020 2020-07-18T03:25:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6fda4c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f8144520 con 0x7ff6f813dbc0 2020-07-18T03:25:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6fda4c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f81449a0 con 0x7ff6f80af320 2020-07-18T03:25:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6f80af320 0x7ff6f80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6f80af320 0x7ff6f80af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58620/0 (socket says 172.21.15.47:58620) 2020-07-18T03:25:51.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 -- 172.21.15.47:0/2426426852 learned_addr learned my addr 172.21.15.47:0/2426426852 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:51.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f67fc700 1 --2- 172.21.15.47:0/2426426852 >> v2:172.21.15.18:3300/0 conn(0x7ff6f813dbc0 0x7ff6f813e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 -- 172.21.15.47:0/2426426852 >> v2:172.21.15.18:3300/0 conn(0x7ff6f813dbc0 msgr2=0x7ff6f813e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 --2- 172.21.15.47:0/2426426852 >> v2:172.21.15.18:3300/0 conn(0x7ff6f813dbc0 0x7ff6f813e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 -- 172.21.15.47:0/2426426852 >> v1:172.21.15.18:6789/0 conn(0x7ff6f80ad020 legacy=0x7ff6f80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 -- 172.21.15.47:0/2426426852 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f813e510 con 0x7ff6f80af320 2020-07-18T03:25:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f67fc700 1 --2- 172.21.15.47:0/2426426852 >> v2:172.21.15.18:3300/0 conn(0x7ff6f813dbc0 0x7ff6f813e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f77fe700 1 --2- 172.21.15.47:0/2426426852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80af320 0x7ff6f80af7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff6f80ad490 tx=0x7ff6e8006e90).ready entity=mon.1 client_cookie=5ada6cc308cc9d28 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.749+0000 7ff6f5ffb700 1 -- 172.21.15.47:0/2426426852 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e8009980 con 0x7ff6f80af320 2020-07-18T03:25:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6f5ffb700 1 -- 172.21.15.47:0/2426426852 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8009ae0 con 0x7ff6f80af320 2020-07-18T03:25:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6f5ffb700 1 -- 172.21.15.47:0/2426426852 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e8009e40 con 0x7ff6f80af320 2020-07-18T03:25:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 -- 172.21.15.47:0/2426426852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80af320 msgr2=0x7ff6f80af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/2426426852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80af320 0x7ff6f80af7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff6f80ad490 tx=0x7ff6e8006e90).stop 2020-07-18T03:25:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 -- 172.21.15.47:0/2426426852 shutdown_connections 2020-07-18T03:25:51.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/2426426852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80af320 0x7ff6f80af7d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/2426426852 >> v2:172.21.15.18:3300/0 conn(0x7ff6f813dbc0 0x7ff6f813e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 -- 172.21.15.47:0/2426426852 >> 172.21.15.47:0/2426426852 conn(0x7ff6f80b93c0 msgr2=0x7ff6f80b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 -- 172.21.15.47:0/2426426852 shutdown_connections 2020-07-18T03:25:51.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.750+0000 7ff6fda4c700 1 -- 172.21.15.47:0/2426426852 wait complete. 2020-07-18T03:25:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.751+0000 7ff6fda4c700 1 Processor -- start 2020-07-18T03:25:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.751+0000 7ff6fda4c700 1 -- start start 2020-07-18T03:25:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6fda4c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6f80ad020 0x7ff6f80b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6fda4c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6f813dbc0 0x7ff6f80b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6fda4c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff6f80b08d0 0x7ff6f8064f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6fda4c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80b20d0 con 0x7ff6f80b08d0 2020-07-18T03:25:51.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6fda4c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f8152c40 con 0x7ff6f80ad020 2020-07-18T03:25:51.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6fda4c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8152ee0 con 0x7ff6f813dbc0 2020-07-18T03:25:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6f80ad020 0x7ff6f80b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6f80ad020 0x7ff6f80b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58622/0 (socket says 172.21.15.47:58622) 2020-07-18T03:25:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 -- 172.21.15.47:0/1620294187 learned_addr learned my addr 172.21.15.47:0/1620294187 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f67fc700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6f813dbc0 0x7ff6f80b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f77fe700 1 --2- 172.21.15.47:0/1620294187 >> v2:172.21.15.18:3300/0 conn(0x7ff6f80b08d0 0x7ff6f8064f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 -- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6f813dbc0 msgr2=0x7ff6f80b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6f813dbc0 0x7ff6f80b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 -- 172.21.15.47:0/1620294187 >> v2:172.21.15.18:3300/0 conn(0x7ff6f80b08d0 msgr2=0x7ff6f8064f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 --2- 172.21.15.47:0/1620294187 >> v2:172.21.15.18:3300/0 conn(0x7ff6f80b08d0 0x7ff6f8064f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.752+0000 7ff6f6ffd700 1 -- 172.21.15.47:0/1620294187 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f80661f0 con 0x7ff6f80ad020 2020-07-18T03:25:51.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6f6ffd700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80ad020 0x7ff6f80b4830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7ff6f8147590 tx=0x7ff6f0001450).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f000ce10 con 0x7ff6f80ad020 2020-07-18T03:25:51.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f000a7b0 con 0x7ff6f80ad020 2020-07-18T03:25:51.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f80b0d10 con 0x7ff6f80ad020 2020-07-18T03:25:51.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f000ab10 con 0x7ff6f80ad020 2020-07-18T03:25:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f80668f0 con 0x7ff6f80ad020 2020-07-18T03:25:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.753+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6f8057450 con 0x7ff6f80ad020 2020-07-18T03:25:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff6f0019030 con 0x7ff6f80ad020 2020-07-18T03:25:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6eeffd700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff6d4050060 0x7ff6d4052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff6f0067580 con 0x7ff6f80ad020 2020-07-18T03:25:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6f67fc700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff6d4050060 0x7ff6d4052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6eeffd700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] conn(0x7ff6d4057410 0x7ff6d4059830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 --> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6d4059e80 con 0x7ff6d4057410 2020-07-18T03:25:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f0069bf0 con 0x7ff6f80ad020 2020-07-18T03:25:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6f67fc700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff6d4050060 0x7ff6d4052520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff6e000c000 tx=0x7ff6e0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.754+0000 7ff6f77fe700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] conn(0x7ff6d4057410 0x7ff6d4059830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.755+0000 7ff6f77fe700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] conn(0x7ff6d4057410 0x7ff6d4059830 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.756+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== osd.6 v2:172.21.15.18:6816/24093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7ff6d4059e80 con 0x7ff6d4057410 2020-07-18T03:25:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.774+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 --> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6f80681b0 con 0x7ff6d4057410 2020-07-18T03:25:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6eeffd700 1 -- 172.21.15.47:0/1620294187 <== osd.6 v2:172.21.15.18:6816/24093 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff6f80681b0 con 0x7ff6d4057410 2020-07-18T03:25:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] conn(0x7ff6d4057410 msgr2=0x7ff6d4059830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.18:6816/24093,v1:172.21.15.18:6817/24093] conn(0x7ff6d4057410 0x7ff6d4059830 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff6d4050060 msgr2=0x7ff6d4052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff6d4050060 0x7ff6d4052520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff6e000c000 tx=0x7ff6e0008040).stop 2020-07-18T03:25:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80ad020 msgr2=0x7ff6f80b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80ad020 0x7ff6f80b4830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7ff6f8147590 tx=0x7ff6f0001450).stop 2020-07-18T03:25:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6f77fe700 1 -- 172.21.15.47:0/1620294187 reap_dead start 2020-07-18T03:25:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 shutdown_connections 2020-07-18T03:25:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 --2- 172.21.15.47:0/1620294187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6f80ad020 0x7ff6f80b4830 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 >> 172.21.15.47:0/1620294187 conn(0x7ff6f80b93c0 msgr2=0x7ff6f813d080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 shutdown_connections 2020-07-18T03:25:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.775+0000 7ff6fda4c700 1 -- 172.21.15.47:0/1620294187 wait complete. 2020-07-18T03:25:51.786 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:51.787 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-07-18T03:25:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.902+0000 7fb85b3ad700 1 Processor -- start 2020-07-18T03:25:51.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.902+0000 7fb85b3ad700 1 -- start start 2020-07-18T03:25:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85b3ad700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb85413e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85b3ad700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 0x7fb8540af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85b3ad700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb854144700 con 0x7fb8540ad020 2020-07-18T03:25:51.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85b3ad700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb854144520 con 0x7fb85413dbc0 2020-07-18T03:25:51.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85b3ad700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8541449a0 con 0x7fb8540af320 2020-07-18T03:25:51.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85994a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 0x7fb8540af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb858948700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb85413e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb859149700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fb8540ad020 0x7fb8540b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58486/0 (socket says 172.21.15.47:58486) 2020-07-18T03:25:51.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb85994a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 0x7fb8540af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51230/0 (socket says 172.21.15.47:51230) 2020-07-18T03:25:51.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb859149700 1 -- 172.21.15.47:0/4211513915 learned_addr learned my addr 172.21.15.47:0/4211513915 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.903+0000 7fb858948700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb85413e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:44312/0 (socket says 172.21.15.47:44312) 2020-07-18T03:25:51.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85994a700 1 -- 172.21.15.47:0/4211513915 >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 msgr2=0x7fb85413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:51.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85994a700 1 --2- 172.21.15.47:0/4211513915 >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb85413e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85994a700 1 -- 172.21.15.47:0/4211513915 >> v1:172.21.15.18:6789/0 conn(0x7fb8540ad020 legacy=0x7fb8540b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85994a700 1 -- 172.21.15.47:0/4211513915 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85413e510 con 0x7fb8540af320 2020-07-18T03:25:51.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85994a700 1 --2- 172.21.15.47:0/4211513915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 0x7fb8540af7d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb84c0032b0 tx=0x7fb84c00b460).ready entity=mon.2 client_cookie=8ee561ecaf6cb1fb server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb847fff700 1 -- 172.21.15.47:0/4211513915 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb84c010660 con 0x7fb8540af320 2020-07-18T03:25:51.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb847fff700 1 -- 172.21.15.47:0/4211513915 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c0107c0 con 0x7fb8540af320 2020-07-18T03:25:51.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb847fff700 1 -- 172.21.15.47:0/4211513915 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb84c010b20 con 0x7fb8540af320 2020-07-18T03:25:51.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 -- 172.21.15.47:0/4211513915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 msgr2=0x7fb8540af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/4211513915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 0x7fb8540af7d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb84c0032b0 tx=0x7fb84c00b460).stop 2020-07-18T03:25:51.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 -- 172.21.15.47:0/4211513915 shutdown_connections 2020-07-18T03:25:51.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/4211513915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8540af320 0x7fb8540af7d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/4211513915 >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb85413e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 -- 172.21.15.47:0/4211513915 >> 172.21.15.47:0/4211513915 conn(0x7fb8540b93c0 msgr2=0x7fb8540b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 -- 172.21.15.47:0/4211513915 shutdown_connections 2020-07-18T03:25:51.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.904+0000 7fb85b3ad700 1 -- 172.21.15.47:0/4211513915 wait complete. 2020-07-18T03:25:51.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.906+0000 7fb85b3ad700 1 Processor -- start 2020-07-18T03:25:51.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.906+0000 7fb85b3ad700 1 -- start start 2020-07-18T03:25:51.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb8540b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8540b4b40 0x7fb854134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8540664f0 con 0x7fb8540ad020 2020-07-18T03:25:51.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb854065c40 con 0x7fb85413dbc0 2020-07-18T03:25:51.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb854065fb0 con 0x7fb8540b4b40 2020-07-18T03:25:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85994a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8540b4b40 0x7fb854134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb859149700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8540ad020 0x7fb85405f760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39726/0 (socket says 172.21.15.47:39726) 2020-07-18T03:25:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb859149700 1 -- 172.21.15.47:0/871072659 learned_addr learned my addr 172.21.15.47:0/871072659 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb858948700 1 --2- 172.21.15.47:0/871072659 >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb8540b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85994a700 1 -- 172.21.15.47:0/871072659 >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 msgr2=0x7fb8540b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85994a700 1 --2- 172.21.15.47:0/871072659 >> v2:172.21.15.18:3300/0 conn(0x7fb85413dbc0 0x7fb8540b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85994a700 1 -- 172.21.15.47:0/871072659 >> v1:172.21.15.47:6789/0 conn(0x7fb8540ad020 legacy=0x7fb85405f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49489903 0 0) 0x7fb8540664f0 con 0x7fb8540ad020 2020-07-18T03:25:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85994a700 1 -- 172.21.15.47:0/871072659 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854066250 con 0x7fb8540b4b40 2020-07-18T03:25:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85994a700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8540b4b40 0x7fb854134d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb84c0032b0 tx=0x7fb84c00b460).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb84c018ce0 con 0x7fb8540b4b40 2020-07-18T03:25:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8540b4540 con 0x7fb8540b4b40 2020-07-18T03:25:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c018e40 con 0x7fb8540b4b40 2020-07-18T03:25:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb84c0235f0 con 0x7fb8540b4b40 2020-07-18T03:25:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.907+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb854135250 con 0x7fb8540b4b40 2020-07-18T03:25:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.908+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8540ae190 con 0x7fb8540b4b40 2020-07-18T03:25:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb84c021070 con 0x7fb8540b4b40 2020-07-18T03:25:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb8457fa700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb82c04fea0 0x7fb82c052360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb84c06b5a0 con 0x7fb8540b4b40 2020-07-18T03:25:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb859149700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb82c04fea0 0x7fb82c052360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb8457fa700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] conn(0x7fb82c0570d0 0x7fb82c0594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 --> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb82c059b40 con 0x7fb82c0570d0 2020-07-18T03:25:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb84c06dc10 con 0x7fb8540b4b40 2020-07-18T03:25:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb859149700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb82c04fea0 0x7fb82c052360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb848000ef0 tx=0x7fb848007880).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.909+0000 7fb858948700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] conn(0x7fb82c0570d0 0x7fb82c0594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:51.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.910+0000 7fb858948700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] conn(0x7fb82c0570d0 0x7fb82c0594f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:51.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.910+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== osd.7 v2:172.21.15.18:6818/24094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23416 (crc 0 0 0) 0x7fb82c059b40 con 0x7fb82c0570d0 2020-07-18T03:25:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.929+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 --> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb854135670 con 0x7fb82c0570d0 2020-07-18T03:25:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.929+0000 7fb8457fa700 1 -- 172.21.15.47:0/871072659 <== osd.7 v2:172.21.15.18:6818/24094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb854135670 con 0x7fb82c0570d0 2020-07-18T03:25:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 >> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] conn(0x7fb82c0570d0 msgr2=0x7fb82c0594f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.18:6818/24094,v1:172.21.15.18:6820/24094] conn(0x7fb82c0570d0 0x7fb82c0594f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb82c04fea0 msgr2=0x7fb82c052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb82c04fea0 0x7fb82c052360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb848000ef0 tx=0x7fb848007880).stop 2020-07-18T03:25:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8540b4b40 msgr2=0x7fb854134d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8540b4b40 0x7fb854134d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb84c0032b0 tx=0x7fb84c00b460).stop 2020-07-18T03:25:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85994a700 1 -- 172.21.15.47:0/871072659 reap_dead start 2020-07-18T03:25:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 shutdown_connections 2020-07-18T03:25:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 --2- 172.21.15.47:0/871072659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8540b4b40 0x7fb854134d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 >> 172.21.15.47:0/871072659 conn(0x7fb8540b93c0 msgr2=0x7fb8540b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 shutdown_connections 2020-07-18T03:25:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:51.930+0000 7fb85b3ad700 1 -- 172.21.15.47:0/871072659 wait complete. 2020-07-18T03:25:51.941 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:51.942 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-07-18T03:25:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.057+0000 7f26f2508700 1 Processor -- start 2020-07-18T03:25:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.057+0000 7f26f2508700 1 -- start start 2020-07-18T03:25:52.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26f2508700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 0x7f26ec0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:52.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26f2508700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f26ec144920 0x7f26ec0616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:52.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26f2508700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec1476f0 con 0x7f26ec0be0b0 2020-07-18T03:25:52.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26f2508700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec147990 con 0x7f26ec144920 2020-07-18T03:25:52.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26f2508700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec147100 con 0x7f26ec0b44c0 2020-07-18T03:25:52.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26ebfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 0x7f26ec0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:52.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26ebfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 0x7f26ec0b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51250/0 (socket says 172.21.15.47:51250) 2020-07-18T03:25:52.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26ebfff700 1 -- 172.21.15.47:0/3935190262 learned_addr learned my addr 172.21.15.47:0/3935190262 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:52.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26f0aa5700 1 --2- 172.21.15.47:0/3935190262 >> v2:172.21.15.18:3300/0 conn(0x7f26ec144920 0x7f26ec0616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.058+0000 7f26eb7fe700 1 --1- 172.21.15.47:0/3935190262 >> v1:172.21.15.18:6789/0 conn(0x7f26ec0be0b0 0x7f26ec144210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58502/0 (socket says 172.21.15.47:58502) 2020-07-18T03:25:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26ebfff700 1 -- 172.21.15.47:0/3935190262 >> v2:172.21.15.18:3300/0 conn(0x7f26ec144920 msgr2=0x7f26ec0616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26ebfff700 1 --2- 172.21.15.47:0/3935190262 >> v2:172.21.15.18:3300/0 conn(0x7f26ec144920 0x7f26ec0616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26ebfff700 1 -- 172.21.15.47:0/3935190262 >> v1:172.21.15.18:6789/0 conn(0x7f26ec0be0b0 legacy=0x7f26ec144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26ebfff700 1 -- 172.21.15.47:0/3935190262 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0be530 con 0x7f26ec0b44c0 2020-07-18T03:25:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26ebfff700 1 --2- 172.21.15.47:0/3935190262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 0x7f26ec0b7110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f26dc002f30 tx=0x7f26dc006e90).ready entity=mon.2 client_cookie=e2f8de6f75c715af server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26eaffd700 1 -- 172.21.15.47:0/3935190262 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26dc009850 con 0x7f26ec0b44c0 2020-07-18T03:25:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26eaffd700 1 -- 172.21.15.47:0/3935190262 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc0099b0 con 0x7f26ec0b44c0 2020-07-18T03:25:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26eaffd700 1 -- 172.21.15.47:0/3935190262 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26dc009d10 con 0x7f26ec0b44c0 2020-07-18T03:25:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 -- 172.21.15.47:0/3935190262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 msgr2=0x7f26ec0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 --2- 172.21.15.47:0/3935190262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 0x7f26ec0b7110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f26dc002f30 tx=0x7f26dc006e90).stop 2020-07-18T03:25:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 -- 172.21.15.47:0/3935190262 shutdown_connections 2020-07-18T03:25:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 --2- 172.21.15.47:0/3935190262 >> v2:172.21.15.18:3300/0 conn(0x7f26ec144920 0x7f26ec0616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 --2- 172.21.15.47:0/3935190262 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec0b44c0 0x7f26ec0b7110 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 -- 172.21.15.47:0/3935190262 >> 172.21.15.47:0/3935190262 conn(0x7f26ec060070 msgr2=0x7f26ec0bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:52.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 -- 172.21.15.47:0/3935190262 shutdown_connections 2020-07-18T03:25:52.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.059+0000 7f26f2508700 1 -- 172.21.15.47:0/3935190262 wait complete. 2020-07-18T03:25:52.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 Processor -- start 2020-07-18T03:25:52.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 -- start start 2020-07-18T03:25:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f26ec0be0b0 0x7f26ec0add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec141310 0x7f26ec141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec0ae350 con 0x7f26ec0ad600 2020-07-18T03:25:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec141c70 con 0x7f26ec0be0b0 2020-07-18T03:25:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.061+0000 7f26f2508700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0afbe0 con 0x7f26ec141310 2020-07-18T03:25:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26eb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f26ec0ad600 0x7f26ec0bc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39744/0 (socket says 172.21.15.47:39744) 2020-07-18T03:25:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26eb7fe700 1 -- 172.21.15.47:0/609102147 learned_addr learned my addr 172.21.15.47:0/609102147 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f0aa5700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec141310 0x7f26ec141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26ebfff700 1 --2- 172.21.15.47:0/609102147 >> v2:172.21.15.18:3300/0 conn(0x7f26ec0be0b0 0x7f26ec0add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017929254 0 0) 0x7f26ec0ae350 con 0x7f26ec0ad600 2020-07-18T03:25:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c4003240 con 0x7f26ec0ad600 2020-07-18T03:25:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f0aa5700 1 -- 172.21.15.47:0/609102147 >> v2:172.21.15.18:3300/0 conn(0x7f26ec0be0b0 msgr2=0x7f26ec0add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f0aa5700 1 --2- 172.21.15.47:0/609102147 >> v2:172.21.15.18:3300/0 conn(0x7f26ec0be0b0 0x7f26ec0add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f0aa5700 1 -- 172.21.15.47:0/609102147 >> v1:172.21.15.47:6789/0 conn(0x7f26ec0ad600 legacy=0x7f26ec0bc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f0aa5700 1 -- 172.21.15.47:0/609102147 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0afe80 con 0x7f26ec141310 2020-07-18T03:25:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f0aa5700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec141310 0x7f26ec141780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f26e400c000 tx=0x7f26e40091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e400ca10 con 0x7f26ec141310 2020-07-18T03:25:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400cb70 con 0x7f26ec141310 2020-07-18T03:25:52.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec0b0320 con 0x7f26ec141310 2020-07-18T03:25:52.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e40123f0 con 0x7f26ec141310 2020-07-18T03:25:52.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.062+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec0b07f0 con 0x7f26ec141310 2020-07-18T03:25:52.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.063+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec0bac50 con 0x7f26ec141310 2020-07-18T03:25:52.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.063+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f26e4022070 con 0x7f26ec141310 2020-07-18T03:25:52.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.064+0000 7f26cbfff700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f26c4050420 0x7f26c40528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:52.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.064+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f26e4068270 con 0x7f26ec141310 2020-07-18T03:25:52.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.064+0000 7f26ebfff700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f26c4050420 0x7f26c40528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:52.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.064+0000 7f26ebfff700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f26c4050420 0x7f26c40528e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f26dc01c000 tx=0x7f26dc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:52.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.068+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f26e4040920 con 0x7f26ec141310 2020-07-18T03:25:52.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.224+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f26ec0b0010 con 0x7f26ec141310 2020-07-18T03:25:52.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.225+0000 7f26cbfff700 1 -- 172.21.15.47:0/609102147 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26e4016020 con 0x7f26ec141310 2020-07-18T03:25:52.227 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2020-07-18T03:25:52.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f26c4050420 msgr2=0x7f26c40528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:52.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f26c4050420 0x7f26c40528e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f26dc01c000 tx=0x7f26dc015000).stop 2020-07-18T03:25:52.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec141310 msgr2=0x7f26ec141780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:52.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec141310 0x7f26ec141780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f26e400c000 tx=0x7f26e40091d0).stop 2020-07-18T03:25:52.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 shutdown_connections 2020-07-18T03:25:52.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f26ec141310 0x7f26ec141780 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 --2- 172.21.15.47:0/609102147 >> v2:172.21.15.18:3300/0 conn(0x7f26ec0be0b0 0x7f26ec0add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 --2- 172.21.15.47:0/609102147 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f26c4050420 0x7f26c40528e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:52.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 >> 172.21.15.47:0/609102147 conn(0x7f26ec060070 msgr2=0x7f26ec143580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:52.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 shutdown_connections 2020-07-18T03:25:52.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:52.226+0000 7f26f2508700 1 -- 172.21.15.47:0/609102147 wait complete. 2020-07-18T03:25:52.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2020-07-18T03:25:53.240 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-07-18T03:25:53.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.371+0000 7f9801abf700 1 Processor -- start 2020-07-18T03:25:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.371+0000 7f9801abf700 1 -- start start 2020-07-18T03:25:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f9801abf700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc0ad020 0x7f97fc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f9801abf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97fc0af320 0x7f97fc0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f9801abf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc144700 con 0x7f97fc13dbc0 2020-07-18T03:25:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f9801abf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc1449a0 con 0x7f97fc0af320 2020-07-18T03:25:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f9801abf700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0b59e0 con 0x7f97fc0ad020 2020-07-18T03:25:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f97fbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97fc0af320 0x7f97fc0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f97faffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f97fc13dbc0 0x7f97fc13e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39752/0 (socket says 172.21.15.47:39752) 2020-07-18T03:25:53.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f97faffd700 1 -- 172.21.15.47:0/500182587 learned_addr learned my addr 172.21.15.47:0/500182587 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:53.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f97fb7fe700 1 --2- 172.21.15.47:0/500182587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc0ad020 0x7f97fc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f97fa7fc700 1 -- 172.21.15.47:0/500182587 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296137267 0 0) 0x7f97fc144700 con 0x7f97fc13dbc0 2020-07-18T03:25:53.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.372+0000 7f97fa7fc700 1 -- 172.21.15.47:0/500182587 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e8003240 con 0x7f97fc13dbc0 2020-07-18T03:25:53.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fbfff700 1 -- 172.21.15.47:0/500182587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc0ad020 msgr2=0x7f97fc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fbfff700 1 --2- 172.21.15.47:0/500182587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc0ad020 0x7f97fc0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fbfff700 1 -- 172.21.15.47:0/500182587 >> v1:172.21.15.47:6789/0 conn(0x7f97fc13dbc0 legacy=0x7f97fc13e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fbfff700 1 -- 172.21.15.47:0/500182587 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0b5b20 con 0x7f97fc0af320 2020-07-18T03:25:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fbfff700 1 --2- 172.21.15.47:0/500182587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc0af320 0x7f97fc0af7d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f97f40032b0 tx=0x7f97f40091d0).ready entity=mon.1 client_cookie=bfeed5bdb26cd4fd server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fa7fc700 1 -- 172.21.15.47:0/500182587 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97f400b940 con 0x7f97fc0af320 2020-07-18T03:25:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fa7fc700 1 -- 172.21.15.47:0/500182587 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f400baa0 con 0x7f97fc0af320 2020-07-18T03:25:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f97fa7fc700 1 -- 172.21.15.47:0/500182587 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97f400bc10 con 0x7f97fc0af320 2020-07-18T03:25:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 -- 172.21.15.47:0/500182587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc0af320 msgr2=0x7f97fc0af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 --2- 172.21.15.47:0/500182587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc0af320 0x7f97fc0af7d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f97f40032b0 tx=0x7f97f40091d0).stop 2020-07-18T03:25:53.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 -- 172.21.15.47:0/500182587 shutdown_connections 2020-07-18T03:25:53.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 --2- 172.21.15.47:0/500182587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc0af320 0x7f97fc0af7d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 --2- 172.21.15.47:0/500182587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc0ad020 0x7f97fc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 -- 172.21.15.47:0/500182587 >> 172.21.15.47:0/500182587 conn(0x7f97fc0b93c0 msgr2=0x7f97fc0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:53.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 -- 172.21.15.47:0/500182587 shutdown_connections 2020-07-18T03:25:53.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.373+0000 7f9801abf700 1 -- 172.21.15.47:0/500182587 wait complete. 2020-07-18T03:25:53.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 Processor -- start 2020-07-18T03:25:53.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 -- start start 2020-07-18T03:25:53.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97fc13dbc0 0x7f97fc13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc065330 0x7f97fc13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc064bd0 con 0x7f97fc0ad020 2020-07-18T03:25:53.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc13d530 con 0x7f97fc13dbc0 2020-07-18T03:25:53.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.375+0000 7f9801abf700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc140de0 con 0x7f97fc065330 2020-07-18T03:25:53.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97fb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f97fc0ad020 0x7f97fc1483b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39756/0 (socket says 172.21.15.47:39756) 2020-07-18T03:25:53.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97faffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97fc13dbc0 0x7f97fc13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97fb7fe700 1 -- 172.21.15.47:0/12954358 learned_addr learned my addr 172.21.15.47:0/12954358 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:53.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97fbfff700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc065330 0x7f97fc13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919922005 0 0) 0x7f97fc064bd0 con 0x7f97fc0ad020 2020-07-18T03:25:53.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97d4003240 con 0x7f97fc0ad020 2020-07-18T03:25:53.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97faffd700 1 -- 172.21.15.47:0/12954358 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc065330 msgr2=0x7f97fc13d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97faffd700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc065330 0x7f97fc13d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97faffd700 1 -- 172.21.15.47:0/12954358 >> v1:172.21.15.47:6789/0 conn(0x7f97fc0ad020 legacy=0x7f97fc1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97faffd700 1 -- 172.21.15.47:0/12954358 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc148dd0 con 0x7f97fc13dbc0 2020-07-18T03:25:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97faffd700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc13dbc0 0x7f97fc13cb50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f97f00010d0 tx=0x7f97f0007980).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97f00072e0 con 0x7f97fc13dbc0 2020-07-18T03:25:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0007440 con 0x7f97fc13dbc0 2020-07-18T03:25:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc149270 con 0x7f97fc13dbc0 2020-07-18T03:25:53.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97f00057e0 con 0x7f97fc13dbc0 2020-07-18T03:25:53.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.376+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc13f730 con 0x7f97fc13dbc0 2020-07-18T03:25:53.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.377+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc0b5100 con 0x7f97fc13dbc0 2020-07-18T03:25:53.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.378+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f97f001a060 con 0x7f97fc13dbc0 2020-07-18T03:25:53.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.378+0000 7f97db7fe700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97d4050380 0x7f97d4052840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.378+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f97f00672d0 con 0x7f97fc13dbc0 2020-07-18T03:25:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.378+0000 7f97fb7fe700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97d4050380 0x7f97d4052840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.378+0000 7f97fb7fe700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97d4050380 0x7f97d4052840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f97ec00a000 tx=0x7f97ec006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.381+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f97f003fa60 con 0x7f97fc13dbc0 2020-07-18T03:25:53.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.539+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97fc148f60 con 0x7f97fc13dbc0 2020-07-18T03:25:53.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.539+0000 7f97db7fe700 1 -- 172.21.15.47:0/12954358 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97f0018020 con 0x7f97fc13dbc0 2020-07-18T03:25:53.541 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.540+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97d4050380 msgr2=0x7f97d4052840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.540+0000 7f9801abf700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97d4050380 0x7f97d4052840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f97ec00a000 tx=0x7f97ec006dd0).stop 2020-07-18T03:25:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.540+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc13dbc0 msgr2=0x7f97fc13cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.540+0000 7f9801abf700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc13dbc0 0x7f97fc13cb50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f97f00010d0 tx=0x7f97f0007980).stop 2020-07-18T03:25:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.540+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 shutdown_connections 2020-07-18T03:25:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.541+0000 7f9801abf700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97fc065330 0x7f97fc13d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.541+0000 7f9801abf700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97d4050380 0x7f97d4052840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.541+0000 7f9801abf700 1 --2- 172.21.15.47:0/12954358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97fc13dbc0 0x7f97fc13cb50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.541+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 >> 172.21.15.47:0/12954358 conn(0x7f97fc0b93c0 msgr2=0x7f97fc05d200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.541+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 shutdown_connections 2020-07-18T03:25:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.541+0000 7f9801abf700 1 -- 172.21.15.47:0/12954358 wait complete. 2020-07-18T03:25:53.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2020-07-18T03:25:53.554 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-07-18T03:25:53.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.669+0000 7f1197c5b700 1 Processor -- start 2020-07-18T03:25:53.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.669+0000 7f1197c5b700 1 -- start start 2020-07-18T03:25:53.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f1197c5b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f119013dbc0 0x7f119013e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f1197c5b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 0x7f11900af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f1197c5b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1190144700 con 0x7f11900ad020 2020-07-18T03:25:53.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f1197c5b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1190144520 con 0x7f119013dbc0 2020-07-18T03:25:53.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f1197c5b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11901449a0 con 0x7f11900af320 2020-07-18T03:25:53.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f11961f8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 0x7f11900af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f11961f8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 0x7f11900af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51274/0 (socket says 172.21.15.47:51274) 2020-07-18T03:25:53.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f11961f8700 1 -- 172.21.15.47:0/2473754555 learned_addr learned my addr 172.21.15.47:0/2473754555 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:53.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.670+0000 7f11951f6700 1 --2- 172.21.15.47:0/2473754555 >> v2:172.21.15.18:3300/0 conn(0x7f119013dbc0 0x7f119013e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11961f8700 1 -- 172.21.15.47:0/2473754555 >> v2:172.21.15.18:3300/0 conn(0x7f119013dbc0 msgr2=0x7f119013e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11961f8700 1 --2- 172.21.15.47:0/2473754555 >> v2:172.21.15.18:3300/0 conn(0x7f119013dbc0 0x7f119013e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11961f8700 1 -- 172.21.15.47:0/2473754555 >> v1:172.21.15.18:6789/0 conn(0x7f11900ad020 legacy=0x7f11900b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11961f8700 1 -- 172.21.15.47:0/2473754555 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119013e510 con 0x7f11900af320 2020-07-18T03:25:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11961f8700 1 --2- 172.21.15.47:0/2473754555 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 0x7f11900af7d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f11880032b0 tx=0x7f1188009270).ready entity=mon.2 client_cookie=3e1facdc2a9d9043 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11949f5700 1 -- 172.21.15.47:0/2473754555 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f118800bdf0 con 0x7f11900af320 2020-07-18T03:25:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11949f5700 1 -- 172.21.15.47:0/2473754555 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1188009e60 con 0x7f11900af320 2020-07-18T03:25:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f11949f5700 1 -- 172.21.15.47:0/2473754555 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1188010660 con 0x7f11900af320 2020-07-18T03:25:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 -- 172.21.15.47:0/2473754555 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 msgr2=0x7f11900af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2473754555 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 0x7f11900af7d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f11880032b0 tx=0x7f1188009270).stop 2020-07-18T03:25:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 -- 172.21.15.47:0/2473754555 shutdown_connections 2020-07-18T03:25:53.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2473754555 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f11900af320 0x7f11900af7d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2473754555 >> v2:172.21.15.18:3300/0 conn(0x7f119013dbc0 0x7f119013e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 -- 172.21.15.47:0/2473754555 >> 172.21.15.47:0/2473754555 conn(0x7f11900b93c0 msgr2=0x7f11900b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:53.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 -- 172.21.15.47:0/2473754555 shutdown_connections 2020-07-18T03:25:53.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.671+0000 7f1197c5b700 1 -- 172.21.15.47:0/2473754555 wait complete. 2020-07-18T03:25:53.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 Processor -- start 2020-07-18T03:25:53.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 -- start start 2020-07-18T03:25:53.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f119013dbc0 0x7f11900b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f11900b28c0 0x7f1190139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1190064740 con 0x7f11900ad020 2020-07-18T03:25:53.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1190064530 con 0x7f11900b28c0 2020-07-18T03:25:53.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f1197c5b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11900649e0 con 0x7f119013dbc0 2020-07-18T03:25:53.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f11959f7700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f11900ad020 0x7f119013b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39768/0 (socket says 172.21.15.47:39768) 2020-07-18T03:25:53.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f11951f6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f119013dbc0 0x7f11900b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.673+0000 7f11959f7700 1 -- 172.21.15.47:0/2781829010 learned_addr learned my addr 172.21.15.47:0/2781829010 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:53.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f11961f8700 1 --2- 172.21.15.47:0/2781829010 >> v2:172.21.15.18:3300/0 conn(0x7f11900b28c0 0x7f1190139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755961632 0 0) 0x7f1190064740 con 0x7f11900ad020 2020-07-18T03:25:53.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003240 con 0x7f11900ad020 2020-07-18T03:25:53.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f11951f6700 1 -- 172.21.15.47:0/2781829010 >> v2:172.21.15.18:3300/0 conn(0x7f11900b28c0 msgr2=0x7f1190139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f11951f6700 1 --2- 172.21.15.47:0/2781829010 >> v2:172.21.15.18:3300/0 conn(0x7f11900b28c0 0x7f1190139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f11951f6700 1 -- 172.21.15.47:0/2781829010 >> v1:172.21.15.47:6789/0 conn(0x7f11900ad020 legacy=0x7f119013b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f11951f6700 1 -- 172.21.15.47:0/2781829010 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900b2500 con 0x7f119013dbc0 2020-07-18T03:25:53.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f11951f6700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f119013dbc0 0x7f11900b1550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f118c00c000 tx=0x7f118c006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f118c00ca10 con 0x7f119013dbc0 2020-07-18T03:25:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1190064070 con 0x7f119013dbc0 2020-07-18T03:25:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c00cb70 con 0x7f119013dbc0 2020-07-18T03:25:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f118c00ee80 con 0x7f119013dbc0 2020-07-18T03:25:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.674+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11900b2e80 con 0x7f119013dbc0 2020-07-18T03:25:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.675+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f118c0123f0 con 0x7f119013dbc0 2020-07-18T03:25:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.675+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1190057450 con 0x7f119013dbc0 2020-07-18T03:25:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.675+0000 7f117dffb700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1168050360 0x7f1168052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.676+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f118c0680b0 con 0x7f119013dbc0 2020-07-18T03:25:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.676+0000 7f11959f7700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1168050360 0x7f1168052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.676+0000 7f11959f7700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1168050360 0x7f1168052820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1180001160 tx=0x7f11800016c0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.679+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f118c040760 con 0x7f119013dbc0 2020-07-18T03:25:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.835+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11900681b0 con 0x7f119013dbc0 2020-07-18T03:25:53.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.835+0000 7f117dffb700 1 -- 172.21.15.47:0/2781829010 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f118c043d80 con 0x7f119013dbc0 2020-07-18T03:25:53.836 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.836+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1168050360 msgr2=0x7f1168052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.836+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1168050360 0x7f1168052820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1180001160 tx=0x7f11800016c0).stop 2020-07-18T03:25:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.836+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f119013dbc0 msgr2=0x7f11900b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.836+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f119013dbc0 0x7f11900b1550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f118c00c000 tx=0x7f118c006dd0).stop 2020-07-18T03:25:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 shutdown_connections 2020-07-18T03:25:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2781829010 >> v2:172.21.15.18:3300/0 conn(0x7f11900b28c0 0x7f1190139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1168050360 0x7f1168052820 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 --2- 172.21.15.47:0/2781829010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f119013dbc0 0x7f11900b1550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 >> 172.21.15.47:0/2781829010 conn(0x7f11900b93c0 msgr2=0x7f1190061910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 shutdown_connections 2020-07-18T03:25:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.837+0000 7f1197c5b700 1 -- 172.21.15.47:0/2781829010 wait complete. 2020-07-18T03:25:53.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2020-07-18T03:25:53.854 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-07-18T03:25:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.975+0000 7fae6ca74700 1 Processor -- start 2020-07-18T03:25:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.975+0000 7fae6ca74700 1 -- start start 2020-07-18T03:25:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fae680ad020 0x7fae680b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae6813dbc0 0x7fae6813e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae68144700 con 0x7fae68147640 2020-07-18T03:25:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae68144510 con 0x7fae6813dbc0 2020-07-18T03:25:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae681449a0 con 0x7fae680ad020 2020-07-18T03:25:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae6813dbc0 0x7fae6813e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae6813dbc0 0x7fae6813e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58690/0 (socket says 172.21.15.47:58690) 2020-07-18T03:25:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 -- 172.21.15.47:0/2063530227 learned_addr learned my addr 172.21.15.47:0/2063530227 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6659c700 1 --2- 172.21.15.47:0/2063530227 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fae680ad020 0x7fae680b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 -- 172.21.15.47:0/2063530227 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fae680ad020 msgr2=0x7fae680b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 --2- 172.21.15.47:0/2063530227 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fae680ad020 0x7fae680b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 -- 172.21.15.47:0/2063530227 >> v1:172.21.15.18:6789/0 conn(0x7fae68147640 legacy=0x7fae68147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 -- 172.21.15.47:0/2063530227 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6813e510 con 0x7fae6813dbc0 2020-07-18T03:25:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae5ffff700 1 --2- 172.21.15.47:0/2063530227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae6813dbc0 0x7fae6813e020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fae50002f30 tx=0x7fae50006e90).ready entity=mon.1 client_cookie=8d98845579d05e00 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae65d9b700 1 -- 172.21.15.47:0/2063530227 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae50009850 con 0x7fae6813dbc0 2020-07-18T03:25:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae65d9b700 1 -- 172.21.15.47:0/2063530227 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae500099b0 con 0x7fae6813dbc0 2020-07-18T03:25:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae65d9b700 1 -- 172.21.15.47:0/2063530227 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae50009d10 con 0x7fae6813dbc0 2020-07-18T03:25:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 -- 172.21.15.47:0/2063530227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae6813dbc0 msgr2=0x7fae6813e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.976+0000 7fae6ca74700 1 --2- 172.21.15.47:0/2063530227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae6813dbc0 0x7fae6813e020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fae50002f30 tx=0x7fae50006e90).stop 2020-07-18T03:25:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.977+0000 7fae6ca74700 1 -- 172.21.15.47:0/2063530227 shutdown_connections 2020-07-18T03:25:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.977+0000 7fae6ca74700 1 --2- 172.21.15.47:0/2063530227 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fae680ad020 0x7fae680b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.977+0000 7fae6ca74700 1 --2- 172.21.15.47:0/2063530227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae6813dbc0 0x7fae6813e020 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.977+0000 7fae6ca74700 1 -- 172.21.15.47:0/2063530227 >> 172.21.15.47:0/2063530227 conn(0x7fae680b93c0 msgr2=0x7fae680b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.977+0000 7fae6ca74700 1 -- 172.21.15.47:0/2063530227 shutdown_connections 2020-07-18T03:25:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.977+0000 7fae6ca74700 1 -- 172.21.15.47:0/2063530227 wait complete. 2020-07-18T03:25:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.978+0000 7fae6ca74700 1 Processor -- start 2020-07-18T03:25:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.978+0000 7fae6ca74700 1 -- start start 2020-07-18T03:25:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6ca74700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae680ad020 0x7fae68066890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6ca74700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fae68147640 0x7fae68143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6ca74700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae68143520 con 0x7fae68142db0 2020-07-18T03:25:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6ca74700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae68143280 con 0x7fae68147640 2020-07-18T03:25:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6ca74700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae680643a0 con 0x7fae680ad020 2020-07-18T03:25:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae680ad020 0x7fae68066890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae680ad020 0x7fae68066890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58694/0 (socket says 172.21.15.47:58694) 2020-07-18T03:25:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae66d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fae68142db0 0x7fae68063c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39784/0 (socket says 172.21.15.47:39784) 2020-07-18T03:25:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 -- 172.21.15.47:0/1594050915 learned_addr learned my addr 172.21.15.47:0/1594050915 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae5ffff700 1 --2- 172.21.15.47:0/1594050915 >> v2:172.21.15.18:3300/0 conn(0x7fae68147640 0x7fae68143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 -- 172.21.15.47:0/1594050915 >> v2:172.21.15.18:3300/0 conn(0x7fae68147640 msgr2=0x7fae68143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 --2- 172.21.15.47:0/1594050915 >> v2:172.21.15.18:3300/0 conn(0x7fae68147640 0x7fae68143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 -- 172.21.15.47:0/1594050915 >> v1:172.21.15.47:6789/0 conn(0x7fae68142db0 legacy=0x7fae68063c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906263111 0 0) 0x7fae68143520 con 0x7fae68142db0 2020-07-18T03:25:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 -- 172.21.15.47:0/1594050915 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae681415d0 con 0x7fae680ad020 2020-07-18T03:25:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae6659c700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae680ad020 0x7fae68066890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fae5800c000 tx=0x7fae58006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5800ca10 con 0x7fae680ad020 2020-07-18T03:25:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.979+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5800cb70 con 0x7fae680ad020 2020-07-18T03:25:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.980+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae68144100 con 0x7fae680ad020 2020-07-18T03:25:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.980+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae580123f0 con 0x7fae680ad020 2020-07-18T03:25:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.980+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae680657d0 con 0x7fae680ad020 2020-07-18T03:25:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.980+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae68057450 con 0x7fae680ad020 2020-07-18T03:25:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.981+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fae580125a0 con 0x7fae680ad020 2020-07-18T03:25:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.981+0000 7fae5e7fc700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fae4404fef0 0x7fae440523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.981+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fae58068240 con 0x7fae680ad020 2020-07-18T03:25:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.981+0000 7fae5ffff700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fae4404fef0 0x7fae440523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.981+0000 7fae5ffff700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fae4404fef0 0x7fae440523b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fae5001c000 tx=0x7fae50015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:53.984+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fae58040970 con 0x7fae680ad020 2020-07-18T03:25:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.144+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fae68066f40 con 0x7fae680ad020 2020-07-18T03:25:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.145+0000 7fae5e7fc700 1 -- 172.21.15.47:0/1594050915 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae58043f90 con 0x7fae680ad020 2020-07-18T03:25:54.146 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fae4404fef0 msgr2=0x7fae440523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fae4404fef0 0x7fae440523b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fae5001c000 tx=0x7fae50015000).stop 2020-07-18T03:25:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae680ad020 msgr2=0x7fae68066890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae680ad020 0x7fae68066890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fae5800c000 tx=0x7fae58006dd0).stop 2020-07-18T03:25:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 shutdown_connections 2020-07-18T03:25:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fae4404fef0 0x7fae440523b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 --2- 172.21.15.47:0/1594050915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae680ad020 0x7fae68066890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 --2- 172.21.15.47:0/1594050915 >> v2:172.21.15.18:3300/0 conn(0x7fae68147640 0x7fae68143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 >> 172.21.15.47:0/1594050915 conn(0x7fae680b93c0 msgr2=0x7fae680b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 shutdown_connections 2020-07-18T03:25:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.146+0000 7fae6ca74700 1 -- 172.21.15.47:0/1594050915 wait complete. 2020-07-18T03:25:54.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2020-07-18T03:25:54.158 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-07-18T03:25:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.272+0000 7fcb49370700 1 Processor -- start 2020-07-18T03:25:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.272+0000 7fcb49370700 1 -- start start 2020-07-18T03:25:54.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcb440aea60 0x7fcb44142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb44143330 0x7fcb441471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb440b5830 con 0x7fcb4405ffb0 2020-07-18T03:25:54.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb440b5a60 con 0x7fcb44143330 2020-07-18T03:25:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44060390 con 0x7fcb440aea60 2020-07-18T03:25:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb437fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb44143330 0x7fcb441471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb42ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcb4405ffb0 0x7fcb440625d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39790/0 (socket says 172.21.15.47:39790) 2020-07-18T03:25:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb42ffd700 1 -- 172.21.15.47:0/498129436 learned_addr learned my addr 172.21.15.47:0/498129436 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb427fc700 1 --2- 172.21.15.47:0/498129436 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcb440aea60 0x7fcb44142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb437fe700 1 -- 172.21.15.47:0/498129436 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcb440aea60 msgr2=0x7fcb44142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb437fe700 1 --2- 172.21.15.47:0/498129436 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcb440aea60 0x7fcb44142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb437fe700 1 -- 172.21.15.47:0/498129436 >> v1:172.21.15.47:6789/0 conn(0x7fcb4405ffb0 legacy=0x7fcb440625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb41ffb700 1 -- 172.21.15.47:0/498129436 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150559362 0 0) 0x7fcb440b5830 con 0x7fcb4405ffb0 2020-07-18T03:25:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb437fe700 1 -- 172.21.15.47:0/498129436 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44146da0 con 0x7fcb44143330 2020-07-18T03:25:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb437fe700 1 --2- 172.21.15.47:0/498129436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb44143330 0x7fcb441471d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fcb3c0032b0 tx=0x7fcb3c0091d0).ready entity=mon.1 client_cookie=23162eba7e5398b3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb41ffb700 1 -- 172.21.15.47:0/498129436 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3c00ba50 con 0x7fcb44143330 2020-07-18T03:25:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb41ffb700 1 -- 172.21.15.47:0/498129436 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c00bbb0 con 0x7fcb44143330 2020-07-18T03:25:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb41ffb700 1 -- 172.21.15.47:0/498129436 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3c00bd20 con 0x7fcb44143330 2020-07-18T03:25:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 -- 172.21.15.47:0/498129436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb44143330 msgr2=0x7fcb441471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.273+0000 7fcb49370700 1 --2- 172.21.15.47:0/498129436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb44143330 0x7fcb441471d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fcb3c0032b0 tx=0x7fcb3c0091d0).stop 2020-07-18T03:25:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.274+0000 7fcb49370700 1 -- 172.21.15.47:0/498129436 shutdown_connections 2020-07-18T03:25:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.274+0000 7fcb49370700 1 --2- 172.21.15.47:0/498129436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb44143330 0x7fcb441471d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.274+0000 7fcb49370700 1 --2- 172.21.15.47:0/498129436 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcb440aea60 0x7fcb44142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.274+0000 7fcb49370700 1 -- 172.21.15.47:0/498129436 >> 172.21.15.47:0/498129436 conn(0x7fcb440b93c0 msgr2=0x7fcb440b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.274+0000 7fcb49370700 1 -- 172.21.15.47:0/498129436 shutdown_connections 2020-07-18T03:25:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.274+0000 7fcb49370700 1 -- 172.21.15.47:0/498129436 wait complete. 2020-07-18T03:25:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.275+0000 7fcb49370700 1 Processor -- start 2020-07-18T03:25:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.275+0000 7fcb49370700 1 -- start start 2020-07-18T03:25:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb49370700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb440aea60 0x7fcb4413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb49370700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb44156260 con 0x7fcb44145bd0 2020-07-18T03:25:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb49370700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb44155fc0 con 0x7fcb4405ffb0 2020-07-18T03:25:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb49370700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4413b700 con 0x7fcb440aea60 2020-07-18T03:25:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb427fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb440aea60 0x7fcb4413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb42ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcb4405ffb0 0x7fcb441556f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39796/0 (socket says 172.21.15.47:39796) 2020-07-18T03:25:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb42ffd700 1 -- 172.21.15.47:0/393338011 learned_addr learned my addr 172.21.15.47:0/393338011 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb427fc700 1 -- 172.21.15.47:0/393338011 >> v1:172.21.15.47:6789/0 conn(0x7fcb4405ffb0 legacy=0x7fcb441556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb427fc700 1 -- 172.21.15.47:0/393338011 >> v1:172.21.15.18:6789/0 conn(0x7fcb44145bd0 legacy=0x7fcb44062d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664401035 0 0) 0x7fcb44155fc0 con 0x7fcb4405ffb0 2020-07-18T03:25:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb427fc700 1 -- 172.21.15.47:0/393338011 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb441462e0 con 0x7fcb440aea60 2020-07-18T03:25:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.276+0000 7fcb427fc700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb440aea60 0x7fcb4413c9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fcb3400a000 tx=0x7fcb34008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.277+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb34009980 con 0x7fcb440aea60 2020-07-18T03:25:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.277+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34009ae0 con 0x7fcb440aea60 2020-07-18T03:25:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.277+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb440634a0 con 0x7fcb440aea60 2020-07-18T03:25:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.277+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb34009e40 con 0x7fcb440aea60 2020-07-18T03:25:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.277+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb44144a10 con 0x7fcb440aea60 2020-07-18T03:25:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.278+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb44155e00 con 0x7fcb440aea60 2020-07-18T03:25:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.278+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fcb3401c460 con 0x7fcb440aea60 2020-07-18T03:25:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.278+0000 7fcb2affd700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcb2004fe70 0x7fcb20052330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.278+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcb34068080 con 0x7fcb440aea60 2020-07-18T03:25:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.278+0000 7fcb42ffd700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcb2004fe70 0x7fcb20052330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.278+0000 7fcb42ffd700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcb2004fe70 0x7fcb20052330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fcb38007650 tx=0x7fcb380078e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.282+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fcb340407b0 con 0x7fcb440aea60 2020-07-18T03:25:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.439+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb44057450 con 0x7fcb440aea60 2020-07-18T03:25:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.439+0000 7fcb2affd700 1 -- 172.21.15.47:0/393338011 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb3400be60 con 0x7fcb440aea60 2020-07-18T03:25:54.442 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcb2004fe70 msgr2=0x7fcb20052330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcb2004fe70 0x7fcb20052330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fcb38007650 tx=0x7fcb380078e0).stop 2020-07-18T03:25:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb440aea60 msgr2=0x7fcb4413c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb440aea60 0x7fcb4413c9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fcb3400a000 tx=0x7fcb34008040).stop 2020-07-18T03:25:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 shutdown_connections 2020-07-18T03:25:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcb2004fe70 0x7fcb20052330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 --2- 172.21.15.47:0/393338011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb440aea60 0x7fcb4413c9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 >> 172.21.15.47:0/393338011 conn(0x7fcb440b93c0 msgr2=0x7fcb4405f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 shutdown_connections 2020-07-18T03:25:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.441+0000 7fcb49370700 1 -- 172.21.15.47:0/393338011 wait complete. 2020-07-18T03:25:54.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2020-07-18T03:25:54.454 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-07-18T03:25:54.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.584+0000 7f0a49855700 1 Processor -- start 2020-07-18T03:25:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.584+0000 7f0a49855700 1 -- start start 2020-07-18T03:25:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.584+0000 7f0a49855700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 0x7f0a440b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a49855700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 0x7f0a440af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a49855700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a44144700 con 0x7f0a4413dbc0 2020-07-18T03:25:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a49855700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a441449a0 con 0x7f0a440ad020 2020-07-18T03:25:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a49855700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440b59e0 con 0x7f0a440af320 2020-07-18T03:25:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a42ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 0x7f0a440b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 0x7f0a440af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 0x7f0a440af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51318/0 (socket says 172.21.15.47:51318) 2020-07-18T03:25:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a42ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 0x7f0a440b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:44396/0 (socket says 172.21.15.47:44396) 2020-07-18T03:25:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 -- 172.21.15.47:0/1905253828 learned_addr learned my addr 172.21.15.47:0/1905253828 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a427fc700 1 --1- 172.21.15.47:0/1905253828 >> v1:172.21.15.18:6789/0 conn(0x7f0a4413dbc0 0x7f0a4413e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58570/0 (socket says 172.21.15.47:58570) 2020-07-18T03:25:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 -- 172.21.15.47:0/1905253828 >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 msgr2=0x7f0a440b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 --2- 172.21.15.47:0/1905253828 >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 0x7f0a440b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 -- 172.21.15.47:0/1905253828 >> v1:172.21.15.18:6789/0 conn(0x7f0a4413dbc0 legacy=0x7f0a4413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 -- 172.21.15.47:0/1905253828 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44144030 con 0x7f0a440af320 2020-07-18T03:25:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a42ffd700 1 --2- 172.21.15.47:0/1905253828 >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 0x7f0a440b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.585+0000 7f0a437fe700 1 --2- 172.21.15.47:0/1905253828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 0x7f0a440af7d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0a34002f30 tx=0x7f0a34006e90).ready entity=mon.2 client_cookie=bd57e6e7369404d4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a41ffb700 1 -- 172.21.15.47:0/1905253828 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a34009850 con 0x7f0a440af320 2020-07-18T03:25:54.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a41ffb700 1 -- 172.21.15.47:0/1905253828 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a340099b0 con 0x7f0a440af320 2020-07-18T03:25:54.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a41ffb700 1 -- 172.21.15.47:0/1905253828 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a34009d10 con 0x7f0a440af320 2020-07-18T03:25:54.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 -- 172.21.15.47:0/1905253828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 msgr2=0x7f0a440af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 --2- 172.21.15.47:0/1905253828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 0x7f0a440af7d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0a34002f30 tx=0x7f0a34006e90).stop 2020-07-18T03:25:54.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 -- 172.21.15.47:0/1905253828 shutdown_connections 2020-07-18T03:25:54.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 --2- 172.21.15.47:0/1905253828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a440af320 0x7f0a440af7d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 --2- 172.21.15.47:0/1905253828 >> v2:172.21.15.18:3300/0 conn(0x7f0a440ad020 0x7f0a440b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 -- 172.21.15.47:0/1905253828 >> 172.21.15.47:0/1905253828 conn(0x7f0a440b93c0 msgr2=0x7f0a440b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 -- 172.21.15.47:0/1905253828 shutdown_connections 2020-07-18T03:25:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.586+0000 7f0a49855700 1 -- 172.21.15.47:0/1905253828 wait complete. 2020-07-18T03:25:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.587+0000 7f0a49855700 1 Processor -- start 2020-07-18T03:25:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a49855700 1 -- start start 2020-07-18T03:25:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a49855700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a4413dbc0 0x7f0a4413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a49855700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a44065d40 con 0x7f0a44065710 2020-07-18T03:25:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a49855700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a44065fe0 con 0x7f0a440ad020 2020-07-18T03:25:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a49855700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a44066280 con 0x7f0a4413dbc0 2020-07-18T03:25:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a427fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a4413dbc0 0x7f0a4413d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a42ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a440ad020 0x7f0a4413cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39810/0 (socket says 172.21.15.47:39810) 2020-07-18T03:25:54.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.588+0000 7f0a42ffd700 1 -- 172.21.15.47:0/115489304 learned_addr learned my addr 172.21.15.47:0/115489304 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:54.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a427fc700 1 -- 172.21.15.47:0/115489304 >> v1:172.21.15.47:6789/0 conn(0x7f0a440ad020 legacy=0x7f0a4413cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a427fc700 1 -- 172.21.15.47:0/115489304 >> v1:172.21.15.18:6789/0 conn(0x7f0a44065710 legacy=0x7f0a44155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942126763 0 0) 0x7f0a44065fe0 con 0x7f0a440ad020 2020-07-18T03:25:54.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a427fc700 1 -- 172.21.15.47:0/115489304 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440b2200 con 0x7f0a4413dbc0 2020-07-18T03:25:54.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a427fc700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4413dbc0 0x7f0a4413d3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0a44060910 tx=0x7f0a38009ca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a38009320 con 0x7f0a4413dbc0 2020-07-18T03:25:54.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a38009480 con 0x7f0a4413dbc0 2020-07-18T03:25:54.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a440b24b0 con 0x7f0a4413dbc0 2020-07-18T03:25:54.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a38007930 con 0x7f0a4413dbc0 2020-07-18T03:25:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.589+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a44155930 con 0x7f0a4413dbc0 2020-07-18T03:25:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.590+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a440b8cd0 con 0x7f0a4413dbc0 2020-07-18T03:25:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.590+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0a38012030 con 0x7f0a4413dbc0 2020-07-18T03:25:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.590+0000 7f0a2affd700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a2004ff00 0x7f0a200523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.590+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0a38069a10 con 0x7f0a4413dbc0 2020-07-18T03:25:54.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.590+0000 7f0a42ffd700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a2004ff00 0x7f0a200523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.591+0000 7f0a42ffd700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a2004ff00 0x7f0a200523c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0a3c007000 tx=0x7f0a3c0091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.594+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0a380421a0 con 0x7f0a4413dbc0 2020-07-18T03:25:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.750+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0a44057450 con 0x7f0a4413dbc0 2020-07-18T03:25:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.751+0000 7f0a2affd700 1 -- 172.21.15.47:0/115489304 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a380457c0 con 0x7f0a4413dbc0 2020-07-18T03:25:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a2004ff00 msgr2=0x7f0a200523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a2004ff00 0x7f0a200523c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0a3c007000 tx=0x7f0a3c0091d0).stop 2020-07-18T03:25:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4413dbc0 msgr2=0x7f0a4413d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4413dbc0 0x7f0a4413d3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0a44060910 tx=0x7f0a38009ca0).stop 2020-07-18T03:25:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 shutdown_connections 2020-07-18T03:25:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a2004ff00 0x7f0a200523c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 --2- 172.21.15.47:0/115489304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4413dbc0 0x7f0a4413d3a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 >> 172.21.15.47:0/115489304 conn(0x7f0a440b93c0 msgr2=0x7f0a440b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 shutdown_connections 2020-07-18T03:25:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.752+0000 7f0a49855700 1 -- 172.21.15.47:0/115489304 wait complete. 2020-07-18T03:25:54.761 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:54.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2020-07-18T03:25:54.766 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-07-18T03:25:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.889+0000 7f775d86d700 1 Processor -- start 2020-07-18T03:25:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f775d86d700 1 -- start start 2020-07-18T03:25:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f775d86d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f775814c7c0 0x7f775814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f775d86d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7758144460 con 0x7f775813dbc0 2020-07-18T03:25:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f775d86d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7758144700 con 0x7f77580ad020 2020-07-18T03:25:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f775d86d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77581449a0 con 0x7f775814c7c0 2020-07-18T03:25:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f7756ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f77580ad020 0x7f77580b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39818/0 (socket says 172.21.15.47:39818) 2020-07-18T03:25:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.890+0000 7f7756ffd700 1 -- 172.21.15.47:0/2633716899 learned_addr learned my addr 172.21.15.47:0/2633716899 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 --2- 172.21.15.47:0/2633716899 >> v2:172.21.15.18:3300/0 conn(0x7f775814c7c0 0x7f775814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233324033 0 0) 0x7f7758144700 con 0x7f77580ad020 2020-07-18T03:25:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7744003240 con 0x7f77580ad020 2020-07-18T03:25:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1585748929 0 0) 0x7f7744003240 con 0x7f77580ad020 2020-07-18T03:25:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 >> v2:172.21.15.18:3300/0 conn(0x7f775814c7c0 msgr2=0x7f775814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 --2- 172.21.15.47:0/2633716899 >> v2:172.21.15.18:3300/0 conn(0x7f775814c7c0 0x7f775814ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 >> v1:172.21.15.18:6789/0 conn(0x7f775813dbc0 legacy=0x7f775813e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758144030 con 0x7f77580ad020 2020-07-18T03:25:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f774c0016d0 con 0x7f77580ad020 2020-07-18T03:25:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 >> v1:172.21.15.47:6789/0 conn(0x7f77580ad020 legacy=0x7f77580b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7744003730 0x7f7744003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77440044e0 0x7f7744004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7744004e50 0x7f774400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77581449a0 con 0x7f77440044e0 2020-07-18T03:25:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744004350 con 0x7f7744004e50 2020-07-18T03:25:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774400bc50 con 0x7f7744003730 2020-07-18T03:25:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7744004e50 0x7f774400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f7756ffd700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7744003730 0x7f7744003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77567fc700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77440044e0 0x7f7744004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 -- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7744003730 msgr2=0x7f7744003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7744003730 0x7f7744003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 -- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77440044e0 msgr2=0x7f7744004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77440044e0 0x7f7744004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.891+0000 7f77577fe700 1 -- 172.21.15.47:0/2633716899 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744003240 con 0x7f7744004e50 2020-07-18T03:25:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f77577fe700 1 -- 172.21.15.47:0/2633716899 reap_dead start 2020-07-18T03:25:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f77577fe700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7744004e50 0x7f774400b730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f775000c000 tx=0x7f7750008040).ready entity=mon.1 client_cookie=c06f84c63131091d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f775000ea90 con 0x7f7744004e50 2020-07-18T03:25:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000ebf0 con 0x7f7744004e50 2020-07-18T03:25:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f7755ffb700 1 -- 172.21.15.47:0/2633716899 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77500123f0 con 0x7f7744004e50 2020-07-18T03:25:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 -- 172.21.15.47:0/2633716899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7744004e50 msgr2=0x7f774400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7744004e50 0x7f774400b730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f775000c000 tx=0x7f7750008040).stop 2020-07-18T03:25:54.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 -- 172.21.15.47:0/2633716899 shutdown_connections 2020-07-18T03:25:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 --2- 172.21.15.47:0/2633716899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7744004e50 0x7f774400b730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 -- 172.21.15.47:0/2633716899 >> 172.21.15.47:0/2633716899 conn(0x7f77580b93c0 msgr2=0x7f77580b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 -- 172.21.15.47:0/2633716899 shutdown_connections 2020-07-18T03:25:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.892+0000 7f775d86d700 1 -- 172.21.15.47:0/2633716899 wait complete. 2020-07-18T03:25:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.893+0000 7f775d86d700 1 Processor -- start 2020-07-18T03:25:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.893+0000 7f775d86d700 1 -- start start 2020-07-18T03:25:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f775d86d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 0x7f77581567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f775d86d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7758062de0 0x7f7758156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f775d86d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77580632b0 con 0x7f7758157190 2020-07-18T03:25:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f775d86d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7758157620 con 0x7f7758062de0 2020-07-18T03:25:54.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f775d86d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77580634e0 con 0x7f7758062a00 2020-07-18T03:25:54.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f77577fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7758157190 0x7f7758065360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39834/0 (socket says 172.21.15.47:39834) 2020-07-18T03:25:54.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f77577fe700 1 -- 172.21.15.47:0/3233609692 learned_addr learned my addr 172.21.15.47:0/3233609692 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:54.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685025406 0 0) 0x7f77580632b0 con 0x7f7758157190 2020-07-18T03:25:54.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7734003240 con 0x7f7758157190 2020-07-18T03:25:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f77567fc700 1 --2- 172.21.15.47:0/3233609692 >> v2:172.21.15.18:3300/0 conn(0x7f7758062de0 0x7f7758156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.894+0000 7f7756ffd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 0x7f77581567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2091453074 0 0) 0x7f7734003240 con 0x7f7758157190 2020-07-18T03:25:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f77580632b0 con 0x7f7758157190 2020-07-18T03:25:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7750006920 con 0x7f7758157190 2020-07-18T03:25:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 msgr2=0x7f77581567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 0x7f77581567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 >> v2:172.21.15.18:3300/0 conn(0x7f7758062de0 msgr2=0x7f7758156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 --2- 172.21.15.47:0/3233609692 >> v2:172.21.15.18:3300/0 conn(0x7f7758062de0 0x7f7758156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 >> v1:172.21.15.47:6789/0 conn(0x7f7758157190 legacy=0x7f7758065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7734003800 0x7f7734003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77340044e0 0x7f7734004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 0x7f7734004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7758157620 con 0x7f7734003800 2020-07-18T03:25:54.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580634e0 con 0x7f77340044e0 2020-07-18T03:25:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7734005300 con 0x7f7758062a00 2020-07-18T03:25:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77340044e0 0x7f7734004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f7756ffd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7734003800 0x7f7734003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77577fe700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 0x7f7734004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 -- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 msgr2=0x7f7734004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7758062a00 0x7f7734004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 -- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7734003800 msgr2=0x7f7734003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7734003800 0x7f7734003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758063780 con 0x7f77340044e0 2020-07-18T03:25:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7758064000 con 0x7f77340044e0 2020-07-18T03:25:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77580b1ac0 con 0x7f77340044e0 2020-07-18T03:25:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.895+0000 7f77567fc700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77340044e0 0x7f7734004920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f7748003000 tx=0x7f7748007e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.896+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f774800fb60 con 0x7f77340044e0 2020-07-18T03:25:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.896+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774800fcc0 con 0x7f77340044e0 2020-07-18T03:25:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.896+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7748006a60 con 0x7f77340044e0 2020-07-18T03:25:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.896+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7758066c50 con 0x7f77340044e0 2020-07-18T03:25:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.896+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7748006bc0 con 0x7f77340044e0 2020-07-18T03:25:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.897+0000 7f773effd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77340582a0 0x7f773405a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.897+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f77480685c0 con 0x7f77340044e0 2020-07-18T03:25:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.897+0000 7f7756ffd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77340582a0 0x7f773405a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.897+0000 7f7756ffd700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77340582a0 0x7f773405a760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f774c005000 tx=0x7f774c006040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:54.900+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7748040d50 con 0x7f77340044e0 2020-07-18T03:25:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.062+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7758057450 con 0x7f77340044e0 2020-07-18T03:25:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.062+0000 7f773effd700 1 -- 172.21.15.47:0/3233609692 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7748044370 con 0x7f77340044e0 2020-07-18T03:25:55.065 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2020-07-18T03:25:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77340582a0 msgr2=0x7f773405a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77340582a0 0x7f773405a760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f774c005000 tx=0x7f774c006040).stop 2020-07-18T03:25:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77340044e0 msgr2=0x7f7734004920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 --2- 172.21.15.47:0/3233609692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77340044e0 0x7f7734004920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f7748003000 tx=0x7f7748007e60).stop 2020-07-18T03:25:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f77577fe700 1 -- 172.21.15.47:0/3233609692 reap_dead start 2020-07-18T03:25:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 shutdown_connections 2020-07-18T03:25:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 >> 172.21.15.47:0/3233609692 conn(0x7f77580b93c0 msgr2=0x7f775805d200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 shutdown_connections 2020-07-18T03:25:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.063+0000 7f775d86d700 1 -- 172.21.15.47:0/3233609692 wait complete. 2020-07-18T03:25:55.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2020-07-18T03:25:55.076 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-07-18T03:25:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.190+0000 7f47e2117700 1 Processor -- start 2020-07-18T03:25:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.190+0000 7f47e2117700 1 -- start start 2020-07-18T03:25:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47e2117700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 0x7f47dc061c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47e2117700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f47dc1492b0 0x7f47dc05fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47e2117700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc145500 con 0x7f47dc148e30 2020-07-18T03:25:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47e2117700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc146150 con 0x7f47dc1492b0 2020-07-18T03:25:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47e2117700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc145770 con 0x7f47dc145120 2020-07-18T03:25:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47db7fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 0x7f47dc061c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47dbfff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f47dc1492b0 0x7f47dc05fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47db7fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 0x7f47dc061c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51354/0 (socket says 172.21.15.47:51354) 2020-07-18T03:25:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47daffd700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f47dc148e30 0x7f47dc05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:58610/0 (socket says 172.21.15.47:58610) 2020-07-18T03:25:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47dbfff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f47dc1492b0 0x7f47dc05fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:44436/0 (socket says 172.21.15.47:44436) 2020-07-18T03:25:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.191+0000 7f47daffd700 1 -- 172.21.15.47:0/2592775310 learned_addr learned my addr 172.21.15.47:0/2592775310 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47da7fc700 1 -- 172.21.15.47:0/2592775310 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229920145 0 0) 0x7f47dc145500 con 0x7f47dc148e30 2020-07-18T03:25:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47da7fc700 1 -- 172.21.15.47:0/2592775310 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c8003240 con 0x7f47dc148e30 2020-07-18T03:25:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47dbfff700 1 -- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 msgr2=0x7f47dc061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47dbfff700 1 --2- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 0x7f47dc061c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47dbfff700 1 -- 172.21.15.47:0/2592775310 >> v1:172.21.15.18:6789/0 conn(0x7f47dc148e30 legacy=0x7f47dc05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47dbfff700 1 -- 172.21.15.47:0/2592775310 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc145a10 con 0x7f47dc1492b0 2020-07-18T03:25:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47db7fe700 1 --2- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 0x7f47dc061c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47dbfff700 1 --2- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc1492b0 0x7f47dc05fcb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f47d40032b0 tx=0x7f47d40091d0).ready entity=mon.0 client_cookie=30a0ffc1f16ae942 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47da7fc700 1 -- 172.21.15.47:0/2592775310 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d400b940 con 0x7f47dc1492b0 2020-07-18T03:25:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47da7fc700 1 -- 172.21.15.47:0/2592775310 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400baa0 con 0x7f47dc1492b0 2020-07-18T03:25:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47da7fc700 1 -- 172.21.15.47:0/2592775310 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d400bc10 con 0x7f47dc1492b0 2020-07-18T03:25:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 -- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc1492b0 msgr2=0x7f47dc05fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 --2- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc1492b0 0x7f47dc05fcb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f47d40032b0 tx=0x7f47d40091d0).stop 2020-07-18T03:25:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 -- 172.21.15.47:0/2592775310 shutdown_connections 2020-07-18T03:25:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 --2- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc1492b0 0x7f47dc05fcb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 --2- 172.21.15.47:0/2592775310 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc145120 0x7f47dc061c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 -- 172.21.15.47:0/2592775310 >> 172.21.15.47:0/2592775310 conn(0x7f47dc0b93c0 msgr2=0x7f47dc0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 -- 172.21.15.47:0/2592775310 shutdown_connections 2020-07-18T03:25:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.192+0000 7f47e2117700 1 -- 172.21.15.47:0/2592775310 wait complete. 2020-07-18T03:25:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.194+0000 7f47e2117700 1 Processor -- start 2020-07-18T03:25:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.194+0000 7f47e2117700 1 -- start start 2020-07-18T03:25:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47dc148e30 0x7f47dc13df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc141cd0 0x7f47dc13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc13fae0 con 0x7f47dc145120 2020-07-18T03:25:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc1472a0 con 0x7f47dc148e30 2020-07-18T03:25:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc140790 con 0x7f47dc141cd0 2020-07-18T03:25:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47dc148e30 0x7f47dc13df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47dc148e30 0x7f47dc13df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58764/0 (socket says 172.21.15.47:58764) 2020-07-18T03:25:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 -- 172.21.15.47:0/4109000907 learned_addr learned my addr 172.21.15.47:0/4109000907 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47dbfff700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc141cd0 0x7f47dc13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 -- 172.21.15.47:0/4109000907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc141cd0 msgr2=0x7f47dc13e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc141cd0 0x7f47dc13e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445746886 0 0) 0x7f47dc13fae0 con 0x7f47dc145120 2020-07-18T03:25:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 -- 172.21.15.47:0/4109000907 >> v1:172.21.15.18:6789/0 conn(0x7f47dc145120 legacy=0x7f47dc13d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 -- 172.21.15.47:0/4109000907 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc140a30 con 0x7f47dc148e30 2020-07-18T03:25:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47dbfff700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc141cd0 0x7f47dc13e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47daffd700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47dc148e30 0x7f47dc13df90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f47d00010d0 tx=0x7f47d0007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d00072e0 con 0x7f47dc148e30 2020-07-18T03:25:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc13f7d0 con 0x7f47dc148e30 2020-07-18T03:25:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d0007440 con 0x7f47dc148e30 2020-07-18T03:25:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d00057e0 con 0x7f47dc148e30 2020-07-18T03:25:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.195+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc13e970 con 0x7f47dc148e30 2020-07-18T03:25:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.196+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc148c00 con 0x7f47dc148e30 2020-07-18T03:25:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.196+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f47d001a060 con 0x7f47dc148e30 2020-07-18T03:25:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.197+0000 7f47bb7fe700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47b404fed0 0x7f47b4052390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.197+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f47d00674c0 con 0x7f47dc148e30 2020-07-18T03:25:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.197+0000 7f47db7fe700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47b404fed0 0x7f47b4052390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.197+0000 7f47db7fe700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47b404fed0 0x7f47b4052390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f47cc002f30 tx=0x7f47cc006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.200+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f47d003fbd0 con 0x7f47dc148e30 2020-07-18T03:25:55.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.357+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f47dc140bc0 con 0x7f47dc148e30 2020-07-18T03:25:55.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.358+0000 7f47bb7fe700 1 -- 172.21.15.47:0/4109000907 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47d0018070 con 0x7f47dc148e30 2020-07-18T03:25:55.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47b404fed0 msgr2=0x7f47b4052390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47b404fed0 0x7f47b4052390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f47cc002f30 tx=0x7f47cc006dd0).stop 2020-07-18T03:25:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47dc148e30 msgr2=0x7f47dc13df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47dc148e30 0x7f47dc13df90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f47d00010d0 tx=0x7f47d0007ab0).stop 2020-07-18T03:25:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 shutdown_connections 2020-07-18T03:25:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47b404fed0 0x7f47b4052390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47dc141cd0 0x7f47dc13e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 --2- 172.21.15.47:0/4109000907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47dc148e30 0x7f47dc13df90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 >> 172.21.15.47:0/4109000907 conn(0x7f47dc0b93c0 msgr2=0x7f47dc061c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 shutdown_connections 2020-07-18T03:25:55.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.359+0000 7f47e2117700 1 -- 172.21.15.47:0/4109000907 wait complete. 2020-07-18T03:25:55.371 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:55.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2020-07-18T03:25:55.372 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-07-18T03:25:55.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.487+0000 7fd34e115700 1 Processor -- start 2020-07-18T03:25:55.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.487+0000 7fd34e115700 1 -- start start 2020-07-18T03:25:55.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd3480ad020 0x7fd3480b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34813dbc0 0x7fd34813e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd348147640 0x7fd348147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd348144700 con 0x7fd3480ad020 2020-07-18T03:25:55.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3481449a0 con 0x7fd34813dbc0 2020-07-18T03:25:55.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34813e510 con 0x7fd348147640 2020-07-18T03:25:55.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34813dbc0 0x7fd34813e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34813dbc0 0x7fd34813e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58774/0 (socket says 172.21.15.47:58774) 2020-07-18T03:25:55.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 -- 172.21.15.47:0/1320746347 learned_addr learned my addr 172.21.15.47:0/1320746347 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:55.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd347fff700 1 --2- 172.21.15.47:0/1320746347 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd348147640 0x7fd348147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd3477fe700 1 --2- 172.21.15.47:0/1320746347 >> v2:172.21.15.18:3300/0 conn(0x7fd3480ad020 0x7fd3480b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 -- 172.21.15.47:0/1320746347 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd348147640 msgr2=0x7fd348147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:55.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 --2- 172.21.15.47:0/1320746347 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd348147640 0x7fd348147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 -- 172.21.15.47:0/1320746347 >> v2:172.21.15.18:3300/0 conn(0x7fd3480ad020 msgr2=0x7fd3480b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 --2- 172.21.15.47:0/1320746347 >> v2:172.21.15.18:3300/0 conn(0x7fd3480ad020 0x7fd3480b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 -- 172.21.15.47:0/1320746347 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480b59e0 con 0x7fd34813dbc0 2020-07-18T03:25:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd346ffd700 1 --2- 172.21.15.47:0/1320746347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813dbc0 0x7fd34813e020 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd33c002f30 tx=0x7fd33c006e90).ready entity=mon.1 client_cookie=b5b2871a60c064ba server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd3467fc700 1 -- 172.21.15.47:0/1320746347 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33c009730 con 0x7fd34813dbc0 2020-07-18T03:25:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd3467fc700 1 -- 172.21.15.47:0/1320746347 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c009890 con 0x7fd34813dbc0 2020-07-18T03:25:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd3467fc700 1 -- 172.21.15.47:0/1320746347 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33c009bf0 con 0x7fd34813dbc0 2020-07-18T03:25:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 -- 172.21.15.47:0/1320746347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813dbc0 msgr2=0x7fd34813e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- 172.21.15.47:0/1320746347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813dbc0 0x7fd34813e020 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd33c002f30 tx=0x7fd33c006e90).stop 2020-07-18T03:25:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 -- 172.21.15.47:0/1320746347 shutdown_connections 2020-07-18T03:25:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- 172.21.15.47:0/1320746347 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd348147640 0x7fd348147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- 172.21.15.47:0/1320746347 >> v2:172.21.15.18:3300/0 conn(0x7fd3480ad020 0x7fd3480b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 --2- 172.21.15.47:0/1320746347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813dbc0 0x7fd34813e020 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.488+0000 7fd34e115700 1 -- 172.21.15.47:0/1320746347 >> 172.21.15.47:0/1320746347 conn(0x7fd3480b93c0 msgr2=0x7fd3480b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.489+0000 7fd34e115700 1 -- 172.21.15.47:0/1320746347 shutdown_connections 2020-07-18T03:25:55.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.489+0000 7fd34e115700 1 -- 172.21.15.47:0/1320746347 wait complete. 2020-07-18T03:25:55.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.490+0000 7fd34e115700 1 Processor -- start 2020-07-18T03:25:55.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.490+0000 7fd34e115700 1 -- start start 2020-07-18T03:25:55.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd34e115700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3480ad020 0x7fd34813d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd34e115700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd34813a0b0 0x7fd348138b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd34e115700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd348139520 con 0x7fd348147640 2020-07-18T03:25:55.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd34e115700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3481397c0 con 0x7fd34813a0b0 2020-07-18T03:25:55.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd34e115700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd348139a60 con 0x7fd3480ad020 2020-07-18T03:25:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd346ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd348147640 0x7fd348137e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39866/0 (socket says 172.21.15.47:39866) 2020-07-18T03:25:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd346ffd700 1 -- 172.21.15.47:0/563500269 learned_addr learned my addr 172.21.15.47:0/563500269 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd347fff700 1 --2- 172.21.15.47:0/563500269 >> v2:172.21.15.47:3300/0 conn(0x7fd34813a0b0 0x7fd348138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd3477fe700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3480ad020 0x7fd34813d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd347fff700 1 -- 172.21.15.47:0/563500269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3480ad020 msgr2=0x7fd34813d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd347fff700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3480ad020 0x7fd34813d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd347fff700 1 -- 172.21.15.47:0/563500269 >> v1:172.21.15.47:6789/0 conn(0x7fd348147640 legacy=0x7fd348137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222816216 0 0) 0x7fd348139520 con 0x7fd348147640 2020-07-18T03:25:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd347fff700 1 -- 172.21.15.47:0/563500269 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34813a840 con 0x7fd34813a0b0 2020-07-18T03:25:55.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd347fff700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813a0b0 0x7fd348138b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd33800c000 tx=0x7fd338006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33800c860 con 0x7fd34813a0b0 2020-07-18T03:25:55.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800c9c0 con 0x7fd34813a0b0 2020-07-18T03:25:55.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.491+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd348065b90 con 0x7fd34813a0b0 2020-07-18T03:25:55.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.492+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33800cd20 con 0x7fd34813a0b0 2020-07-18T03:25:55.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.492+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd348139230 con 0x7fd34813a0b0 2020-07-18T03:25:55.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.492+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34805b150 con 0x7fd34813a0b0 2020-07-18T03:25:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.493+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd33801c460 con 0x7fd34813a0b0 2020-07-18T03:25:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.493+0000 7fd3277fe700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd32004ff20 0x7fd3200523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.493+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd338068010 con 0x7fd34813a0b0 2020-07-18T03:25:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.493+0000 7fd3477fe700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd32004ff20 0x7fd3200523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.493+0000 7fd3477fe700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd32004ff20 0x7fd3200523e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd34000b000 tx=0x7fd3400091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.496+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd3380407f0 con 0x7fd34813a0b0 2020-07-18T03:25:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.658+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd348057450 con 0x7fd34813a0b0 2020-07-18T03:25:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.658+0000 7fd3277fe700 1 -- 172.21.15.47:0/563500269 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd338043e10 con 0x7fd34813a0b0 2020-07-18T03:25:55.661 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2020-07-18T03:25:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd32004ff20 msgr2=0x7fd3200523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd32004ff20 0x7fd3200523e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd34000b000 tx=0x7fd3400091d0).stop 2020-07-18T03:25:55.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813a0b0 msgr2=0x7fd348138b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813a0b0 0x7fd348138b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd33800c000 tx=0x7fd338006dd0).stop 2020-07-18T03:25:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 shutdown_connections 2020-07-18T03:25:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd34813a0b0 0x7fd348138b00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3480ad020 0x7fd34813d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 --2- 172.21.15.47:0/563500269 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd32004ff20 0x7fd3200523e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 >> 172.21.15.47:0/563500269 conn(0x7fd3480b93c0 msgr2=0x7fd3480b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 shutdown_connections 2020-07-18T03:25:55.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.660+0000 7fd34e115700 1 -- 172.21.15.47:0/563500269 wait complete. 2020-07-18T03:25:55.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2020-07-18T03:25:55.672 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-07-18T03:25:55.673 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-07-18T03:25:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.786+0000 7fe17cdc6700 1 Processor -- start 2020-07-18T03:25:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.786+0000 7fe17cdc6700 1 -- start start 2020-07-18T03:25:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe17cdc6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe1780ad100 0x7fe1780b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe17cdc6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe178147a00 0x7fe178147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe17cdc6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe178147330 con 0x7fe17805f5a0 2020-07-18T03:25:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe17cdc6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1780ae530 con 0x7fe1780ad100 2020-07-18T03:25:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe17cdc6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1780b57e0 con 0x7fe178147a00 2020-07-18T03:25:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe175d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe178147a00 0x7fe178147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe175d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe178147a00 0x7fe178147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58784/0 (socket says 172.21.15.47:58784) 2020-07-18T03:25:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.787+0000 7fe175d9b700 1 -- 172.21.15.47:0/3083168470 learned_addr learned my addr 172.21.15.47:0/3083168470 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17659c700 1 --2- 172.21.15.47:0/3083168470 >> v2:172.21.15.18:3300/0 conn(0x7fe1780ad100 0x7fe1780b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe175d9b700 1 -- 172.21.15.47:0/3083168470 >> v2:172.21.15.18:3300/0 conn(0x7fe1780ad100 msgr2=0x7fe1780b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe175d9b700 1 --2- 172.21.15.47:0/3083168470 >> v2:172.21.15.18:3300/0 conn(0x7fe1780ad100 0x7fe1780b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe175d9b700 1 -- 172.21.15.47:0/3083168470 >> v1:172.21.15.18:6789/0 conn(0x7fe17805f5a0 legacy=0x7fe17805f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe175d9b700 1 -- 172.21.15.47:0/3083168470 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780b5a80 con 0x7fe178147a00 2020-07-18T03:25:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe175d9b700 1 --2- 172.21.15.47:0/3083168470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe178147a00 0x7fe178147e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe17805ca30 tx=0x7fe168006e90).ready entity=mon.1 client_cookie=255ea6e23cc51d8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17559a700 1 -- 172.21.15.47:0/3083168470 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe168009930 con 0x7fe178147a00 2020-07-18T03:25:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17559a700 1 -- 172.21.15.47:0/3083168470 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168009a90 con 0x7fe178147a00 2020-07-18T03:25:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17559a700 1 -- 172.21.15.47:0/3083168470 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe168009e30 con 0x7fe178147a00 2020-07-18T03:25:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3083168470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe178147a00 msgr2=0x7fe178147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3083168470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe178147a00 0x7fe178147e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe17805ca30 tx=0x7fe168006e90).stop 2020-07-18T03:25:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3083168470 shutdown_connections 2020-07-18T03:25:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3083168470 >> v2:172.21.15.18:3300/0 conn(0x7fe1780ad100 0x7fe1780b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3083168470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe178147a00 0x7fe178147e40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3083168470 >> 172.21.15.47:0/3083168470 conn(0x7fe1780b9340 msgr2=0x7fe1780b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3083168470 shutdown_connections 2020-07-18T03:25:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.788+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3083168470 wait complete. 2020-07-18T03:25:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17cdc6700 1 Processor -- start 2020-07-18T03:25:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17cdc6700 1 -- start start 2020-07-18T03:25:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17cdc6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe17805f5a0 0x7fe178066580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17cdc6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe178065900 con 0x7fe178157310 2020-07-18T03:25:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17cdc6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe178065ba0 con 0x7fe1780ad100 2020-07-18T03:25:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17cdc6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe178065e40 con 0x7fe17805f5a0 2020-07-18T03:25:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe175d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe1780ad100 0x7fe1780b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39876/0 (socket says 172.21.15.47:39876) 2020-07-18T03:25:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe175d9b700 1 -- 172.21.15.47:0/3387533324 learned_addr learned my addr 172.21.15.47:0/3387533324 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.790+0000 7fe17659c700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe17805f5a0 0x7fe178066580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701017070 0 0) 0x7fe178065ba0 con 0x7fe1780ad100 2020-07-18T03:25:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe154003240 con 0x7fe1780ad100 2020-07-18T03:25:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343568536 0 0) 0x7fe178065900 con 0x7fe178157310 2020-07-18T03:25:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178065ba0 con 0x7fe178157310 2020-07-18T03:25:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2938800566 0 0) 0x7fe154003240 con 0x7fe1780ad100 2020-07-18T03:25:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe17659c700 1 -- 172.21.15.47:0/3387533324 >> v1:172.21.15.47:6789/0 conn(0x7fe1780ad100 legacy=0x7fe1780b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe17659c700 1 -- 172.21.15.47:0/3387533324 >> v1:172.21.15.18:6789/0 conn(0x7fe178157310 legacy=0x7fe17812da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe17659c700 1 -- 172.21.15.47:0/3387533324 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780660e0 con 0x7fe17805f5a0 2020-07-18T03:25:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe17659c700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe17805f5a0 0x7fe178066580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fe16800b180 tx=0x7fe160008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe168009930 con 0x7fe17805f5a0 2020-07-18T03:25:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe178157760 con 0x7fe17805f5a0 2020-07-18T03:25:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16000c030 con 0x7fe17805f5a0 2020-07-18T03:25:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe16000f600 con 0x7fe17805f5a0 2020-07-18T03:25:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.791+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17812e170 con 0x7fe17805f5a0 2020-07-18T03:25:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.792+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178066270 con 0x7fe17805f5a0 2020-07-18T03:25:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.792+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe16000f760 con 0x7fe17805f5a0 2020-07-18T03:25:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.793+0000 7fe16e7fc700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe154050420 0x7fe1540528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.793+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe160066e60 con 0x7fe17805f5a0 2020-07-18T03:25:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.793+0000 7fe175d9b700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe154050420 0x7fe1540528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.793+0000 7fe175d9b700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe154050420 0x7fe1540528e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe16800b870 tx=0x7fe168015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.797+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe16003f4f0 con 0x7fe17805f5a0 2020-07-18T03:25:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.950+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 --> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe178057450 con 0x7fe154050420 2020-07-18T03:25:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.953+0000 7fe16e7fc700 1 -- 172.21.15.47:0/3387533324 <== mgr.4108 v2:172.21.15.47:6800/23227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82297 (secure 0 0 0) 0x7fe178057450 con 0x7fe154050420 2020-07-18T03:25:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe154050420 msgr2=0x7fe1540528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe154050420 0x7fe1540528e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe16800b870 tx=0x7fe168015000).stop 2020-07-18T03:25:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe17805f5a0 msgr2=0x7fe178066580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe17805f5a0 0x7fe178066580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fe16800b180 tx=0x7fe160008040).stop 2020-07-18T03:25:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 shutdown_connections 2020-07-18T03:25:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe17805f5a0 0x7fe178066580 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 --2- 172.21.15.47:0/3387533324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe154050420 0x7fe1540528e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 >> 172.21.15.47:0/3387533324 conn(0x7fe1780b9340 msgr2=0x7fe1780b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 shutdown_connections 2020-07-18T03:25:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:55.955+0000 7fe17cdc6700 1 -- 172.21.15.47:0/3387533324 wait complete. 2020-07-18T03:25:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2020-07-18T03:25:55.968 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2020-07-18T03:25:55.687192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":829864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001873"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.560167+0000","last_change":"2020-07-18T03:25:45.559076+0000","last_active":"2020-07-18T03:25:45.560167+0000","last_peered":"2020-07-18T03:25:45.560167+0000","last_clean":"2020-07-18T03:25:45.560167+0000","last_became_active":"2020-07-18T03:25:45.558416+0000","last_became_peered":"2020-07-18T03:25:45.558416+0000","last_unstale":"2020-07-18T03:25:45.560167+0000","last_undegraded":"2020-07-18T03:25:45.560167+0000","last_fullsized":"2020-07-18T03:25:45.560167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.534879+0000","last_change":"2020-07-18T03:25:39.516626+0000","last_active":"2020-07-18T03:25:45.534879+0000","last_peered":"2020-07-18T03:25:45.534879+0000","last_clean":"2020-07-18T03:25:45.534879+0000","last_became_active":"2020-07-18T03:25:39.516260+0000","last_became_peered":"2020-07-18T03:25:39.516260+0000","last_unstale":"2020-07-18T03:25:45.534879+0000","last_undegraded":"2020-07-18T03:25:45.534879+0000","last_fullsized":"2020-07-18T03:25:45.534879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.594523+0000","last_change":"2020-07-18T03:25:42.530140+0000","last_active":"2020-07-18T03:25:48.594523+0000","last_peered":"2020-07-18T03:25:48.594523+0000","last_clean":"2020-07-18T03:25:48.594523+0000","last_became_active":"2020-07-18T03:25:42.529759+0000","last_became_peered":"2020-07-18T03:25:42.529759+0000","last_unstale":"2020-07-18T03:25:48.594523+0000","last_undegraded":"2020-07-18T03:25:48.594523+0000","last_fullsized":"2020-07-18T03:25:48.594523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.564041+0000","last_change":"2020-07-18T03:25:45.562946+0000","last_active":"2020-07-18T03:25:45.564041+0000","last_peered":"2020-07-18T03:25:45.564041+0000","last_clean":"2020-07-18T03:25:45.564041+0000","last_became_active":"2020-07-18T03:25:45.561799+0000","last_became_peered":"2020-07-18T03:25:45.561799+0000","last_unstale":"2020-07-18T03:25:45.564041+0000","last_undegraded":"2020-07-18T03:25:45.564041+0000","last_fullsized":"2020-07-18T03:25:45.564041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532126+0000","last_change":"2020-07-18T03:25:39.515578+0000","last_active":"2020-07-18T03:25:45.532126+0000","last_peered":"2020-07-18T03:25:45.532126+0000","last_clean":"2020-07-18T03:25:45.532126+0000","last_became_active":"2020-07-18T03:25:39.515122+0000","last_became_peered":"2020-07-18T03:25:39.515122+0000","last_unstale":"2020-07-18T03:25:45.532126+0000","last_undegraded":"2020-07-18T03:25:45.532126+0000","last_fullsized":"2020-07-18T03:25:45.532126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.591392+0000","last_change":"2020-07-18T03:25:42.529236+0000","last_active":"2020-07-18T03:25:48.591392+0000","last_peered":"2020-07-18T03:25:48.591392+0000","last_clean":"2020-07-18T03:25:48.591392+0000","last_became_active":"2020-07-18T03:25:42.528469+0000","last_became_peered":"2020-07-18T03:25:42.528469+0000","last_unstale":"2020-07-18T03:25:48.591392+0000","last_undegraded":"2020-07-18T03:25:48.591392+0000","last_fullsized":"2020-07-18T03:25:48.591392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.563291+0000","last_change":"2020-07-18T03:25:45.561907+0000","last_active":"2020-07-18T03:25:45.563291+0000","last_peered":"2020-07-18T03:25:45.563291+0000","last_clean":"2020-07-18T03:25:45.563291+0000","last_became_active":"2020-07-18T03:25:45.561150+0000","last_became_peered":"2020-07-18T03:25:45.561150+0000","last_unstale":"2020-07-18T03:25:45.563291+0000","last_undegraded":"2020-07-18T03:25:45.563291+0000","last_fullsized":"2020-07-18T03:25:45.563291+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.556180+0000","last_change":"2020-07-18T03:25:39.516123+0000","last_active":"2020-07-18T03:25:45.556180+0000","last_peered":"2020-07-18T03:25:45.556180+0000","last_clean":"2020-07-18T03:25:45.556180+0000","last_became_active":"2020-07-18T03:25:39.515676+0000","last_became_peered":"2020-07-18T03:25:39.515676+0000","last_unstale":"2020-07-18T03:25:45.556180+0000","last_undegraded":"2020-07-18T03:25:45.556180+0000","last_fullsized":"2020-07-18T03:25:45.556180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.598653+0000","last_change":"2020-07-18T03:25:43.018556+0000","last_active":"2020-07-18T03:25:48.598653+0000","last_peered":"2020-07-18T03:25:48.598653+0000","last_clean":"2020-07-18T03:25:48.598653+0000","last_became_active":"2020-07-18T03:25:43.018093+0000","last_became_peered":"2020-07-18T03:25:43.018093+0000","last_unstale":"2020-07-18T03:25:48.598653+0000","last_undegraded":"2020-07-18T03:25:48.598653+0000","last_fullsized":"2020-07-18T03:25:48.598653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.561085+0000","last_change":"2020-07-18T03:25:45.559375+0000","last_active":"2020-07-18T03:25:45.561085+0000","last_peered":"2020-07-18T03:25:45.561085+0000","last_clean":"2020-07-18T03:25:45.561085+0000","last_became_active":"2020-07-18T03:25:45.558097+0000","last_became_peered":"2020-07-18T03:25:45.558097+0000","last_unstale":"2020-07-18T03:25:45.561085+0000","last_undegraded":"2020-07-18T03:25:45.561085+0000","last_fullsized":"2020-07-18T03:25:45.561085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532121+0000","last_change":"2020-07-18T03:25:40.021509+0000","last_active":"2020-07-18T03:25:45.532121+0000","last_peered":"2020-07-18T03:25:45.532121+0000","last_clean":"2020-07-18T03:25:45.532121+0000","last_became_active":"2020-07-18T03:25:40.021077+0000","last_became_peered":"2020-07-18T03:25:40.021077+0000","last_unstale":"2020-07-18T03:25:45.532121+0000","last_undegraded":"2020-07-18T03:25:45.532121+0000","last_fullsized":"2020-07-18T03:25:45.532121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.556446+0000","last_change":"2020-07-18T03:25:35.015179+0000","last_active":"2020-07-18T03:25:45.556446+0000","last_peered":"2020-07-18T03:25:45.556446+0000","last_clean":"2020-07-18T03:25:45.556446+0000","last_became_active":"2020-07-18T03:25:35.014863+0000","last_became_peered":"2020-07-18T03:25:35.014863+0000","last_unstale":"2020-07-18T03:25:45.556446+0000","last_undegraded":"2020-07-18T03:25:45.556446+0000","last_fullsized":"2020-07-18T03:25:45.556446+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":5,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:27.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:27.868182+0000","last_clean_scrub_stamp":"2020-07-18T03:25:27.868182+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:45.902747+0000","last_change":"2020-07-18T03:25:42.528243+0000","last_active":"2020-07-18T03:25:45.902747+0000","last_peered":"2020-07-18T03:25:45.902747+0000","last_clean":"2020-07-18T03:25:45.902747+0000","last_became_active":"2020-07-18T03:25:42.527785+0000","last_became_peered":"2020-07-18T03:25:42.527785+0000","last_unstale":"2020-07-18T03:25:45.902747+0000","last_undegraded":"2020-07-18T03:25:45.902747+0000","last_fullsized":"2020-07-18T03:25:45.902747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.559390+0000","last_change":"2020-07-18T03:25:45.558635+0000","last_active":"2020-07-18T03:25:45.559390+0000","last_peered":"2020-07-18T03:25:45.559390+0000","last_clean":"2020-07-18T03:25:45.559390+0000","last_became_active":"2020-07-18T03:25:45.557980+0000","last_became_peered":"2020-07-18T03:25:45.557980+0000","last_unstale":"2020-07-18T03:25:45.559390+0000","last_undegraded":"2020-07-18T03:25:45.559390+0000","last_fullsized":"2020-07-18T03:25:45.559390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532128+0000","last_change":"2020-07-18T03:25:39.518438+0000","last_active":"2020-07-18T03:25:45.532128+0000","last_peered":"2020-07-18T03:25:45.532128+0000","last_clean":"2020-07-18T03:25:45.532128+0000","last_became_active":"2020-07-18T03:25:39.518013+0000","last_became_peered":"2020-07-18T03:25:39.518013+0000","last_unstale":"2020-07-18T03:25:45.532128+0000","last_undegraded":"2020-07-18T03:25:45.532128+0000","last_fullsized":"2020-07-18T03:25:45.532128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532680+0000","last_change":"2020-07-18T03:25:42.528662+0000","last_active":"2020-07-18T03:25:45.532680+0000","last_peered":"2020-07-18T03:25:45.532680+0000","last_clean":"2020-07-18T03:25:45.532680+0000","last_became_active":"2020-07-18T03:25:42.528211+0000","last_became_peered":"2020-07-18T03:25:42.528211+0000","last_unstale":"2020-07-18T03:25:45.532680+0000","last_undegraded":"2020-07-18T03:25:45.532680+0000","last_fullsized":"2020-07-18T03:25:45.532680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.560976+0000","last_change":"2020-07-18T03:25:45.558873+0000","last_active":"2020-07-18T03:25:45.560976+0000","last_peered":"2020-07-18T03:25:45.560976+0000","last_clean":"2020-07-18T03:25:45.560976+0000","last_became_active":"2020-07-18T03:25:45.558201+0000","last_became_peered":"2020-07-18T03:25:45.558201+0000","last_unstale":"2020-07-18T03:25:45.560976+0000","last_undegraded":"2020-07-18T03:25:45.560976+0000","last_fullsized":"2020-07-18T03:25:45.560976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.533031+0000","last_change":"2020-07-18T03:25:39.518082+0000","last_active":"2020-07-18T03:25:45.533031+0000","last_peered":"2020-07-18T03:25:45.533031+0000","last_clean":"2020-07-18T03:25:45.533031+0000","last_became_active":"2020-07-18T03:25:39.517664+0000","last_became_peered":"2020-07-18T03:25:39.517664+0000","last_unstale":"2020-07-18T03:25:45.533031+0000","last_undegraded":"2020-07-18T03:25:45.533031+0000","last_fullsized":"2020-07-18T03:25:45.533031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.596378+0000","last_change":"2020-07-18T03:25:42.527102+0000","last_active":"2020-07-18T03:25:48.596378+0000","last_peered":"2020-07-18T03:25:48.596378+0000","last_clean":"2020-07-18T03:25:48.596378+0000","last_became_active":"2020-07-18T03:25:42.526748+0000","last_became_peered":"2020-07-18T03:25:42.526748+0000","last_unstale":"2020-07-18T03:25:48.596378+0000","last_undegraded":"2020-07-18T03:25:48.596378+0000","last_fullsized":"2020-07-18T03:25:48.596378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.561403+0000","last_change":"2020-07-18T03:25:45.560783+0000","last_active":"2020-07-18T03:25:45.561403+0000","last_peered":"2020-07-18T03:25:45.561403+0000","last_clean":"2020-07-18T03:25:45.561403+0000","last_became_active":"2020-07-18T03:25:45.559895+0000","last_became_peered":"2020-07-18T03:25:45.559895+0000","last_unstale":"2020-07-18T03:25:45.561403+0000","last_undegraded":"2020-07-18T03:25:45.561403+0000","last_fullsized":"2020-07-18T03:25:45.561403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"24'6","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.595088+0000","last_change":"2020-07-18T03:25:42.525475+0000","last_active":"2020-07-18T03:25:48.595088+0000","last_peered":"2020-07-18T03:25:48.595088+0000","last_clean":"2020-07-18T03:25:48.595088+0000","last_became_active":"2020-07-18T03:25:42.525035+0000","last_became_peered":"2020-07-18T03:25:42.525035+0000","last_unstale":"2020-07-18T03:25:48.595088+0000","last_undegraded":"2020-07-18T03:25:48.595088+0000","last_fullsized":"2020-07-18T03:25:48.595088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.531914+0000","last_change":"2020-07-18T03:25:39.962705+0000","last_active":"2020-07-18T03:25:45.531914+0000","last_peered":"2020-07-18T03:25:45.531914+0000","last_clean":"2020-07-18T03:25:45.531914+0000","last_became_active":"2020-07-18T03:25:39.962375+0000","last_became_peered":"2020-07-18T03:25:39.962375+0000","last_unstale":"2020-07-18T03:25:45.531914+0000","last_undegraded":"2020-07-18T03:25:45.531914+0000","last_fullsized":"2020-07-18T03:25:45.531914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.560278+0000","last_change":"2020-07-18T03:25:45.558814+0000","last_active":"2020-07-18T03:25:45.560278+0000","last_peered":"2020-07-18T03:25:45.560278+0000","last_clean":"2020-07-18T03:25:45.560278+0000","last_became_active":"2020-07-18T03:25:45.558190+0000","last_became_peered":"2020-07-18T03:25:45.558190+0000","last_unstale":"2020-07-18T03:25:45.560278+0000","last_undegraded":"2020-07-18T03:25:45.560278+0000","last_fullsized":"2020-07-18T03:25:45.560278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532300+0000","last_change":"2020-07-18T03:25:39.962515+0000","last_active":"2020-07-18T03:25:45.532300+0000","last_peered":"2020-07-18T03:25:45.532300+0000","last_clean":"2020-07-18T03:25:45.532300+0000","last_became_active":"2020-07-18T03:25:39.962206+0000","last_became_peered":"2020-07-18T03:25:39.962206+0000","last_unstale":"2020-07-18T03:25:45.532300+0000","last_undegraded":"2020-07-18T03:25:45.532300+0000","last_fullsized":"2020-07-18T03:25:45.532300+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.595535+0000","last_change":"2020-07-18T03:25:43.018396+0000","last_active":"2020-07-18T03:25:48.595535+0000","last_peered":"2020-07-18T03:25:48.595535+0000","last_clean":"2020-07-18T03:25:48.595535+0000","last_became_active":"2020-07-18T03:25:43.018006+0000","last_became_peered":"2020-07-18T03:25:43.018006+0000","last_unstale":"2020-07-18T03:25:48.595535+0000","last_undegraded":"2020-07-18T03:25:48.595535+0000","last_fullsized":"2020-07-18T03:25:48.595535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":105427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":103191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]}]}]}} 2020-07-18T03:25:55.971 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-07-18T03:25:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 Processor -- start 2020-07-18T03:25:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 -- start start 2020-07-18T03:25:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6c80b5370 0x7fd6c805fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6c80b41c0 0x7fd6c80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80b3ac0 con 0x7fd6c80b4620 2020-07-18T03:25:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c80b5a10 con 0x7fd6c80b5370 2020-07-18T03:25:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.087+0000 7fd6d0343700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80b0040 con 0x7fd6c80b41c0 2020-07-18T03:25:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce0df700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6c80b5370 0x7fd6c805fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce8e0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd6c80b4620 0x7fd6c805f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39882/0 (socket says 172.21.15.47:39882) 2020-07-18T03:25:56.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce8e0700 1 -- 172.21.15.47:0/527558710 learned_addr learned my addr 172.21.15.47:0/527558710 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6cd8de700 1 --2- 172.21.15.47:0/527558710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6c80b41c0 0x7fd6c80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce0df700 1 -- 172.21.15.47:0/527558710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6c80b41c0 msgr2=0x7fd6c80ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce0df700 1 --2- 172.21.15.47:0/527558710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6c80b41c0 0x7fd6c80ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce0df700 1 -- 172.21.15.47:0/527558710 >> v1:172.21.15.47:6789/0 conn(0x7fd6c80b4620 legacy=0x7fd6c805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6cd0dd700 1 -- 172.21.15.47:0/527558710 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844478491 0 0) 0x7fd6c80b3ac0 con 0x7fd6c80b4620 2020-07-18T03:25:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce0df700 1 -- 172.21.15.47:0/527558710 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80b3d20 con 0x7fd6c80b5370 2020-07-18T03:25:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6ce0df700 1 --2- 172.21.15.47:0/527558710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c80b5370 0x7fd6c805fff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd6b8002f30 tx=0x7fd6b8006dd0).ready entity=mon.1 client_cookie=e8831bd34d26c4a0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6cd0dd700 1 -- 172.21.15.47:0/527558710 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6b8009760 con 0x7fd6c80b5370 2020-07-18T03:25:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6cd0dd700 1 -- 172.21.15.47:0/527558710 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b80098c0 con 0x7fd6c80b5370 2020-07-18T03:25:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6cd0dd700 1 -- 172.21.15.47:0/527558710 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6b8009c20 con 0x7fd6c80b5370 2020-07-18T03:25:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 -- 172.21.15.47:0/527558710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c80b5370 msgr2=0x7fd6c805fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 --2- 172.21.15.47:0/527558710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c80b5370 0x7fd6c805fff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd6b8002f30 tx=0x7fd6b8006dd0).stop 2020-07-18T03:25:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 -- 172.21.15.47:0/527558710 shutdown_connections 2020-07-18T03:25:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 --2- 172.21.15.47:0/527558710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c80b5370 0x7fd6c805fff0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 --2- 172.21.15.47:0/527558710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6c80b41c0 0x7fd6c80ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 -- 172.21.15.47:0/527558710 >> 172.21.15.47:0/527558710 conn(0x7fd6c80b9340 msgr2=0x7fd6c80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.088+0000 7fd6d0343700 1 -- 172.21.15.47:0/527558710 shutdown_connections 2020-07-18T03:25:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.089+0000 7fd6d0343700 1 -- 172.21.15.47:0/527558710 wait complete. 2020-07-18T03:25:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.090+0000 7fd6d0343700 1 Processor -- start 2020-07-18T03:25:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.090+0000 7fd6d0343700 1 -- start start 2020-07-18T03:25:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd6c80b41c0 0x7fd6c8145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6c8146cd0 0x7fd6c8147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c8149110 con 0x7fd6c8144090 2020-07-18T03:25:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c8147630 con 0x7fd6c80b41c0 2020-07-18T03:25:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c81493b0 con 0x7fd6c8146cd0 2020-07-18T03:25:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce8e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6c8146cd0 0x7fd6c8147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6cd8de700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd6c8144090 0x7fd6c8140f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39890/0 (socket says 172.21.15.47:39890) 2020-07-18T03:25:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6cd8de700 1 -- 172.21.15.47:0/1149194638 learned_addr learned my addr 172.21.15.47:0/1149194638 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce0df700 1 --2- 172.21.15.47:0/1149194638 >> v2:172.21.15.18:3300/0 conn(0x7fd6c80b41c0 0x7fd6c8145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce8e0700 1 -- 172.21.15.47:0/1149194638 >> v2:172.21.15.18:3300/0 conn(0x7fd6c80b41c0 msgr2=0x7fd6c8145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce8e0700 1 --2- 172.21.15.47:0/1149194638 >> v2:172.21.15.18:3300/0 conn(0x7fd6c80b41c0 0x7fd6c8145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce8e0700 1 -- 172.21.15.47:0/1149194638 >> v1:172.21.15.47:6789/0 conn(0x7fd6c8144090 legacy=0x7fd6c8140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501218006 0 0) 0x7fd6c8149110 con 0x7fd6c8144090 2020-07-18T03:25:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce8e0700 1 -- 172.21.15.47:0/1149194638 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8149650 con 0x7fd6c8146cd0 2020-07-18T03:25:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6ce8e0700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c8146cd0 0x7fd6c8147140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fd6c0000e90 tx=0x7fd6c00078c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c0007320 con 0x7fd6c8146cd0 2020-07-18T03:25:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0007480 con 0x7fd6c8146cd0 2020-07-18T03:25:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c0005220 con 0x7fd6c8146cd0 2020-07-18T03:25:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c81498f0 con 0x7fd6c8146cd0 2020-07-18T03:25:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.091+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c8149d90 con 0x7fd6c8146cd0 2020-07-18T03:25:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.092+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c8149a80 con 0x7fd6c8146cd0 2020-07-18T03:25:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.092+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd6c00055f0 con 0x7fd6c8146cd0 2020-07-18T03:25:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.093+0000 7fd6b67fc700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6a004fe90 0x7fd6a0052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.093+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd6c00675a0 con 0x7fd6c8146cd0 2020-07-18T03:25:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.093+0000 7fd6ce0df700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6a004fe90 0x7fd6a0052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.093+0000 7fd6ce0df700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6a004fe90 0x7fd6a0052350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd6b8016000 tx=0x7fd6b8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.096+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd6c003fcd0 con 0x7fd6c8146cd0 2020-07-18T03:25:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.246+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 --> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6c8057450 con 0x7fd6a004fe90 2020-07-18T03:25:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.249+0000 7fd6b67fc700 1 -- 172.21.15.47:0/1149194638 <== mgr.4108 v2:172.21.15.47:6800/23227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82297 (secure 0 0 0) 0x7fd6c8057450 con 0x7fd6a004fe90 2020-07-18T03:25:56.251 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.250+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6a004fe90 msgr2=0x7fd6a0052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.250+0000 7fd6d0343700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6a004fe90 0x7fd6a0052350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd6b8016000 tx=0x7fd6b8015000).stop 2020-07-18T03:25:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.250+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c8146cd0 msgr2=0x7fd6c8147140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.250+0000 7fd6d0343700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c8146cd0 0x7fd6c8147140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fd6c0000e90 tx=0x7fd6c00078c0).stop 2020-07-18T03:25:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 shutdown_connections 2020-07-18T03:25:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c8146cd0 0x7fd6c8147140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 --2- 172.21.15.47:0/1149194638 >> v2:172.21.15.18:3300/0 conn(0x7fd6c80b41c0 0x7fd6c8145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 --2- 172.21.15.47:0/1149194638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6a004fe90 0x7fd6a0052350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 >> 172.21.15.47:0/1149194638 conn(0x7fd6c80b9340 msgr2=0x7fd6c805fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 shutdown_connections 2020-07-18T03:25:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.251+0000 7fd6d0343700 1 -- 172.21.15.47:0/1149194638 wait complete. 2020-07-18T03:25:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2020-07-18T03:25:56.262 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2020-07-18T03:25:55.687192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":829864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001873"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.560167+0000","last_change":"2020-07-18T03:25:45.559076+0000","last_active":"2020-07-18T03:25:45.560167+0000","last_peered":"2020-07-18T03:25:45.560167+0000","last_clean":"2020-07-18T03:25:45.560167+0000","last_became_active":"2020-07-18T03:25:45.558416+0000","last_became_peered":"2020-07-18T03:25:45.558416+0000","last_unstale":"2020-07-18T03:25:45.560167+0000","last_undegraded":"2020-07-18T03:25:45.560167+0000","last_fullsized":"2020-07-18T03:25:45.560167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.534879+0000","last_change":"2020-07-18T03:25:39.516626+0000","last_active":"2020-07-18T03:25:45.534879+0000","last_peered":"2020-07-18T03:25:45.534879+0000","last_clean":"2020-07-18T03:25:45.534879+0000","last_became_active":"2020-07-18T03:25:39.516260+0000","last_became_peered":"2020-07-18T03:25:39.516260+0000","last_unstale":"2020-07-18T03:25:45.534879+0000","last_undegraded":"2020-07-18T03:25:45.534879+0000","last_fullsized":"2020-07-18T03:25:45.534879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.594523+0000","last_change":"2020-07-18T03:25:42.530140+0000","last_active":"2020-07-18T03:25:48.594523+0000","last_peered":"2020-07-18T03:25:48.594523+0000","last_clean":"2020-07-18T03:25:48.594523+0000","last_became_active":"2020-07-18T03:25:42.529759+0000","last_became_peered":"2020-07-18T03:25:42.529759+0000","last_unstale":"2020-07-18T03:25:48.594523+0000","last_undegraded":"2020-07-18T03:25:48.594523+0000","last_fullsized":"2020-07-18T03:25:48.594523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.564041+0000","last_change":"2020-07-18T03:25:45.562946+0000","last_active":"2020-07-18T03:25:45.564041+0000","last_peered":"2020-07-18T03:25:45.564041+0000","last_clean":"2020-07-18T03:25:45.564041+0000","last_became_active":"2020-07-18T03:25:45.561799+0000","last_became_peered":"2020-07-18T03:25:45.561799+0000","last_unstale":"2020-07-18T03:25:45.564041+0000","last_undegraded":"2020-07-18T03:25:45.564041+0000","last_fullsized":"2020-07-18T03:25:45.564041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532126+0000","last_change":"2020-07-18T03:25:39.515578+0000","last_active":"2020-07-18T03:25:45.532126+0000","last_peered":"2020-07-18T03:25:45.532126+0000","last_clean":"2020-07-18T03:25:45.532126+0000","last_became_active":"2020-07-18T03:25:39.515122+0000","last_became_peered":"2020-07-18T03:25:39.515122+0000","last_unstale":"2020-07-18T03:25:45.532126+0000","last_undegraded":"2020-07-18T03:25:45.532126+0000","last_fullsized":"2020-07-18T03:25:45.532126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.591392+0000","last_change":"2020-07-18T03:25:42.529236+0000","last_active":"2020-07-18T03:25:48.591392+0000","last_peered":"2020-07-18T03:25:48.591392+0000","last_clean":"2020-07-18T03:25:48.591392+0000","last_became_active":"2020-07-18T03:25:42.528469+0000","last_became_peered":"2020-07-18T03:25:42.528469+0000","last_unstale":"2020-07-18T03:25:48.591392+0000","last_undegraded":"2020-07-18T03:25:48.591392+0000","last_fullsized":"2020-07-18T03:25:48.591392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.563291+0000","last_change":"2020-07-18T03:25:45.561907+0000","last_active":"2020-07-18T03:25:45.563291+0000","last_peered":"2020-07-18T03:25:45.563291+0000","last_clean":"2020-07-18T03:25:45.563291+0000","last_became_active":"2020-07-18T03:25:45.561150+0000","last_became_peered":"2020-07-18T03:25:45.561150+0000","last_unstale":"2020-07-18T03:25:45.563291+0000","last_undegraded":"2020-07-18T03:25:45.563291+0000","last_fullsized":"2020-07-18T03:25:45.563291+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.556180+0000","last_change":"2020-07-18T03:25:39.516123+0000","last_active":"2020-07-18T03:25:45.556180+0000","last_peered":"2020-07-18T03:25:45.556180+0000","last_clean":"2020-07-18T03:25:45.556180+0000","last_became_active":"2020-07-18T03:25:39.515676+0000","last_became_peered":"2020-07-18T03:25:39.515676+0000","last_unstale":"2020-07-18T03:25:45.556180+0000","last_undegraded":"2020-07-18T03:25:45.556180+0000","last_fullsized":"2020-07-18T03:25:45.556180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.598653+0000","last_change":"2020-07-18T03:25:43.018556+0000","last_active":"2020-07-18T03:25:48.598653+0000","last_peered":"2020-07-18T03:25:48.598653+0000","last_clean":"2020-07-18T03:25:48.598653+0000","last_became_active":"2020-07-18T03:25:43.018093+0000","last_became_peered":"2020-07-18T03:25:43.018093+0000","last_unstale":"2020-07-18T03:25:48.598653+0000","last_undegraded":"2020-07-18T03:25:48.598653+0000","last_fullsized":"2020-07-18T03:25:48.598653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.561085+0000","last_change":"2020-07-18T03:25:45.559375+0000","last_active":"2020-07-18T03:25:45.561085+0000","last_peered":"2020-07-18T03:25:45.561085+0000","last_clean":"2020-07-18T03:25:45.561085+0000","last_became_active":"2020-07-18T03:25:45.558097+0000","last_became_peered":"2020-07-18T03:25:45.558097+0000","last_unstale":"2020-07-18T03:25:45.561085+0000","last_undegraded":"2020-07-18T03:25:45.561085+0000","last_fullsized":"2020-07-18T03:25:45.561085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532121+0000","last_change":"2020-07-18T03:25:40.021509+0000","last_active":"2020-07-18T03:25:45.532121+0000","last_peered":"2020-07-18T03:25:45.532121+0000","last_clean":"2020-07-18T03:25:45.532121+0000","last_became_active":"2020-07-18T03:25:40.021077+0000","last_became_peered":"2020-07-18T03:25:40.021077+0000","last_unstale":"2020-07-18T03:25:45.532121+0000","last_undegraded":"2020-07-18T03:25:45.532121+0000","last_fullsized":"2020-07-18T03:25:45.532121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.556446+0000","last_change":"2020-07-18T03:25:35.015179+0000","last_active":"2020-07-18T03:25:45.556446+0000","last_peered":"2020-07-18T03:25:45.556446+0000","last_clean":"2020-07-18T03:25:45.556446+0000","last_became_active":"2020-07-18T03:25:35.014863+0000","last_became_peered":"2020-07-18T03:25:35.014863+0000","last_unstale":"2020-07-18T03:25:45.556446+0000","last_undegraded":"2020-07-18T03:25:45.556446+0000","last_fullsized":"2020-07-18T03:25:45.556446+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":5,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:27.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:27.868182+0000","last_clean_scrub_stamp":"2020-07-18T03:25:27.868182+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:45.902747+0000","last_change":"2020-07-18T03:25:42.528243+0000","last_active":"2020-07-18T03:25:45.902747+0000","last_peered":"2020-07-18T03:25:45.902747+0000","last_clean":"2020-07-18T03:25:45.902747+0000","last_became_active":"2020-07-18T03:25:42.527785+0000","last_became_peered":"2020-07-18T03:25:42.527785+0000","last_unstale":"2020-07-18T03:25:45.902747+0000","last_undegraded":"2020-07-18T03:25:45.902747+0000","last_fullsized":"2020-07-18T03:25:45.902747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.559390+0000","last_change":"2020-07-18T03:25:45.558635+0000","last_active":"2020-07-18T03:25:45.559390+0000","last_peered":"2020-07-18T03:25:45.559390+0000","last_clean":"2020-07-18T03:25:45.559390+0000","last_became_active":"2020-07-18T03:25:45.557980+0000","last_became_peered":"2020-07-18T03:25:45.557980+0000","last_unstale":"2020-07-18T03:25:45.559390+0000","last_undegraded":"2020-07-18T03:25:45.559390+0000","last_fullsized":"2020-07-18T03:25:45.559390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532128+0000","last_change":"2020-07-18T03:25:39.518438+0000","last_active":"2020-07-18T03:25:45.532128+0000","last_peered":"2020-07-18T03:25:45.532128+0000","last_clean":"2020-07-18T03:25:45.532128+0000","last_became_active":"2020-07-18T03:25:39.518013+0000","last_became_peered":"2020-07-18T03:25:39.518013+0000","last_unstale":"2020-07-18T03:25:45.532128+0000","last_undegraded":"2020-07-18T03:25:45.532128+0000","last_fullsized":"2020-07-18T03:25:45.532128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532680+0000","last_change":"2020-07-18T03:25:42.528662+0000","last_active":"2020-07-18T03:25:45.532680+0000","last_peered":"2020-07-18T03:25:45.532680+0000","last_clean":"2020-07-18T03:25:45.532680+0000","last_became_active":"2020-07-18T03:25:42.528211+0000","last_became_peered":"2020-07-18T03:25:42.528211+0000","last_unstale":"2020-07-18T03:25:45.532680+0000","last_undegraded":"2020-07-18T03:25:45.532680+0000","last_fullsized":"2020-07-18T03:25:45.532680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.560976+0000","last_change":"2020-07-18T03:25:45.558873+0000","last_active":"2020-07-18T03:25:45.560976+0000","last_peered":"2020-07-18T03:25:45.560976+0000","last_clean":"2020-07-18T03:25:45.560976+0000","last_became_active":"2020-07-18T03:25:45.558201+0000","last_became_peered":"2020-07-18T03:25:45.558201+0000","last_unstale":"2020-07-18T03:25:45.560976+0000","last_undegraded":"2020-07-18T03:25:45.560976+0000","last_fullsized":"2020-07-18T03:25:45.560976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.533031+0000","last_change":"2020-07-18T03:25:39.518082+0000","last_active":"2020-07-18T03:25:45.533031+0000","last_peered":"2020-07-18T03:25:45.533031+0000","last_clean":"2020-07-18T03:25:45.533031+0000","last_became_active":"2020-07-18T03:25:39.517664+0000","last_became_peered":"2020-07-18T03:25:39.517664+0000","last_unstale":"2020-07-18T03:25:45.533031+0000","last_undegraded":"2020-07-18T03:25:45.533031+0000","last_fullsized":"2020-07-18T03:25:45.533031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.596378+0000","last_change":"2020-07-18T03:25:42.527102+0000","last_active":"2020-07-18T03:25:48.596378+0000","last_peered":"2020-07-18T03:25:48.596378+0000","last_clean":"2020-07-18T03:25:48.596378+0000","last_became_active":"2020-07-18T03:25:42.526748+0000","last_became_peered":"2020-07-18T03:25:42.526748+0000","last_unstale":"2020-07-18T03:25:48.596378+0000","last_undegraded":"2020-07-18T03:25:48.596378+0000","last_fullsized":"2020-07-18T03:25:48.596378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.561403+0000","last_change":"2020-07-18T03:25:45.560783+0000","last_active":"2020-07-18T03:25:45.561403+0000","last_peered":"2020-07-18T03:25:45.561403+0000","last_clean":"2020-07-18T03:25:45.561403+0000","last_became_active":"2020-07-18T03:25:45.559895+0000","last_became_peered":"2020-07-18T03:25:45.559895+0000","last_unstale":"2020-07-18T03:25:45.561403+0000","last_undegraded":"2020-07-18T03:25:45.561403+0000","last_fullsized":"2020-07-18T03:25:45.561403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"24'6","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.595088+0000","last_change":"2020-07-18T03:25:42.525475+0000","last_active":"2020-07-18T03:25:48.595088+0000","last_peered":"2020-07-18T03:25:48.595088+0000","last_clean":"2020-07-18T03:25:48.595088+0000","last_became_active":"2020-07-18T03:25:42.525035+0000","last_became_peered":"2020-07-18T03:25:42.525035+0000","last_unstale":"2020-07-18T03:25:48.595088+0000","last_undegraded":"2020-07-18T03:25:48.595088+0000","last_fullsized":"2020-07-18T03:25:48.595088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.531914+0000","last_change":"2020-07-18T03:25:39.962705+0000","last_active":"2020-07-18T03:25:45.531914+0000","last_peered":"2020-07-18T03:25:45.531914+0000","last_clean":"2020-07-18T03:25:45.531914+0000","last_became_active":"2020-07-18T03:25:39.962375+0000","last_became_peered":"2020-07-18T03:25:39.962375+0000","last_unstale":"2020-07-18T03:25:45.531914+0000","last_undegraded":"2020-07-18T03:25:45.531914+0000","last_fullsized":"2020-07-18T03:25:45.531914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.560278+0000","last_change":"2020-07-18T03:25:45.558814+0000","last_active":"2020-07-18T03:25:45.560278+0000","last_peered":"2020-07-18T03:25:45.560278+0000","last_clean":"2020-07-18T03:25:45.560278+0000","last_became_active":"2020-07-18T03:25:45.558190+0000","last_became_peered":"2020-07-18T03:25:45.558190+0000","last_unstale":"2020-07-18T03:25:45.560278+0000","last_undegraded":"2020-07-18T03:25:45.560278+0000","last_fullsized":"2020-07-18T03:25:45.560278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:44.526992+0000","last_clean_scrub_stamp":"2020-07-18T03:25:44.526992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-07-18T03:25:45.532300+0000","last_change":"2020-07-18T03:25:39.962515+0000","last_active":"2020-07-18T03:25:45.532300+0000","last_peered":"2020-07-18T03:25:45.532300+0000","last_clean":"2020-07-18T03:25:45.532300+0000","last_became_active":"2020-07-18T03:25:39.962206+0000","last_became_peered":"2020-07-18T03:25:39.962206+0000","last_unstale":"2020-07-18T03:25:45.532300+0000","last_undegraded":"2020-07-18T03:25:45.532300+0000","last_fullsized":"2020-07-18T03:25:45.532300+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:38.504811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:38.504811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-07-18T03:25:48.595535+0000","last_change":"2020-07-18T03:25:43.018396+0000","last_active":"2020-07-18T03:25:48.595535+0000","last_peered":"2020-07-18T03:25:48.595535+0000","last_clean":"2020-07-18T03:25:48.595535+0000","last_became_active":"2020-07-18T03:25:43.018006+0000","last_became_peered":"2020-07-18T03:25:43.018006+0000","last_unstale":"2020-07-18T03:25:48.595535+0000","last_undegraded":"2020-07-18T03:25:48.595535+0000","last_fullsized":"2020-07-18T03:25:48.595535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-07-18T03:25:41.516811+0000","last_clean_scrub_stamp":"2020-07-18T03:25:41.516811+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":105427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":103191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":103157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]}]}]}} 2020-07-18T03:25:56.265 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-07-18T03:25:56.266 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-07-18T03:25:56.266 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-07-18T03:25:56.267 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-07-18T03:25:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.379+0000 7f0a9bfff700 1 Processor -- start 2020-07-18T03:25:56.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.379+0000 7f0a9bfff700 1 -- start start 2020-07-18T03:25:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c066560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a9c0b5160 0x7f0a9c1437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0b37b0 con 0x7f0a9c143350 2020-07-18T03:25:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c063930 con 0x7f0a9c0b5160 2020-07-18T03:25:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c065e70 con 0x7f0a9c063550 2020-07-18T03:25:56.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a9c0b5160 0x7f0a9c1437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9a7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a9c143350 0x7f0a9c064bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39896/0 (socket says 172.21.15.47:39896) 2020-07-18T03:25:56.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9a7fc700 1 -- 172.21.15.47:0/2168007470 learned_addr learned my addr 172.21.15.47:0/2168007470 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:56.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9affd700 1 --2- 172.21.15.47:0/2168007470 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c066560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/2168007470 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 msgr2=0x7f0a9c066560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:56.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9b7fe700 1 --2- 172.21.15.47:0/2168007470 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c066560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/2168007470 >> v1:172.21.15.47:6789/0 conn(0x7f0a9c143350 legacy=0x7f0a9c064bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/2168007470 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0b3a10 con 0x7f0a9c0b5160 2020-07-18T03:25:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a99ffb700 1 -- 172.21.15.47:0/2168007470 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631480494 0 0) 0x7f0a9c0b37b0 con 0x7f0a9c143350 2020-07-18T03:25:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9b7fe700 1 --2- 172.21.15.47:0/2168007470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c0b5160 0x7f0a9c1437b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f0a940032b0 tx=0x7f0a940091d0).ready entity=mon.1 client_cookie=b7954089f12e0f64 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a99ffb700 1 -- 172.21.15.47:0/2168007470 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9400ba50 con 0x7f0a9c0b5160 2020-07-18T03:25:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a99ffb700 1 -- 172.21.15.47:0/2168007470 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9400bbb0 con 0x7f0a9c0b5160 2020-07-18T03:25:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a99ffb700 1 -- 172.21.15.47:0/2168007470 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9400bd20 con 0x7f0a9c0b5160 2020-07-18T03:25:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 -- 172.21.15.47:0/2168007470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c0b5160 msgr2=0x7f0a9c1437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/2168007470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c0b5160 0x7f0a9c1437b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f0a940032b0 tx=0x7f0a940091d0).stop 2020-07-18T03:25:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 -- 172.21.15.47:0/2168007470 shutdown_connections 2020-07-18T03:25:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/2168007470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c0b5160 0x7f0a9c1437b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/2168007470 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c066560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.380+0000 7f0a9bfff700 1 -- 172.21.15.47:0/2168007470 >> 172.21.15.47:0/2168007470 conn(0x7f0a9c0b91b0 msgr2=0x7f0a9c0b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.381+0000 7f0a9bfff700 1 -- 172.21.15.47:0/2168007470 shutdown_connections 2020-07-18T03:25:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.381+0000 7f0a9bfff700 1 -- 172.21.15.47:0/2168007470 wait complete. 2020-07-18T03:25:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.382+0000 7f0a9bfff700 1 Processor -- start 2020-07-18T03:25:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.382+0000 7f0a9bfff700 1 -- start start 2020-07-18T03:25:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c1450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a9c1369d0 0x7f0a9c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0af150 con 0x7f0a9c143350 2020-07-18T03:25:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0aef60 con 0x7f0a9c1369d0 2020-07-18T03:25:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0af3f0 con 0x7f0a9c063550 2020-07-18T03:25:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a9c1369d0 0x7f0a9c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a9c1369d0 0x7f0a9c138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58818/0 (socket says 172.21.15.47:58818) 2020-07-18T03:25:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/1291392687 learned_addr learned my addr 172.21.15.47:0/1291392687 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9affd700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c1450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/1291392687 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 msgr2=0x7f0a9c1450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c1450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/1291392687 >> v1:172.21.15.18:6789/0 conn(0x7f0a9c143350 legacy=0x7f0a9c0afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 -- 172.21.15.47:0/1291392687 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0af690 con 0x7f0a9c1369d0 2020-07-18T03:25:56.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9b7fe700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c1369d0 0x7f0a9c138af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f0a940032b0 tx=0x7f0a94016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a940107a0 con 0x7f0a9c1369d0 2020-07-18T03:25:56.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a94010900 con 0x7f0a9c1369d0 2020-07-18T03:25:56.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c138fe0 con 0x7f0a9c1369d0 2020-07-18T03:25:56.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a94010c60 con 0x7f0a9c1369d0 2020-07-18T03:25:56.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.383+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c135ff0 con 0x7f0a9c1369d0 2020-07-18T03:25:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.384+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c0690d0 con 0x7f0a9c1369d0 2020-07-18T03:25:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.384+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0a94027030 con 0x7f0a9c1369d0 2020-07-18T03:25:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.385+0000 7f0a92ffd700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a78050040 0x7f0a78052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.385+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0a94068f20 con 0x7f0a9c1369d0 2020-07-18T03:25:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.385+0000 7f0a9affd700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a78050040 0x7f0a78052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.385+0000 7f0a9affd700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a78050040 0x7f0a78052500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0a8400c000 tx=0x7f0a84008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.388+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0a94041ca0 con 0x7f0a9c1369d0 2020-07-18T03:25:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.607+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0a9c069f30 con 0x7f0a9c1369d0 2020-07-18T03:25:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.607+0000 7f0a92ffd700 1 -- 172.21.15.47:0/1291392687 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f0a94044cd0 con 0x7f0a9c1369d0 2020-07-18T03:25:56.610 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a78050040 msgr2=0x7f0a78052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a78050040 0x7f0a78052500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0a8400c000 tx=0x7f0a84008040).stop 2020-07-18T03:25:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c1369d0 msgr2=0x7f0a9c138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c1369d0 0x7f0a9c138af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f0a940032b0 tx=0x7f0a94016000).stop 2020-07-18T03:25:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 shutdown_connections 2020-07-18T03:25:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a78050040 0x7f0a78052500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c1369d0 0x7f0a9c138af0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 --2- 172.21.15.47:0/1291392687 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a9c063550 0x7f0a9c1450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 >> 172.21.15.47:0/1291392687 conn(0x7f0a9c0b91b0 msgr2=0x7f0a9c0b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 shutdown_connections 2020-07-18T03:25:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.609+0000 7f0a9bfff700 1 -- 172.21.15.47:0/1291392687 wait complete. 2020-07-18T03:25:56.622 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-07-18T03:25:56.623 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-07-18T03:25:56.623 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.736+0000 7f3e5862d700 1 Processor -- start 2020-07-18T03:25:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.736+0000 7f3e5862d700 1 -- start start 2020-07-18T03:25:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e5862d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 0x7f3e500b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e5862d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e50147330 con 0x7f3e50147a00 2020-07-18T03:25:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e5862d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e500603c0 con 0x7f3e501443f0 2020-07-18T03:25:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e5862d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500b57e0 con 0x7f3e500ad100 2020-07-18T03:25:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e56bca700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3e501443f0 0x7f3e5005fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39912/0 (socket says 172.21.15.47:39912) 2020-07-18T03:25:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e56bca700 1 -- 172.21.15.47:0/3721051679 learned_addr learned my addr 172.21.15.47:0/3721051679 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e563c9700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 0x7f3e500b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213478074 0 0) 0x7f3e500603c0 con 0x7f3e501443f0 2020-07-18T03:25:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.737+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e38003240 con 0x7f3e501443f0 2020-07-18T03:25:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3771712385 0 0) 0x7f3e38003240 con 0x7f3e501443f0 2020-07-18T03:25:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 msgr2=0x7f3e500b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 0x7f3e500b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 >> v1:172.21.15.18:6789/0 conn(0x7f3e50147a00 legacy=0x7f3e5005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500b5a80 con 0x7f3e501443f0 2020-07-18T03:25:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f3e48001a50 con 0x7f3e501443f0 2020-07-18T03:25:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 >> v1:172.21.15.47:6789/0 conn(0x7f3e501443f0 legacy=0x7f3e5005fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e38003730 0x7f3e38003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 0x7f3e380044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e380049d0 0x7f3e38004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500b57e0 con 0x7f3e38003730 2020-07-18T03:25:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38004140 con 0x7f3e380049d0 2020-07-18T03:25:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38004300 con 0x7f3e500ad100 2020-07-18T03:25:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e380049d0 0x7f3e38004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e563c9700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e38003730 0x7f3e38003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e55bc8700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 0x7f3e380044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 -- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 msgr2=0x7f3e380044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3e500ad100 0x7f3e380044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 -- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e38003730 msgr2=0x7f3e38003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e38003730 0x7f3e38003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 -- 172.21.15.47:0/3721051679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38003240 con 0x7f3e380049d0 2020-07-18T03:25:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e380049d0 0x7f3e38004e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f3e48001760 tx=0x7f3e480021d0).ready entity=mon.1 client_cookie=90ef1ab73fcbffc1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e48005b80 con 0x7f3e380049d0 2020-07-18T03:25:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48005ce0 con 0x7f3e380049d0 2020-07-18T03:25:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e553c7700 1 -- 172.21.15.47:0/3721051679 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e480072c0 con 0x7f3e380049d0 2020-07-18T03:25:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e5862d700 1 -- 172.21.15.47:0/3721051679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e380049d0 msgr2=0x7f3e38004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e5862d700 1 --2- 172.21.15.47:0/3721051679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e380049d0 0x7f3e38004e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f3e48001760 tx=0x7f3e480021d0).stop 2020-07-18T03:25:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e56bca700 1 -- 172.21.15.47:0/3721051679 reap_dead start 2020-07-18T03:25:56.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e5862d700 1 -- 172.21.15.47:0/3721051679 shutdown_connections 2020-07-18T03:25:56.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e5862d700 1 -- 172.21.15.47:0/3721051679 >> 172.21.15.47:0/3721051679 conn(0x7f3e500b9340 msgr2=0x7f3e500b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.738+0000 7f3e5862d700 1 -- 172.21.15.47:0/3721051679 shutdown_connections 2020-07-18T03:25:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.739+0000 7f3e5862d700 1 -- 172.21.15.47:0/3721051679 wait complete. 2020-07-18T03:25:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.740+0000 7f3e5862d700 1 Processor -- start 2020-07-18T03:25:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.740+0000 7f3e5862d700 1 -- start start 2020-07-18T03:25:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3e5013efc0 0x7f3e500b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3e501310d0 0x7f3e50140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5013fee0 con 0x7f3e5013f3a0 2020-07-18T03:25:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5013fcd0 con 0x7f3e5013efc0 2020-07-18T03:25:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500b1660 con 0x7f3e501310d0 2020-07-18T03:25:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e55bc8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3e5013f3a0 0x7f3e501309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39924/0 (socket says 172.21.15.47:39924) 2020-07-18T03:25:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e55bc8700 1 -- 172.21.15.47:0/1033121159 learned_addr learned my addr 172.21.15.47:0/1033121159 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e56bca700 1 --2- 172.21.15.47:0/1033121159 >> v2:172.21.15.47:3300/0 conn(0x7f3e501310d0 0x7f3e50140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e563c9700 1 --2- 172.21.15.47:0/1033121159 >> v2:172.21.15.18:3300/0 conn(0x7f3e5013efc0 0x7f3e500b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191060608 0 0) 0x7f3e5013fee0 con 0x7f3e5013f3a0 2020-07-18T03:25:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e28003240 con 0x7f3e5013f3a0 2020-07-18T03:25:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e56bca700 1 -- 172.21.15.47:0/1033121159 >> v2:172.21.15.18:3300/0 conn(0x7f3e5013efc0 msgr2=0x7f3e500b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e56bca700 1 --2- 172.21.15.47:0/1033121159 >> v2:172.21.15.18:3300/0 conn(0x7f3e5013efc0 0x7f3e500b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e56bca700 1 -- 172.21.15.47:0/1033121159 >> v1:172.21.15.47:6789/0 conn(0x7f3e5013f3a0 legacy=0x7f3e501309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e56bca700 1 -- 172.21.15.47:0/1033121159 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500b2a90 con 0x7f3e501310d0 2020-07-18T03:25:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e56bca700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e501310d0 0x7f3e50140380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f3e50068140 tx=0x7f3e48017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e48005b80 con 0x7f3e501310d0 2020-07-18T03:25:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48007800 con 0x7f3e501310d0 2020-07-18T03:25:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e500b2ce0 con 0x7f3e501310d0 2020-07-18T03:25:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e48010d50 con 0x7f3e501310d0 2020-07-18T03:25:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.741+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e50140ab0 con 0x7f3e501310d0 2020-07-18T03:25:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.742+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50057450 con 0x7f3e501310d0 2020-07-18T03:25:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.742+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3e48011470 con 0x7f3e501310d0 2020-07-18T03:25:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.743+0000 7f3e427fc700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3e280503b0 0x7f3e28052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.743+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3e4806bda0 con 0x7f3e501310d0 2020-07-18T03:25:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.743+0000 7f3e563c9700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3e280503b0 0x7f3e28052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.743+0000 7f3e563c9700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3e280503b0 0x7f3e28052870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3e44008000 tx=0x7f3e44008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.746+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3e48044590 con 0x7f3e501310d0 2020-07-18T03:25:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.903+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e5006a100 con 0x7f3e501310d0 2020-07-18T03:25:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.904+0000 7f3e427fc700 1 -- 172.21.15.47:0/1033121159 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f3e48047af0 con 0x7f3e501310d0 2020-07-18T03:25:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:56.907 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3e280503b0 msgr2=0x7f3e28052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3e280503b0 0x7f3e28052870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3e44008000 tx=0x7f3e44008040).stop 2020-07-18T03:25:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e501310d0 msgr2=0x7f3e50140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e501310d0 0x7f3e50140380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f3e50068140 tx=0x7f3e48017000).stop 2020-07-18T03:25:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 shutdown_connections 2020-07-18T03:25:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e501310d0 0x7f3e50140380 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 --2- 172.21.15.47:0/1033121159 >> v2:172.21.15.18:3300/0 conn(0x7f3e5013efc0 0x7f3e500b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 --2- 172.21.15.47:0/1033121159 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3e280503b0 0x7f3e28052870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:56.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 >> 172.21.15.47:0/1033121159 conn(0x7f3e500b9340 msgr2=0x7f3e50146ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:56.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 shutdown_connections 2020-07-18T03:25:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:56.905+0000 7f3e5862d700 1 -- 172.21.15.47:0/1033121159 wait complete. 2020-07-18T03:25:56.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:25:56.935 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:25:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.051+0000 7f2e5c171700 1 Processor -- start 2020-07-18T03:25:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.051+0000 7f2e5c171700 1 -- start start 2020-07-18T03:25:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5c171700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e540ad330 0x7f2e54143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5c171700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e540b59b0 0x7f2e540b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5c171700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e54147de0 con 0x7f2e540b6300 2020-07-18T03:25:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5c171700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e540ae000 con 0x7f2e540b59b0 2020-07-18T03:25:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5c171700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54148010 con 0x7f2e540ad330 2020-07-18T03:25:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5970c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e540b59b0 0x7f2e540b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5970c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e540b59b0 0x7f2e540b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58842/0 (socket says 172.21.15.47:58842) 2020-07-18T03:25:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e5970c700 1 -- 172.21.15.47:0/875286052 learned_addr learned my addr 172.21.15.47:0/875286052 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.052+0000 7f2e59f0d700 1 --2- 172.21.15.47:0/875286052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e540ad330 0x7f2e54143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5970c700 1 -- 172.21.15.47:0/875286052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e540ad330 msgr2=0x7f2e54143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5970c700 1 --2- 172.21.15.47:0/875286052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e540ad330 0x7f2e54143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5970c700 1 -- 172.21.15.47:0/875286052 >> v1:172.21.15.18:6789/0 conn(0x7f2e540b6300 legacy=0x7f2e540b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5970c700 1 -- 172.21.15.47:0/875286052 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e540ad710 con 0x7f2e540b59b0 2020-07-18T03:25:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5970c700 1 --2- 172.21.15.47:0/875286052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b59b0 0x7f2e540b5e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f2e50002f30 tx=0x7f2e50006e90).ready entity=mon.1 client_cookie=2861697f0525286e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e58f0b700 1 -- 172.21.15.47:0/875286052 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e50009850 con 0x7f2e540b59b0 2020-07-18T03:25:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e58f0b700 1 -- 172.21.15.47:0/875286052 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500099b0 con 0x7f2e540b59b0 2020-07-18T03:25:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e58f0b700 1 -- 172.21.15.47:0/875286052 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e50009d10 con 0x7f2e540b59b0 2020-07-18T03:25:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 -- 172.21.15.47:0/875286052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b59b0 msgr2=0x7f2e540b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 --2- 172.21.15.47:0/875286052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b59b0 0x7f2e540b5e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f2e50002f30 tx=0x7f2e50006e90).stop 2020-07-18T03:25:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 -- 172.21.15.47:0/875286052 shutdown_connections 2020-07-18T03:25:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 --2- 172.21.15.47:0/875286052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e540ad330 0x7f2e54143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 --2- 172.21.15.47:0/875286052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b59b0 0x7f2e540b5e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 -- 172.21.15.47:0/875286052 >> 172.21.15.47:0/875286052 conn(0x7f2e5405f5a0 msgr2=0x7f2e5405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 -- 172.21.15.47:0/875286052 shutdown_connections 2020-07-18T03:25:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.053+0000 7f2e5c171700 1 -- 172.21.15.47:0/875286052 wait complete. 2020-07-18T03:25:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.054+0000 7f2e5c171700 1 Processor -- start 2020-07-18T03:25:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.054+0000 7f2e5c171700 1 -- start start 2020-07-18T03:25:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5c171700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2e540ad330 0x7f2e54065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5c171700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e540b6300 0x7f2e5413a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5c171700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e54061d40 0x7f2e54136c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5c171700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e54139e70 con 0x7f2e540ad330 2020-07-18T03:25:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5c171700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5413a050 con 0x7f2e540b6300 2020-07-18T03:25:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5c171700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5413a2f0 con 0x7f2e54061d40 2020-07-18T03:25:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e540b6300 0x7f2e5413a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e540b6300 0x7f2e5413a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58848/0 (socket says 172.21.15.47:58848) 2020-07-18T03:25:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 -- 172.21.15.47:0/96725581 learned_addr learned my addr 172.21.15.47:0/96725581 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e59f0d700 1 --2- 172.21.15.47:0/96725581 >> v2:172.21.15.18:3300/0 conn(0x7f2e540ad330 0x7f2e54065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5a70e700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e54061d40 0x7f2e54136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 -- 172.21.15.47:0/96725581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e54061d40 msgr2=0x7f2e54136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e54061d40 0x7f2e54136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 -- 172.21.15.47:0/96725581 >> v2:172.21.15.18:3300/0 conn(0x7f2e540ad330 msgr2=0x7f2e54065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 --2- 172.21.15.47:0/96725581 >> v2:172.21.15.18:3300/0 conn(0x7f2e540ad330 0x7f2e54065bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.055+0000 7f2e5970c700 1 -- 172.21.15.47:0/96725581 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54139a30 con 0x7f2e540b6300 2020-07-18T03:25:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e59f0d700 1 --2- 172.21.15.47:0/96725581 >> v2:172.21.15.18:3300/0 conn(0x7f2e540ad330 0x7f2e54065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e5970c700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b6300 0x7f2e5413a790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f2e50002f30 tx=0x7f2e50015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e50009d90 con 0x7f2e540b6300 2020-07-18T03:25:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50011e40 con 0x7f2e540b6300 2020-07-18T03:25:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e5001d7c0 con 0x7f2e540b6300 2020-07-18T03:25:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5413cd30 con 0x7f2e540b6300 2020-07-18T03:25:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.056+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e54137110 con 0x7f2e540b6300 2020-07-18T03:25:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.057+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e54139bc0 con 0x7f2e540b6300 2020-07-18T03:25:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.057+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2e50025030 con 0x7f2e540b6300 2020-07-18T03:25:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.057+0000 7f2e41ffb700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e2c050000 0x7f2e2c0524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.057+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2e50069350 con 0x7f2e540b6300 2020-07-18T03:25:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.057+0000 7f2e59f0d700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e2c050000 0x7f2e2c0524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.058+0000 7f2e59f0d700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e2c050000 0x7f2e2c0524c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f2e44008000 tx=0x7f2e44008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.061+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2e50041ae0 con 0x7f2e540b6300 2020-07-18T03:25:57.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.218+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2e5413a460 con 0x7f2e540b6300 2020-07-18T03:25:57.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.218+0000 7f2e41ffb700 1 -- 172.21.15.47:0/96725581 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2e50045100 con 0x7f2e540b6300 2020-07-18T03:25:57.221 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:25:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.219+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e2c050000 msgr2=0x7f2e2c0524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.219+0000 7f2e5c171700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e2c050000 0x7f2e2c0524c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f2e44008000 tx=0x7f2e44008040).stop 2020-07-18T03:25:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.219+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b6300 msgr2=0x7f2e5413a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.219+0000 7f2e5c171700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b6300 0x7f2e5413a790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f2e50002f30 tx=0x7f2e50015000).stop 2020-07-18T03:25:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 shutdown_connections 2020-07-18T03:25:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2e2c050000 0x7f2e2c0524c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2e54061d40 0x7f2e54136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 --2- 172.21.15.47:0/96725581 >> v2:172.21.15.18:3300/0 conn(0x7f2e540ad330 0x7f2e54065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 --2- 172.21.15.47:0/96725581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e540b6300 0x7f2e5413a790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 >> 172.21.15.47:0/96725581 conn(0x7f2e5405f5a0 msgr2=0x7f2e5413d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 shutdown_connections 2020-07-18T03:25:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.220+0000 7f2e5c171700 1 -- 172.21.15.47:0/96725581 wait complete. 2020-07-18T03:25:57.232 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:25:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.352+0000 7f5cc2132700 1 Processor -- start 2020-07-18T03:25:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.352+0000 7f5cc2132700 1 -- start start 2020-07-18T03:25:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cc2132700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cc2132700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5cbc142430 0x7f5cbc148b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cc2132700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc061db0 con 0x7f5cbc141fb0 2020-07-18T03:25:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cc2132700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc0b14c0 con 0x7f5cbc142430 2020-07-18T03:25:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cc2132700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc1428c0 con 0x7f5cbc144e70 2020-07-18T03:25:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cbbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5cbc142430 0x7f5cbc148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cbaffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5cbc141fb0 0x7f5cbc148440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39944/0 (socket says 172.21.15.47:39944) 2020-07-18T03:25:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cbaffd700 1 -- 172.21.15.47:0/997000091 learned_addr learned my addr 172.21.15.47:0/997000091 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.353+0000 7f5cbb7fe700 1 --2- 172.21.15.47:0/997000091 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cbbfff700 1 -- 172.21.15.47:0/997000091 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 msgr2=0x7f5cbc0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cbbfff700 1 --2- 172.21.15.47:0/997000091 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc0b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cbbfff700 1 -- 172.21.15.47:0/997000091 >> v1:172.21.15.47:6789/0 conn(0x7f5cbc141fb0 legacy=0x7f5cbc148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cbbfff700 1 -- 172.21.15.47:0/997000091 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc062050 con 0x7f5cbc142430 2020-07-18T03:25:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cba7fc700 1 -- 172.21.15.47:0/997000091 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556812222 0 0) 0x7f5cbc061db0 con 0x7f5cbc141fb0 2020-07-18T03:25:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cbbfff700 1 --2- 172.21.15.47:0/997000091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc142430 0x7f5cbc148b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5cbc142a00 tx=0x7f5cb40091d0).ready entity=mon.1 client_cookie=1aecb61f699f20e0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cba7fc700 1 -- 172.21.15.47:0/997000091 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cb400b990 con 0x7f5cbc142430 2020-07-18T03:25:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cba7fc700 1 -- 172.21.15.47:0/997000091 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb400baf0 con 0x7f5cbc142430 2020-07-18T03:25:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cba7fc700 1 -- 172.21.15.47:0/997000091 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cb400bc60 con 0x7f5cbc142430 2020-07-18T03:25:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 -- 172.21.15.47:0/997000091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc142430 msgr2=0x7f5cbc148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 --2- 172.21.15.47:0/997000091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc142430 0x7f5cbc148b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5cbc142a00 tx=0x7f5cb40091d0).stop 2020-07-18T03:25:57.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 -- 172.21.15.47:0/997000091 shutdown_connections 2020-07-18T03:25:57.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 --2- 172.21.15.47:0/997000091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc142430 0x7f5cbc148b50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 --2- 172.21.15.47:0/997000091 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 -- 172.21.15.47:0/997000091 >> 172.21.15.47:0/997000091 conn(0x7f5cbc05f5a0 msgr2=0x7f5cbc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 -- 172.21.15.47:0/997000091 shutdown_connections 2020-07-18T03:25:57.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.354+0000 7f5cc2132700 1 -- 172.21.15.47:0/997000091 wait complete. 2020-07-18T03:25:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 Processor -- start 2020-07-18T03:25:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 -- start start 2020-07-18T03:25:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5cbc141fb0 0x7f5cbc15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc15a7e0 con 0x7f5cbc1486f0 2020-07-18T03:25:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc12d6c0 con 0x7f5cbc141fb0 2020-07-18T03:25:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cc2132700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc147ac0 con 0x7f5cbc144e70 2020-07-18T03:25:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cbb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5cbc141fb0 0x7f5cbc15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cbbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5cbc1486f0 0x7f5cbc146620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39952/0 (socket says 172.21.15.47:39952) 2020-07-18T03:25:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.356+0000 7f5cbbfff700 1 -- 172.21.15.47:0/1025040779 learned_addr learned my addr 172.21.15.47:0/1025040779 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbb7fe700 1 -- 172.21.15.47:0/1025040779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 msgr2=0x7f5cbc148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbaffd700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbb7fe700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbb7fe700 1 -- 172.21.15.47:0/1025040779 >> v1:172.21.15.47:6789/0 conn(0x7f5cbc1486f0 legacy=0x7f5cbc146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489242019 0 0) 0x7f5cbc15a7e0 con 0x7f5cbc1486f0 2020-07-18T03:25:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbb7fe700 1 -- 172.21.15.47:0/1025040779 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc147d60 con 0x7f5cbc141fb0 2020-07-18T03:25:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbaffd700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:25:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cbb7fe700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc141fb0 0x7f5cbc15a220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5cac00c000 tx=0x7f5cac006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cac00ca10 con 0x7f5cbc141fb0 2020-07-18T03:25:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac00cb70 con 0x7f5cbc141fb0 2020-07-18T03:25:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc149250 con 0x7f5cbc141fb0 2020-07-18T03:25:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cac0123f0 con 0x7f5cbc141fb0 2020-07-18T03:25:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.357+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc146d30 con 0x7f5cbc141fb0 2020-07-18T03:25:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.358+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc147060 con 0x7f5cbc141fb0 2020-07-18T03:25:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.358+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5cac0125a0 con 0x7f5cbc141fb0 2020-07-18T03:25:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.358+0000 7f5c9b7fe700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c9404fe20 0x7f5c940522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.358+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5cac068120 con 0x7f5cbc141fb0 2020-07-18T03:25:57.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.358+0000 7f5cbaffd700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c9404fe20 0x7f5c940522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.359+0000 7f5cbaffd700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c9404fe20 0x7f5c940522e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5cb0008000 tx=0x7f5cb0009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.362+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5cac040850 con 0x7f5cbc141fb0 2020-07-18T03:25:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.520+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5cbc0af5f0 con 0x7f5cbc141fb0 2020-07-18T03:25:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.520+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/1025040779 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5cac043e70 con 0x7f5cbc141fb0 2020-07-18T03:25:57.522 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c9404fe20 msgr2=0x7f5c940522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c9404fe20 0x7f5c940522e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5cb0008000 tx=0x7f5cb0009040).stop 2020-07-18T03:25:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc141fb0 msgr2=0x7f5cbc15a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc141fb0 0x7f5cbc15a220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5cac00c000 tx=0x7f5cac006dd0).stop 2020-07-18T03:25:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 shutdown_connections 2020-07-18T03:25:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c9404fe20 0x7f5c940522e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cbc141fb0 0x7f5cbc15a220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 --2- 172.21.15.47:0/1025040779 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5cbc144e70 0x7f5cbc148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 >> 172.21.15.47:0/1025040779 conn(0x7f5cbc05f5a0 msgr2=0x7f5cbc0b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 shutdown_connections 2020-07-18T03:25:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.521+0000 7f5cc2132700 1 -- 172.21.15.47:0/1025040779 wait complete. 2020-07-18T03:25:57.534 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:25:57.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.649+0000 7f5f88305700 1 Processor -- start 2020-07-18T03:25:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.649+0000 7f5f88305700 1 -- start start 2020-07-18T03:25:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.650+0000 7f5f88305700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f800ad330 0x7f5f80143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.650+0000 7f5f88305700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5f800b3a10 0x7f5f800b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.650+0000 7f5f88305700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f80147de0 con 0x7f5f800b59b0 2020-07-18T03:25:57.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.650+0000 7f5f88305700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f800ae000 con 0x7f5f800ad330 2020-07-18T03:25:57.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.650+0000 7f5f88305700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f80148010 con 0x7f5f800b3a10 2020-07-18T03:25:57.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f800ad330 0x7f5f80143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f800ad330 0x7f5f80143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58870/0 (socket says 172.21.15.47:58870) 2020-07-18T03:25:57.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f858a0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5f800b59b0 0x7f5f800b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39958/0 (socket says 172.21.15.47:39958) 2020-07-18T03:25:57.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 -- 172.21.15.47:0/877003798 learned_addr learned my addr 172.21.15.47:0/877003798 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:57.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f868a2700 1 --2- 172.21.15.47:0/877003798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5f800b3a10 0x7f5f800b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 -- 172.21.15.47:0/877003798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5f800b3a10 msgr2=0x7f5f800b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:57.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 --2- 172.21.15.47:0/877003798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5f800b3a10 0x7f5f800b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 -- 172.21.15.47:0/877003798 >> v1:172.21.15.47:6789/0 conn(0x7f5f800b59b0 legacy=0x7f5f800b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f8509f700 1 -- 172.21.15.47:0/877003798 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500684624 0 0) 0x7f5f80147de0 con 0x7f5f800b59b0 2020-07-18T03:25:57.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 -- 172.21.15.47:0/877003798 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f800ad710 con 0x7f5f800ad330 2020-07-18T03:25:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f860a1700 1 --2- 172.21.15.47:0/877003798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800ad330 0x7f5f80143640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f5f74002f30 tx=0x7f5f74006e90).ready entity=mon.1 client_cookie=107b6aa96b41e40e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f8509f700 1 -- 172.21.15.47:0/877003798 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f74009970 con 0x7f5f800ad330 2020-07-18T03:25:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f8509f700 1 -- 172.21.15.47:0/877003798 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f74009ad0 con 0x7f5f800ad330 2020-07-18T03:25:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f8509f700 1 -- 172.21.15.47:0/877003798 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f74009e30 con 0x7f5f800ad330 2020-07-18T03:25:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 -- 172.21.15.47:0/877003798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800ad330 msgr2=0x7f5f80143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 --2- 172.21.15.47:0/877003798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800ad330 0x7f5f80143640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f5f74002f30 tx=0x7f5f74006e90).stop 2020-07-18T03:25:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 -- 172.21.15.47:0/877003798 shutdown_connections 2020-07-18T03:25:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 --2- 172.21.15.47:0/877003798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5f800b3a10 0x7f5f800b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 --2- 172.21.15.47:0/877003798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800ad330 0x7f5f80143640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 -- 172.21.15.47:0/877003798 >> 172.21.15.47:0/877003798 conn(0x7f5f8005f5a0 msgr2=0x7f5f8005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 -- 172.21.15.47:0/877003798 shutdown_connections 2020-07-18T03:25:57.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.651+0000 7f5f88305700 1 -- 172.21.15.47:0/877003798 wait complete. 2020-07-18T03:25:57.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f88305700 1 Processor -- start 2020-07-18T03:25:57.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f88305700 1 -- start start 2020-07-18T03:25:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f88305700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f800b3a10 0x7f5f80066090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f88305700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f80144770 con 0x7f5f800b59b0 2020-07-18T03:25:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f88305700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f80145010 con 0x7f5f80143800 2020-07-18T03:25:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f88305700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f80066d50 con 0x7f5f800b3a10 2020-07-18T03:25:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.653+0000 7f5f860a1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f800b3a10 0x7f5f80066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f868a2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5f80143800 0x7f5f80143c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39968/0 (socket says 172.21.15.47:39968) 2020-07-18T03:25:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f868a2700 1 -- 172.21.15.47:0/133441169 learned_addr learned my addr 172.21.15.47:0/133441169 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f860a1700 1 -- 172.21.15.47:0/133441169 >> v1:172.21.15.47:6789/0 conn(0x7f5f80143800 legacy=0x7f5f80143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f860a1700 1 -- 172.21.15.47:0/133441169 >> v1:172.21.15.18:6789/0 conn(0x7f5f800b59b0 legacy=0x7f5f80064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f860a1700 1 -- 172.21.15.47:0/133441169 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80064f20 con 0x7f5f800b3a10 2020-07-18T03:25:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987312146 0 0) 0x7f5f80145010 con 0x7f5f80143800 2020-07-18T03:25:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f860a1700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800b3a10 0x7f5f80066090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f5f800ba020 tx=0x7f5f74015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f74009970 con 0x7f5f800b3a10 2020-07-18T03:25:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f74009e90 con 0x7f5f800b3a10 2020-07-18T03:25:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f800669d0 con 0x7f5f800b3a10 2020-07-18T03:25:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7401d7c0 con 0x7f5f800b3a10 2020-07-18T03:25:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.654+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f80066740 con 0x7f5f800b3a10 2020-07-18T03:25:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.655+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5f74025030 con 0x7f5f800b3a10 2020-07-18T03:25:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.655+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f80066bc0 con 0x7f5f800b3a10 2020-07-18T03:25:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.655+0000 7f5f727fc700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5f5804feb0 0x7f5f58052370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.655+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5f74069360 con 0x7f5f800b3a10 2020-07-18T03:25:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.656+0000 7f5f858a0700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5f5804feb0 0x7f5f58052370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.656+0000 7f5f858a0700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5f5804feb0 0x7f5f58052370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5f7c000b80 tx=0x7f5f7c0075e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.659+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5f74041af0 con 0x7f5f800b3a10 2020-07-18T03:25:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.816+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f8005b120 con 0x7f5f800b3a10 2020-07-18T03:25:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.817+0000 7f5f727fc700 1 -- 172.21.15.47:0/133441169 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5f74045110 con 0x7f5f800b3a10 2020-07-18T03:25:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5f5804feb0 msgr2=0x7f5f58052370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5f5804feb0 0x7f5f58052370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5f7c000b80 tx=0x7f5f7c0075e0).stop 2020-07-18T03:25:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800b3a10 msgr2=0x7f5f80066090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800b3a10 0x7f5f80066090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f5f800ba020 tx=0x7f5f74015000).stop 2020-07-18T03:25:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 shutdown_connections 2020-07-18T03:25:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5f5804feb0 0x7f5f58052370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 --2- 172.21.15.47:0/133441169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f800b3a10 0x7f5f80066090 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 >> 172.21.15.47:0/133441169 conn(0x7f5f8005f5a0 msgr2=0x7f5f800b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 shutdown_connections 2020-07-18T03:25:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.818+0000 7f5f88305700 1 -- 172.21.15.47:0/133441169 wait complete. 2020-07-18T03:25:57.830 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:25:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.946+0000 7f0e90a4f700 1 Processor -- start 2020-07-18T03:25:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.946+0000 7f0e90a4f700 1 -- start start 2020-07-18T03:25:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e90a4f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c061c60 0x7f0e8c0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e90a4f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c0af4b0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e90a4f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c0b6d80 con 0x7f0e8c142a20 2020-07-18T03:25:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e90a4f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0af710 con 0x7f0e8c061c60 2020-07-18T03:25:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e8a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0e8c142a20 0x7f0e8c142e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39972/0 (socket says 172.21.15.47:39972) 2020-07-18T03:25:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e8a59c700 1 -- 172.21.15.47:0/4114137770 learned_addr learned my addr 172.21.15.47:0/4114137770 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e89d9b700 1 --2- 172.21.15.47:0/4114137770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c061c60 0x7f0e8c0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e8959a700 1 -- 172.21.15.47:0/4114137770 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118054917 0 0) 0x7f0e8c0b6d80 con 0x7f0e8c142a20 2020-07-18T03:25:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e8959a700 1 -- 172.21.15.47:0/4114137770 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e70003240 con 0x7f0e8c142a20 2020-07-18T03:25:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e89d9b700 1 -- 172.21.15.47:0/4114137770 >> v1:172.21.15.47:6789/0 conn(0x7f0e8c142a20 legacy=0x7f0e8c142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e89d9b700 1 -- 172.21.15.47:0/4114137770 >> v1:172.21.15.18:6789/0 conn(0x7f0e8c0b3af0 legacy=0x7f0e8c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e8959a700 1 -- 172.21.15.47:0/4114137770 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617742704 0 0) 0x7f0e8c0af4b0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e89d9b700 1 -- 172.21.15.47:0/4114137770 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0af9b0 con 0x7f0e8c061c60 2020-07-18T03:25:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e89d9b700 1 --2- 172.21.15.47:0/4114137770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c061c60 0x7f0e8c0620c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0e80002f30 tx=0x7f0e80006dd0).ready entity=mon.2 client_cookie=c0ff2c0f5d51bfca server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.947+0000 7f0e8959a700 1 -- 172.21.15.47:0/4114137770 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e800097f0 con 0x7f0e8c061c60 2020-07-18T03:25:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e8959a700 1 -- 172.21.15.47:0/4114137770 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80009950 con 0x7f0e8c061c60 2020-07-18T03:25:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e8959a700 1 -- 172.21.15.47:0/4114137770 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e80009cb0 con 0x7f0e8c061c60 2020-07-18T03:25:57.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4114137770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c061c60 msgr2=0x7f0e8c0620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4114137770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c061c60 0x7f0e8c0620c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0e80002f30 tx=0x7f0e80006dd0).stop 2020-07-18T03:25:57.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4114137770 shutdown_connections 2020-07-18T03:25:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4114137770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c061c60 0x7f0e8c0620c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4114137770 >> 172.21.15.47:0/4114137770 conn(0x7f0e8c05f5a0 msgr2=0x7f0e8c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4114137770 shutdown_connections 2020-07-18T03:25:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.948+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4114137770 wait complete. 2020-07-18T03:25:57.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.949+0000 7f0e90a4f700 1 Processor -- start 2020-07-18T03:25:57.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.949+0000 7f0e90a4f700 1 -- start start 2020-07-18T03:25:57.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e90a4f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e8c0b3af0 0x7f0e8c0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e90a4f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c145df0 0x7f0e8c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e90a4f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c0b2500 con 0x7f0e8c142a20 2020-07-18T03:25:57.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e90a4f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b2790 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e90a4f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b0630 con 0x7f0e8c145df0 2020-07-18T03:25:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e8c0b3af0 0x7f0e8c0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e8c0b3af0 0x7f0e8c0b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58890/0 (socket says 172.21.15.47:58890) 2020-07-18T03:25:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 -- 172.21.15.47:0/4222743670 learned_addr learned my addr 172.21.15.47:0/4222743670 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8ad9d700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c145df0 0x7f0e8c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 -- 172.21.15.47:0/4222743670 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c145df0 msgr2=0x7f0e8c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c145df0 0x7f0e8c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:57.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 -- 172.21.15.47:0/4222743670 >> v1:172.21.15.18:6789/0 conn(0x7f0e8c142a20 legacy=0x7f0e8c141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:57.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 -- 172.21.15.47:0/4222743670 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c150fe0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e8a59c700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e8c0b3af0 0x7f0e8c0b2010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f0e84001450 tx=0x7f0e84007d80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e84005b80 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c0b0ad0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e84005ce0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.950+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e84007300 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.951+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c0b0ff0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.951+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c05b120 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.952+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0e84010030 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.952+0000 7f0e7a7fc700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0e6004ff50 0x7f0e60052410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:57.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.952+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0e84067ce0 con 0x7f0e8c0b3af0 2020-07-18T03:25:57.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.952+0000 7f0e89d9b700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0e6004ff50 0x7f0e60052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:57.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.952+0000 7f0e89d9b700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0e6004ff50 0x7f0e60052410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0e8c0aef60 tx=0x7f0e80015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:57.955+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0e84040390 con 0x7f0e8c0b3af0 2020-07-18T03:25:58.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.113+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e8c0b07a0 con 0x7f0e8c0b3af0 2020-07-18T03:25:58.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.113+0000 7f0e7a7fc700 1 -- 172.21.15.47:0/4222743670 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0e840439b0 con 0x7f0e8c0b3af0 2020-07-18T03:25:58.116 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.114+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0e6004ff50 msgr2=0x7f0e60052410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.114+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0e6004ff50 0x7f0e60052410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0e8c0aef60 tx=0x7f0e80015000).stop 2020-07-18T03:25:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.114+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e8c0b3af0 msgr2=0x7f0e8c0b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.114+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e8c0b3af0 0x7f0e8c0b2010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f0e84001450 tx=0x7f0e84007d80).stop 2020-07-18T03:25:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 shutdown_connections 2020-07-18T03:25:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0e6004ff50 0x7f0e60052410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0e8c145df0 0x7f0e8c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 --2- 172.21.15.47:0/4222743670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e8c0b3af0 0x7f0e8c0b2010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 >> 172.21.15.47:0/4222743670 conn(0x7f0e8c05f5a0 msgr2=0x7f0e8c0b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 shutdown_connections 2020-07-18T03:25:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.115+0000 7f0e90a4f700 1 -- 172.21.15.47:0/4222743670 wait complete. 2020-07-18T03:25:58.127 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:25:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.240+0000 7fa8e30e7700 1 Processor -- start 2020-07-18T03:25:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.240+0000 7fa8e30e7700 1 -- start start 2020-07-18T03:25:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc0b5370 0x7fa8dc05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8dc0b41c0 0x7fa8dc0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0b3ac0 con 0x7fa8dc0b4620 2020-07-18T03:25:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0b5a10 con 0x7fa8dc0b41c0 2020-07-18T03:25:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0b0040 con 0x7fa8dc0b5370 2020-07-18T03:25:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8dbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8dc0b41c0 0x7fa8dc0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e1684700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa8dc0b4620 0x7fa8dc05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39990/0 (socket says 172.21.15.47:39990) 2020-07-18T03:25:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e1684700 1 -- 172.21.15.47:0/1006362870 learned_addr learned my addr 172.21.15.47:0/1006362870 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e0e83700 1 --2- 172.21.15.47:0/1006362870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc0b5370 0x7fa8dc05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8dbfff700 1 -- 172.21.15.47:0/1006362870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc0b5370 msgr2=0x7fa8dc05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8dbfff700 1 --2- 172.21.15.47:0/1006362870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc0b5370 0x7fa8dc05fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8dbfff700 1 -- 172.21.15.47:0/1006362870 >> v1:172.21.15.47:6789/0 conn(0x7fa8dc0b4620 legacy=0x7fa8dc05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8dbfff700 1 -- 172.21.15.47:0/1006362870 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0b3d20 con 0x7fa8dc0b41c0 2020-07-18T03:25:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8db7fe700 1 -- 172.21.15.47:0/1006362870 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131177522 0 0) 0x7fa8dc0b3ac0 con 0x7fa8dc0b4620 2020-07-18T03:25:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8dbfff700 1 --2- 172.21.15.47:0/1006362870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc0b41c0 0x7fa8dc0ad2d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fa8d0002f30 tx=0x7fa8d0006dd0).ready entity=mon.1 client_cookie=6e72df0a3558506d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8db7fe700 1 -- 172.21.15.47:0/1006362870 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d00097f0 con 0x7fa8dc0b41c0 2020-07-18T03:25:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8db7fe700 1 -- 172.21.15.47:0/1006362870 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d0009950 con 0x7fa8dc0b41c0 2020-07-18T03:25:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8db7fe700 1 -- 172.21.15.47:0/1006362870 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d0009cb0 con 0x7fa8dc0b41c0 2020-07-18T03:25:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1006362870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc0b41c0 msgr2=0x7fa8dc0ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.241+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1006362870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc0b41c0 0x7fa8dc0ad2d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fa8d0002f30 tx=0x7fa8d0006dd0).stop 2020-07-18T03:25:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.242+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1006362870 shutdown_connections 2020-07-18T03:25:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.242+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1006362870 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc0b5370 0x7fa8dc05fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.242+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1006362870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc0b41c0 0x7fa8dc0ad2d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.242+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1006362870 >> 172.21.15.47:0/1006362870 conn(0x7fa8dc0b9340 msgr2=0x7fa8dc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.242+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1006362870 shutdown_connections 2020-07-18T03:25:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.242+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1006362870 wait complete. 2020-07-18T03:25:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.243+0000 7fa8e30e7700 1 Processor -- start 2020-07-18T03:25:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.243+0000 7fa8e30e7700 1 -- start start 2020-07-18T03:25:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e30e7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc159b20 0x7fa8dc13fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e30e7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8dc1417e0 0x7fa8dc141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e30e7700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc159140 con 0x7fa8dc0b5370 2020-07-18T03:25:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e30e7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc1593e0 con 0x7fa8dc1417e0 2020-07-18T03:25:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e30e7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc159680 con 0x7fa8dc159b20 2020-07-18T03:25:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8dc1417e0 0x7fa8dc141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8dc1417e0 0x7fa8dc141c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58908/0 (socket says 172.21.15.47:58908) 2020-07-18T03:25:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 -- 172.21.15.47:0/1880901505 learned_addr learned my addr 172.21.15.47:0/1880901505 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8dbfff700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc159b20 0x7fa8dc13fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 -- 172.21.15.47:0/1880901505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc159b20 msgr2=0x7fa8dc13fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc159b20 0x7fa8dc13fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 -- 172.21.15.47:0/1880901505 >> v1:172.21.15.18:6789/0 conn(0x7fa8dc0b5370 legacy=0x7fa8dc0661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 -- 172.21.15.47:0/1880901505 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc1401b0 con 0x7fa8dc1417e0 2020-07-18T03:25:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8dbfff700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc159b20 0x7fa8dc13fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:25:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.244+0000 7fa8e1684700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc1417e0 0x7fa8dc141c90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa8cc000b80 tx=0x7fa8cc007940).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.245+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8cc0072e0 con 0x7fa8dc1417e0 2020-07-18T03:25:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.245+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc007440 con 0x7fa8dc1417e0 2020-07-18T03:25:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.245+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc142180 con 0x7fa8dc1417e0 2020-07-18T03:25:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.245+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8cc0057e0 con 0x7fa8dc1417e0 2020-07-18T03:25:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.245+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc155a60 con 0x7fa8dc1417e0 2020-07-18T03:25:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.246+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa8cc01a060 con 0x7fa8dc1417e0 2020-07-18T03:25:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.246+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc0b0cf0 con 0x7fa8dc1417e0 2020-07-18T03:25:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.246+0000 7fa8d8ff9700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa8b4050040 0x7fa8b4052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.246+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa8cc067570 con 0x7fa8dc1417e0 2020-07-18T03:25:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.246+0000 7fa8e0e83700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa8b4050040 0x7fa8b4052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.246+0000 7fa8e0e83700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa8b4050040 0x7fa8b4052500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa8dc13cf60 tx=0x7fa8d4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.249+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa8cc03fc80 con 0x7fa8dc1417e0 2020-07-18T03:25:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.462+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8dc05d450 con 0x7fa8dc1417e0 2020-07-18T03:25:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.462+0000 7fa8d8ff9700 1 -- 172.21.15.47:0/1880901505 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2124 (secure 0 0 0) 0x7fa8cc018070 con 0x7fa8dc1417e0 2020-07-18T03:25:58.464 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:58.465 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:49.581963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4387":{"gid":4387,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.47:6835/308881804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":308881804},{"type":"v1","addr":"172.21.15.47:6835","nonce":308881804}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:25:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa8b4050040 msgr2=0x7fa8b4052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa8b4050040 0x7fa8b4052500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa8dc13cf60 tx=0x7fa8d4008040).stop 2020-07-18T03:25:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc1417e0 msgr2=0x7fa8dc141c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc1417e0 0x7fa8dc141c90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa8cc000b80 tx=0x7fa8cc007940).stop 2020-07-18T03:25:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 shutdown_connections 2020-07-18T03:25:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa8b4050040 0x7fa8b4052500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8dc1417e0 0x7fa8dc141c90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 --2- 172.21.15.47:0/1880901505 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa8dc159b20 0x7fa8dc13fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 >> 172.21.15.47:0/1880901505 conn(0x7fa8dc0b9340 msgr2=0x7fa8dc0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 shutdown_connections 2020-07-18T03:25:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:58.463+0000 7fa8e30e7700 1 -- 172.21.15.47:0/1880901505 wait complete. 2020-07-18T03:25:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2020-07-18T03:25:58.679 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:25:45.899311+0000', 'modified': '2020-07-18T03:25:49.581963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4387': {'gid': 4387, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.47:6835/308881804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 308881804}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 308881804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:25:58.679 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:25:58.680 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:25:58.816 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4387,"want_state":"up:active","state":"up:active","rank_uptime":10.223593907,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":10.480345875999999} 2020-07-18T03:25:58.817 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4387, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 10.223593907, "state": "up:active", "uptime": 10.480345876, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:25:58.817 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-07-18T03:25:58.883 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-07-18T03:25:58.884 INFO:tasks.ceph_fuse:config is None 2020-07-18T03:25:58.884 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None, 'client.3': None} 2020-07-18T03:25:58.885 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-07-18T03:25:58.885 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:25:59.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.000+0000 7f5a786d0700 1 Processor -- start 2020-07-18T03:25:59.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.000+0000 7f5a786d0700 1 -- start start 2020-07-18T03:25:59.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a786d0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a700b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a786d0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700b95a0 con 0x7f5a700b82f0 2020-07-18T03:25:59.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a786d0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700b5c80 con 0x7f5a700b42a0 2020-07-18T03:25:59.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a786d0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700b5a90 con 0x7f5a700b7420 2020-07-18T03:25:59.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a7646c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5a700b42a0 0x7f5a700b6660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40000/0 (socket says 172.21.15.47:40000) 2020-07-18T03:25:59.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a7646c700 1 -- 172.21.15.47:0/1332479845 learned_addr learned my addr 172.21.15.47:0/1332479845 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a75c6b700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a700b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675935874 0 0) 0x7f5a700b5c80 con 0x7f5a700b42a0 2020-07-18T03:25:59.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.001+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a58003240 con 0x7f5a700b42a0 2020-07-18T03:25:59.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2840859649 0 0) 0x7f5a58003240 con 0x7f5a700b42a0 2020-07-18T03:25:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 msgr2=0x7f5a700b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a700b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 >> v1:172.21.15.18:6789/0 conn(0x7f5a700b82f0 legacy=0x7f5a7014a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700b5f20 con 0x7f5a700b42a0 2020-07-18T03:25:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5a640016d0 con 0x7f5a700b42a0 2020-07-18T03:25:59.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 >> v1:172.21.15.47:6789/0 conn(0x7f5a700b42a0 legacy=0x7f5a700b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a58003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a58003c20 0x7f5a580044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a580049d0 0x7f5a58004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a58004140 con 0x7f5a58003c20 2020-07-18T03:25:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580042b0 con 0x7f5a580049d0 2020-07-18T03:25:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5800b730 con 0x7f5a700b7420 2020-07-18T03:25:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a58003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a75c6b700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a58003c20 0x7f5a580044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a76c6d700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a580049d0 0x7f5a58004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 -- 172.21.15.47:0/1332479845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a580049d0 msgr2=0x7f5a58004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:59.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a580049d0 0x7f5a58004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 -- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a58003c20 msgr2=0x7f5a580044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:59.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a58003c20 0x7f5a580044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 -- 172.21.15.47:0/1332479845 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58003240 con 0x7f5a700b7420 2020-07-18T03:25:59.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a75c6b700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a58003c20 0x7f5a580044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:25:59.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.002+0000 7f5a7646c700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a58003730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f5a640013e0 tx=0x7f5a700b5e10).ready entity=mon.2 client_cookie=5f6b9b7aa5ac4b08 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6400ce60 con 0x7f5a700b7420 2020-07-18T03:25:59.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a640070d0 con 0x7f5a700b7420 2020-07-18T03:25:59.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a7546a700 1 -- 172.21.15.47:0/1332479845 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a64007430 con 0x7f5a700b7420 2020-07-18T03:25:59.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a786d0700 1 -- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 msgr2=0x7f5a58003730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a786d0700 1 --2- 172.21.15.47:0/1332479845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5a700b7420 0x7f5a58003730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f5a640013e0 tx=0x7f5a700b5e10).stop 2020-07-18T03:25:59.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a76c6d700 1 -- 172.21.15.47:0/1332479845 reap_dead start 2020-07-18T03:25:59.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a786d0700 1 -- 172.21.15.47:0/1332479845 shutdown_connections 2020-07-18T03:25:59.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a786d0700 1 -- 172.21.15.47:0/1332479845 >> 172.21.15.47:0/1332479845 conn(0x7f5a7005f5a0 msgr2=0x7f5a7005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a786d0700 1 -- 172.21.15.47:0/1332479845 shutdown_connections 2020-07-18T03:25:59.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.003+0000 7f5a786d0700 1 -- 172.21.15.47:0/1332479845 wait complete. 2020-07-18T03:25:59.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.004+0000 7f5a786d0700 1 Processor -- start 2020-07-18T03:25:59.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.004+0000 7f5a786d0700 1 -- start start 2020-07-18T03:25:59.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a786d0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5a70145990 0x7f5a70158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a786d0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a701384a0 0x7f5a700b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a786d0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a70144710 con 0x7f5a70145d70 2020-07-18T03:25:59.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a786d0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a700af240 con 0x7f5a70145990 2020-07-18T03:25:59.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a786d0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a701449b0 con 0x7f5a701384a0 2020-07-18T03:25:59.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a701384a0 0x7f5a700b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a75c6b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5a70145d70 0x7f5a70137d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40014/0 (socket says 172.21.15.47:40014) 2020-07-18T03:25:59.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a701384a0 0x7f5a700b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58928/0 (socket says 172.21.15.47:58928) 2020-07-18T03:25:59.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a75c6b700 1 -- 172.21.15.47:0/3863677498 learned_addr learned my addr 172.21.15.47:0/3863677498 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a7646c700 1 --2- 172.21.15.47:0/3863677498 >> v2:172.21.15.18:3300/0 conn(0x7f5a70145990 0x7f5a70158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 -- 172.21.15.47:0/3863677498 >> v2:172.21.15.18:3300/0 conn(0x7f5a70145990 msgr2=0x7f5a70158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 --2- 172.21.15.47:0/3863677498 >> v2:172.21.15.18:3300/0 conn(0x7f5a70145990 0x7f5a70158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 -- 172.21.15.47:0/3863677498 >> v1:172.21.15.47:6789/0 conn(0x7f5a70145d70 legacy=0x7f5a70137d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 -- 172.21.15.47:0/3863677498 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a70144c50 con 0x7f5a701384a0 2020-07-18T03:25:59.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352577037 0 0) 0x7f5a70144710 con 0x7f5a70145d70 2020-07-18T03:25:59.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.005+0000 7f5a76c6d700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a701384a0 0x7f5a700b2de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f5a700b2a80 tx=0x7f5a68007e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.006+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a68006d40 con 0x7f5a701384a0 2020-07-18T03:25:59.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.006+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68007070 con 0x7f5a701384a0 2020-07-18T03:25:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.006+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a68006ea0 con 0x7f5a701384a0 2020-07-18T03:25:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.006+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a701590c0 con 0x7f5a701384a0 2020-07-18T03:25:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.006+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a700af440 con 0x7f5a701384a0 2020-07-18T03:25:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.007+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a70057450 con 0x7f5a701384a0 2020-07-18T03:25:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.007+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5a680073d0 con 0x7f5a701384a0 2020-07-18T03:25:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.008+0000 7f5a627fc700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4804fec0 0x7f5a48052380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.008+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5a68067f00 con 0x7f5a701384a0 2020-07-18T03:25:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.008+0000 7f5a7646c700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4804fec0 0x7f5a48052380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.008+0000 7f5a7646c700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4804fec0 0x7f5a48052380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5a6401b000 tx=0x7f5a64015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.010+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5a680405f0 con 0x7f5a701384a0 2020-07-18T03:25:59.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.168+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a7005da00 con 0x7f5a701384a0 2020-07-18T03:25:59.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.169+0000 7f5a627fc700 1 -- 172.21.15.47:0/3863677498 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f5a68043c10 con 0x7f5a701384a0 2020-07-18T03:25:59.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:25:59.172 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:25:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4804fec0 msgr2=0x7f5a48052380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4804fec0 0x7f5a48052380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5a6401b000 tx=0x7f5a64015000).stop 2020-07-18T03:25:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a701384a0 msgr2=0x7f5a700b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a701384a0 0x7f5a700b2de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f5a700b2a80 tx=0x7f5a68007e60).stop 2020-07-18T03:25:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 shutdown_connections 2020-07-18T03:25:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a701384a0 0x7f5a700b2de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 --2- 172.21.15.47:0/3863677498 >> v2:172.21.15.18:3300/0 conn(0x7f5a70145990 0x7f5a70158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 --2- 172.21.15.47:0/3863677498 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4804fec0 0x7f5a48052380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 >> 172.21.15.47:0/3863677498 conn(0x7f5a7005f5a0 msgr2=0x7f5a701465d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 shutdown_connections 2020-07-18T03:25:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.170+0000 7f5a786d0700 1 -- 172.21.15.47:0/3863677498 wait complete. 2020-07-18T03:25:59.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:25:59.185 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:25:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 Processor -- start 2020-07-18T03:25:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 -- start start 2020-07-18T03:25:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fefc40ad330 0x7fefc4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fefc40b3a10 0x7fefc40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc4147de0 con 0x7fefc40b59b0 2020-07-18T03:25:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc40ae000 con 0x7fefc40ad330 2020-07-18T03:25:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.301+0000 7fefca545700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc4148010 con 0x7fefc40b3a10 2020-07-18T03:25:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fefc40ad330 0x7fefc4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fefc40b59b0 0x7fefc40b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40020/0 (socket says 172.21.15.47:40020) 2020-07-18T03:25:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc37fe700 1 -- 172.21.15.47:0/3628979909 learned_addr learned my addr 172.21.15.47:0/3628979909 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc8ae2700 1 --2- 172.21.15.47:0/3628979909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fefc40b3a10 0x7fefc40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc3fff700 1 -- 172.21.15.47:0/3628979909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fefc40b3a10 msgr2=0x7fefc40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc3fff700 1 --2- 172.21.15.47:0/3628979909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fefc40b3a10 0x7fefc40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc3fff700 1 -- 172.21.15.47:0/3628979909 >> v1:172.21.15.47:6789/0 conn(0x7fefc40b59b0 legacy=0x7fefc40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc3fff700 1 -- 172.21.15.47:0/3628979909 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc40ad710 con 0x7fefc40ad330 2020-07-18T03:25:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc3fff700 1 --2- 172.21.15.47:0/3628979909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40ad330 0x7fefc4143640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fefb4002f30 tx=0x7fefb4006dd0).ready entity=mon.1 client_cookie=f7a613e292327722 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc2ffd700 1 -- 172.21.15.47:0/3628979909 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb40097f0 con 0x7fefc40ad330 2020-07-18T03:25:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc2ffd700 1 -- 172.21.15.47:0/3628979909 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb4009950 con 0x7fefc40ad330 2020-07-18T03:25:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefc2ffd700 1 -- 172.21.15.47:0/3628979909 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb4009cb0 con 0x7fefc40ad330 2020-07-18T03:25:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 -- 172.21.15.47:0/3628979909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40ad330 msgr2=0x7fefc4143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 --2- 172.21.15.47:0/3628979909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40ad330 0x7fefc4143640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fefb4002f30 tx=0x7fefb4006dd0).stop 2020-07-18T03:25:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 -- 172.21.15.47:0/3628979909 shutdown_connections 2020-07-18T03:25:59.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 --2- 172.21.15.47:0/3628979909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fefc40b3a10 0x7fefc40b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 --2- 172.21.15.47:0/3628979909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40ad330 0x7fefc4143640 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 -- 172.21.15.47:0/3628979909 >> 172.21.15.47:0/3628979909 conn(0x7fefc405f5a0 msgr2=0x7fefc405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 -- 172.21.15.47:0/3628979909 shutdown_connections 2020-07-18T03:25:59.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.302+0000 7fefca545700 1 -- 172.21.15.47:0/3628979909 wait complete. 2020-07-18T03:25:59.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.304+0000 7fefca545700 1 Processor -- start 2020-07-18T03:25:59.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.304+0000 7fefca545700 1 -- start start 2020-07-18T03:25:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefca545700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fefc40b59b0 0x7fefc41438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefca545700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc413b8c0 con 0x7fefc41479a0 2020-07-18T03:25:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefca545700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc413baf0 con 0x7fefc40b3a10 2020-07-18T03:25:59.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefca545700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc4064700 con 0x7fefc40b59b0 2020-07-18T03:25:59.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fefc40b59b0 0x7fefc41438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fefc40b3a10 0x7fefc4147290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40026/0 (socket says 172.21.15.47:40026) 2020-07-18T03:25:59.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc3fff700 1 -- 172.21.15.47:0/1893555833 learned_addr learned my addr 172.21.15.47:0/1893555833 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc37fe700 1 -- 172.21.15.47:0/1893555833 >> v1:172.21.15.47:6789/0 conn(0x7fefc40b3a10 legacy=0x7fefc4147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc37fe700 1 -- 172.21.15.47:0/1893555833 >> v1:172.21.15.18:6789/0 conn(0x7fefc41479a0 legacy=0x7fefc4065ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767334813 0 0) 0x7fefc413baf0 con 0x7fefc40b3a10 2020-07-18T03:25:59.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc37fe700 1 -- 172.21.15.47:0/1893555833 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4064840 con 0x7fefc40b59b0 2020-07-18T03:25:59.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefc37fe700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40b59b0 0x7fefc41438e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fefc413b7b0 tx=0x7fefb8007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb8007380 con 0x7fefc40b59b0 2020-07-18T03:25:59.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb80055f0 con 0x7fefc40b59b0 2020-07-18T03:25:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc40665b0 con 0x7fefc40b59b0 2020-07-18T03:25:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb8005950 con 0x7fefc40b59b0 2020-07-18T03:25:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.305+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc4066a80 con 0x7fefc40b59b0 2020-07-18T03:25:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.306+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc4066d00 con 0x7fefc40b59b0 2020-07-18T03:25:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.306+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fefb8010070 con 0x7fefc40b59b0 2020-07-18T03:25:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.307+0000 7fefa3fff700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef9c04fe60 0x7fef9c052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.307+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fefb8067330 con 0x7fefc40b59b0 2020-07-18T03:25:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.307+0000 7fefc3fff700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef9c04fe60 0x7fef9c052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.307+0000 7fefc3fff700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef9c04fe60 0x7fef9c052320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fefb4000b80 tx=0x7fefb4015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.310+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fefb803fac0 con 0x7fefc40b59b0 2020-07-18T03:25:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.469+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fefc40b0fb0 con 0x7fefc40b59b0 2020-07-18T03:25:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.469+0000 7fefa3fff700 1 -- 172.21.15.47:0/1893555833 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fefb80430e0 con 0x7fefc40b59b0 2020-07-18T03:25:59.472 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:25:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef9c04fe60 msgr2=0x7fef9c052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef9c04fe60 0x7fef9c052320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fefb4000b80 tx=0x7fefb4015000).stop 2020-07-18T03:25:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40b59b0 msgr2=0x7fefc41438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40b59b0 0x7fefc41438e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fefc413b7b0 tx=0x7fefb8007b30).stop 2020-07-18T03:25:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 shutdown_connections 2020-07-18T03:25:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef9c04fe60 0x7fef9c052320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 --2- 172.21.15.47:0/1893555833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefc40b59b0 0x7fefc41438e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 >> 172.21.15.47:0/1893555833 conn(0x7fefc405f5a0 msgr2=0x7fefc40af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 shutdown_connections 2020-07-18T03:25:59.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.471+0000 7fefca545700 1 -- 172.21.15.47:0/1893555833 wait complete. 2020-07-18T03:25:59.483 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:25:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.599+0000 7efcaf500700 1 Processor -- start 2020-07-18T03:25:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.599+0000 7efcaf500700 1 -- start start 2020-07-18T03:25:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaf500700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efca80b59b0 0x7efca80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaf500700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca8147de0 con 0x7efca80b6300 2020-07-18T03:25:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaf500700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80ae000 con 0x7efca80ad330 2020-07-18T03:25:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaf500700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca8148010 con 0x7efca80b59b0 2020-07-18T03:25:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcad29c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efca80ad330 0x7efca8143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40034/0 (socket says 172.21.15.47:40034) 2020-07-18T03:25:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcad29c700 1 -- 172.21.15.47:0/1211138061 learned_addr learned my addr 172.21.15.47:0/1211138061 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaca9b700 1 --2- 172.21.15.47:0/1211138061 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efca80b59b0 0x7efca80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efca7fff700 1 -- 172.21.15.47:0/1211138061 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107915018 0 0) 0x7efca80ae000 con 0x7efca80ad330 2020-07-18T03:25:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efca7fff700 1 -- 172.21.15.47:0/1211138061 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc94003240 con 0x7efca80ad330 2020-07-18T03:25:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efca7fff700 1 -- 172.21.15.47:0/1211138061 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3225034905 0 0) 0x7efc94003240 con 0x7efca80ad330 2020-07-18T03:25:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaca9b700 1 -- 172.21.15.47:0/1211138061 >> v1:172.21.15.47:6789/0 conn(0x7efca80ad330 legacy=0x7efca8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaca9b700 1 -- 172.21.15.47:0/1211138061 >> v1:172.21.15.18:6789/0 conn(0x7efca80b6300 legacy=0x7efca80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.600+0000 7efcaca9b700 1 -- 172.21.15.47:0/1211138061 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80ad710 con 0x7efca80b59b0 2020-07-18T03:25:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaca9b700 1 --2- 172.21.15.47:0/1211138061 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efca80b59b0 0x7efca80b5e10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7efc980010d0 tx=0x7efc9c006dd0).ready entity=mon.2 client_cookie=ed91329178dcd861 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efca7fff700 1 -- 172.21.15.47:0/1211138061 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efca0001a50 con 0x7efca80b59b0 2020-07-18T03:25:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efca7fff700 1 -- 172.21.15.47:0/1211138061 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c00d570 con 0x7efca80b59b0 2020-07-18T03:25:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efca7fff700 1 -- 172.21.15.47:0/1211138061 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc9c00d6e0 con 0x7efca80b59b0 2020-07-18T03:25:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 -- 172.21.15.47:0/1211138061 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efca80b59b0 msgr2=0x7efca80b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 --2- 172.21.15.47:0/1211138061 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efca80b59b0 0x7efca80b5e10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7efc980010d0 tx=0x7efc9c006dd0).stop 2020-07-18T03:25:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 -- 172.21.15.47:0/1211138061 shutdown_connections 2020-07-18T03:25:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 --2- 172.21.15.47:0/1211138061 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efca80b59b0 0x7efca80b5e10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 -- 172.21.15.47:0/1211138061 >> 172.21.15.47:0/1211138061 conn(0x7efca805f5a0 msgr2=0x7efca805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 -- 172.21.15.47:0/1211138061 shutdown_connections 2020-07-18T03:25:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.601+0000 7efcaf500700 1 -- 172.21.15.47:0/1211138061 wait complete. 2020-07-18T03:25:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.602+0000 7efcaf500700 1 Processor -- start 2020-07-18T03:25:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.602+0000 7efcaf500700 1 -- start start 2020-07-18T03:25:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaf500700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efca80ad330 0x7efca8145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaf500700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efca8063f90 0x7efca80afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaf500700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca8146210 con 0x7efca80b6300 2020-07-18T03:25:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaf500700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca8064550 con 0x7efca80ad330 2020-07-18T03:25:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaf500700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca8144060 con 0x7efca8063f90 2020-07-18T03:25:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcada9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efca8063f90 0x7efca80afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaca9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efca80b6300 0x7efca80647f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40044/0 (socket says 172.21.15.47:40044) 2020-07-18T03:25:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcaca9b700 1 -- 172.21.15.47:0/2206064439 learned_addr learned my addr 172.21.15.47:0/2206064439 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efcad29c700 1 --2- 172.21.15.47:0/2206064439 >> v2:172.21.15.18:3300/0 conn(0x7efca80ad330 0x7efca8145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454651721 0 0) 0x7efca8146210 con 0x7efca80b6300 2020-07-18T03:25:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.603+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc80003240 con 0x7efca80b6300 2020-07-18T03:25:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcada9d700 1 -- 172.21.15.47:0/2206064439 >> v2:172.21.15.18:3300/0 conn(0x7efca80ad330 msgr2=0x7efca8145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcada9d700 1 --2- 172.21.15.47:0/2206064439 >> v2:172.21.15.18:3300/0 conn(0x7efca80ad330 0x7efca8145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcada9d700 1 -- 172.21.15.47:0/2206064439 >> v1:172.21.15.47:6789/0 conn(0x7efca80b6300 legacy=0x7efca80647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcada9d700 1 -- 172.21.15.47:0/2206064439 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80aff70 con 0x7efca8063f90 2020-07-18T03:25:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcada9d700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8063f90 0x7efca80afa80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7efca8063460 tx=0x7efc980079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc98013410 con 0x7efca8063f90 2020-07-18T03:25:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98013570 con 0x7efca8063f90 2020-07-18T03:25:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc980138d0 con 0x7efca8063f90 2020-07-18T03:25:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca813aa90 con 0x7efca8063f90 2020-07-18T03:25:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.604+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca813af60 con 0x7efca8063f90 2020-07-18T03:25:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.605+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca8143ec0 con 0x7efca8063f90 2020-07-18T03:25:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.605+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7efc98010030 con 0x7efca8063f90 2020-07-18T03:25:59.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.605+0000 7efca57fa700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efc80050350 0x7efc80052810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.605+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efc98067a50 con 0x7efca8063f90 2020-07-18T03:25:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.606+0000 7efcad29c700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efc80050350 0x7efc80052810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.606+0000 7efcad29c700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efc80050350 0x7efc80052810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7efca000d000 tx=0x7efca0000f00).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.609+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7efca8143ec0 con 0x7efca8063f90 2020-07-18T03:25:59.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.765+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efca8143ec0 con 0x7efca8063f90 2020-07-18T03:25:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.765+0000 7efca57fa700 1 -- 172.21.15.47:0/2206064439 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7efca8143ec0 con 0x7efca8063f90 2020-07-18T03:25:59.768 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:25:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efc80050350 msgr2=0x7efc80052810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efc80050350 0x7efc80052810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7efca000d000 tx=0x7efca0000f00).stop 2020-07-18T03:25:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8063f90 msgr2=0x7efca80afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8063f90 0x7efca80afa80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7efca8063460 tx=0x7efc980079e0).stop 2020-07-18T03:25:59.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 shutdown_connections 2020-07-18T03:25:59.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8063f90 0x7efca80afa80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 --2- 172.21.15.47:0/2206064439 >> v2:172.21.15.18:3300/0 conn(0x7efca80ad330 0x7efca8145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 --2- 172.21.15.47:0/2206064439 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efc80050350 0x7efc80052810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 >> 172.21.15.47:0/2206064439 conn(0x7efca805f5a0 msgr2=0x7efca80b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 shutdown_connections 2020-07-18T03:25:59.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.767+0000 7efcaf500700 1 -- 172.21.15.47:0/2206064439 wait complete. 2020-07-18T03:25:59.779 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:25:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.895+0000 7feabeb86700 1 Processor -- start 2020-07-18T03:25:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.895+0000 7feabeb86700 1 -- start start 2020-07-18T03:25:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab80ad330 0x7feab8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feab80b59b0 0x7feab80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab8147de0 con 0x7feab80b6300 2020-07-18T03:25:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab80ae000 con 0x7feab80b59b0 2020-07-18T03:25:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab8148010 con 0x7feab80ad330 2020-07-18T03:25:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feab80b59b0 0x7feab80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabd123700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feab80b6300 0x7feab80b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40048/0 (socket says 172.21.15.47:40048) 2020-07-18T03:25:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabd123700 1 -- 172.21.15.47:0/1036775952 learned_addr learned my addr 172.21.15.47:0/1036775952 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:25:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabc922700 1 --2- 172.21.15.47:0/1036775952 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab80ad330 0x7feab8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab7fff700 1 -- 172.21.15.47:0/1036775952 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab80ad330 msgr2=0x7feab8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:25:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab7fff700 1 --2- 172.21.15.47:0/1036775952 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab80ad330 0x7feab8143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab7fff700 1 -- 172.21.15.47:0/1036775952 >> v1:172.21.15.47:6789/0 conn(0x7feab80b6300 legacy=0x7feab80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab7fff700 1 -- 172.21.15.47:0/1036775952 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab80ad710 con 0x7feab80b59b0 2020-07-18T03:25:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab7fff700 1 --2- 172.21.15.47:0/1036775952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b59b0 0x7feab80b5e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7feaac002f30 tx=0x7feaac006dd0).ready entity=mon.1 client_cookie=8bc99fd99516dc4d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab77fe700 1 -- 172.21.15.47:0/1036775952 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaac0097f0 con 0x7feab80b59b0 2020-07-18T03:25:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab77fe700 1 -- 172.21.15.47:0/1036775952 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac009950 con 0x7feab80b59b0 2020-07-18T03:25:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feab77fe700 1 -- 172.21.15.47:0/1036775952 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaac009cb0 con 0x7feab80b59b0 2020-07-18T03:25:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 -- 172.21.15.47:0/1036775952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b59b0 msgr2=0x7feab80b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.896+0000 7feabeb86700 1 --2- 172.21.15.47:0/1036775952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b59b0 0x7feab80b5e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7feaac002f30 tx=0x7feaac006dd0).stop 2020-07-18T03:25:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.897+0000 7feabeb86700 1 -- 172.21.15.47:0/1036775952 shutdown_connections 2020-07-18T03:25:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.897+0000 7feabeb86700 1 --2- 172.21.15.47:0/1036775952 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab80ad330 0x7feab8143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.897+0000 7feabeb86700 1 --2- 172.21.15.47:0/1036775952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b59b0 0x7feab80b5e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.897+0000 7feabeb86700 1 -- 172.21.15.47:0/1036775952 >> 172.21.15.47:0/1036775952 conn(0x7feab805f5a0 msgr2=0x7feab805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:25:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.897+0000 7feabeb86700 1 -- 172.21.15.47:0/1036775952 shutdown_connections 2020-07-18T03:25:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.897+0000 7feabeb86700 1 -- 172.21.15.47:0/1036775952 wait complete. 2020-07-18T03:25:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.898+0000 7feabeb86700 1 Processor -- start 2020-07-18T03:25:59.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.898+0000 7feabeb86700 1 -- start start 2020-07-18T03:25:59.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabeb86700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feab80ad330 0x7feab81395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabeb86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feab80b6300 0x7feab80b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabeb86700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab8139ab0 0x7feab813b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabeb86700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab80635b0 con 0x7feab80ad330 2020-07-18T03:25:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabeb86700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab8063850 con 0x7feab80b6300 2020-07-18T03:25:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabeb86700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab8063af0 con 0x7feab8139ab0 2020-07-18T03:25:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feab80b6300 0x7feab80b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feab80b6300 0x7feab80b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58964/0 (socket says 172.21.15.47:58964) 2020-07-18T03:25:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 -- 172.21.15.47:0/3648508328 learned_addr learned my addr 172.21.15.47:0/3648508328 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:25:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabd123700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab8139ab0 0x7feab813b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feabc922700 1 --2- 172.21.15.47:0/3648508328 >> v2:172.21.15.18:3300/0 conn(0x7feab80ad330 0x7feab81395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 -- 172.21.15.47:0/3648508328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab8139ab0 msgr2=0x7feab813b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab8139ab0 0x7feab813b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 -- 172.21.15.47:0/3648508328 >> v2:172.21.15.18:3300/0 conn(0x7feab80ad330 msgr2=0x7feab81395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:25:59.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 --2- 172.21.15.47:0/3648508328 >> v2:172.21.15.18:3300/0 conn(0x7feab80ad330 0x7feab81395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:25:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 -- 172.21.15.47:0/3648508328 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8147920 con 0x7feab80b6300 2020-07-18T03:25:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.899+0000 7feab7fff700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b6300 0x7feab80b2910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7feaac002f30 tx=0x7feaac015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.900+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaac01e460 con 0x7feab80b6300 2020-07-18T03:25:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.900+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac01e5c0 con 0x7feab80b6300 2020-07-18T03:25:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.900+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab80630a0 con 0x7feab80b6300 2020-07-18T03:25:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.900+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaac01e920 con 0x7feab80b6300 2020-07-18T03:25:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.900+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab814e150 con 0x7feab80b6300 2020-07-18T03:25:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.901+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab8063c60 con 0x7feab80b6300 2020-07-18T03:25:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.901+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7feaac025030 con 0x7feab80b6300 2020-07-18T03:25:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.901+0000 7feab4ff9700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea90050000 0x7fea900524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:25:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.901+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feaac069450 con 0x7feab80b6300 2020-07-18T03:25:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.901+0000 7feabc922700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea90050000 0x7fea900524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:25:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.901+0000 7feabc922700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea90050000 0x7fea900524c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7feaa800c000 tx=0x7feaa8008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:25:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:25:59.905+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7feaac041be0 con 0x7feab80b6300 2020-07-18T03:26:00.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.061+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feab814e390 con 0x7feab80b6300 2020-07-18T03:26:00.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.062+0000 7feab4ff9700 1 -- 172.21.15.47:0/3648508328 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7feaac045200 con 0x7feab80b6300 2020-07-18T03:26:00.065 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea90050000 msgr2=0x7fea900524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea90050000 0x7fea900524c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7feaa800c000 tx=0x7feaa8008040).stop 2020-07-18T03:26:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b6300 msgr2=0x7feab80b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b6300 0x7feab80b2910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7feaac002f30 tx=0x7feaac015000).stop 2020-07-18T03:26:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 shutdown_connections 2020-07-18T03:26:00.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea90050000 0x7fea900524c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feab8139ab0 0x7feab813b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 --2- 172.21.15.47:0/3648508328 >> v2:172.21.15.18:3300/0 conn(0x7feab80ad330 0x7feab81395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 --2- 172.21.15.47:0/3648508328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feab80b6300 0x7feab80b2910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.063+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 >> 172.21.15.47:0/3648508328 conn(0x7feab805f5a0 msgr2=0x7feab814e680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.064+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 shutdown_connections 2020-07-18T03:26:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.064+0000 7feabeb86700 1 -- 172.21.15.47:0/3648508328 wait complete. 2020-07-18T03:26:00.076 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.193+0000 7f5805393700 1 Processor -- start 2020-07-18T03:26:00.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.193+0000 7f5805393700 1 -- start start 2020-07-18T03:26:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f5805393700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f58000ad330 0x7f5800143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f5805393700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58000b3a10 0x7f58000b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f5805393700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5800147de0 con 0x7f58000b59b0 2020-07-18T03:26:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f5805393700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000ae000 con 0x7f58000b3a10 2020-07-18T03:26:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f5805393700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5800148010 con 0x7f58000ad330 2020-07-18T03:26:00.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58000b3a10 0x7f58000b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58000b3a10 0x7f58000b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58974/0 (socket says 172.21.15.47:58974) 2020-07-18T03:26:00.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 -- 172.21.15.47:0/1181420307 learned_addr learned my addr 172.21.15.47:0/1181420307 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:00.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57feffd700 1 --2- 172.21.15.47:0/1181420307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f58000ad330 0x7f5800143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 -- 172.21.15.47:0/1181420307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f58000ad330 msgr2=0x7f5800143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:00.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 --2- 172.21.15.47:0/1181420307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f58000ad330 0x7f5800143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 -- 172.21.15.47:0/1181420307 >> v1:172.21.15.18:6789/0 conn(0x7f58000b59b0 legacy=0x7f58000b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:00.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 -- 172.21.15.47:0/1181420307 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000ad710 con 0x7f58000b3a10 2020-07-18T03:26:00.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57ff7fe700 1 --2- 172.21.15.47:0/1181420307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b3a10 0x7f58000b3ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f58000ae4e0 tx=0x7f57f8009270).ready entity=mon.1 client_cookie=1585f367c446ebcb server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:00.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57fdffb700 1 -- 172.21.15.47:0/1181420307 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57f800bbc0 con 0x7f58000b3a10 2020-07-18T03:26:00.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57fdffb700 1 -- 172.21.15.47:0/1181420307 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f800bd20 con 0x7f58000b3a10 2020-07-18T03:26:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.194+0000 7f57fdffb700 1 -- 172.21.15.47:0/1181420307 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57f8011400 con 0x7f58000b3a10 2020-07-18T03:26:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 -- 172.21.15.47:0/1181420307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b3a10 msgr2=0x7f58000b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 --2- 172.21.15.47:0/1181420307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b3a10 0x7f58000b3ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f58000ae4e0 tx=0x7f57f8009270).stop 2020-07-18T03:26:00.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 -- 172.21.15.47:0/1181420307 shutdown_connections 2020-07-18T03:26:00.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 --2- 172.21.15.47:0/1181420307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b3a10 0x7f58000b3ea0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 --2- 172.21.15.47:0/1181420307 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f58000ad330 0x7f5800143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 -- 172.21.15.47:0/1181420307 >> 172.21.15.47:0/1181420307 conn(0x7f580005f5a0 msgr2=0x7f580005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:00.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 -- 172.21.15.47:0/1181420307 shutdown_connections 2020-07-18T03:26:00.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.195+0000 7f5805393700 1 -- 172.21.15.47:0/1181420307 wait complete. 2020-07-18T03:26:00.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.196+0000 7f5805393700 1 Processor -- start 2020-07-18T03:26:00.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.196+0000 7f5805393700 1 -- start start 2020-07-18T03:26:00.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f5805393700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f58000ad330 0x7f5800140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f5805393700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58000b59b0 0x7f5800141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f5805393700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58000653d0 con 0x7f5800147760 2020-07-18T03:26:00.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f5805393700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5800065640 con 0x7f58000ad330 2020-07-18T03:26:00.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f5805393700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000658e0 con 0x7f58000b59b0 2020-07-18T03:26:00.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57ff7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5800147760 0x7f5800061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40070/0 (socket says 172.21.15.47:40070) 2020-07-18T03:26:00.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57fe7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58000b59b0 0x7f5800141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57ff7fe700 1 -- 172.21.15.47:0/445129190 learned_addr learned my addr 172.21.15.47:0/445129190 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:00.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57feffd700 1 --2- 172.21.15.47:0/445129190 >> v2:172.21.15.18:3300/0 conn(0x7f58000ad330 0x7f5800140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57fe7fc700 1 -- 172.21.15.47:0/445129190 >> v2:172.21.15.18:3300/0 conn(0x7f58000ad330 msgr2=0x7f5800140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57fe7fc700 1 --2- 172.21.15.47:0/445129190 >> v2:172.21.15.18:3300/0 conn(0x7f58000ad330 0x7f5800140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57fe7fc700 1 -- 172.21.15.47:0/445129190 >> v1:172.21.15.47:6789/0 conn(0x7f5800147760 legacy=0x7f5800061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143007105 0 0) 0x7f58000653d0 con 0x7f5800147760 2020-07-18T03:26:00.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57fe7fc700 1 -- 172.21.15.47:0/445129190 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800065090 con 0x7f58000b59b0 2020-07-18T03:26:00.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.197+0000 7f57fe7fc700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b59b0 0x7f5800141cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f58000b1e30 tx=0x7f57f0007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.198+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57f000f470 con 0x7f58000b59b0 2020-07-18T03:26:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.198+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f000f5d0 con 0x7f58000b59b0 2020-07-18T03:26:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.198+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5800140820 con 0x7f58000b59b0 2020-07-18T03:26:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.198+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57f000f930 con 0x7f58000b59b0 2020-07-18T03:26:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.198+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58001421b0 con 0x7f58000b59b0 2020-07-18T03:26:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.199+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f57f0019030 con 0x7f58000b59b0 2020-07-18T03:26:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.199+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5800147bc0 con 0x7f58000b59b0 2020-07-18T03:26:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.199+0000 7f57e6ffd700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f57dc04fdf0 0x7f57dc0522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.199+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f57f00671b0 con 0x7f58000b59b0 2020-07-18T03:26:00.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.199+0000 7f57feffd700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f57dc04fdf0 0x7f57dc0522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.199+0000 7f57feffd700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f57dc04fdf0 0x7f57dc0522b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f57f4008000 tx=0x7f57f4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:00.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.202+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f57f003f940 con 0x7f58000b59b0 2020-07-18T03:26:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.360+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58000b6610 con 0x7f58000b59b0 2020-07-18T03:26:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.361+0000 7f57e6ffd700 1 -- 172.21.15.47:0/445129190 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f57f0042f60 con 0x7f58000b59b0 2020-07-18T03:26:00.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f57dc04fdf0 msgr2=0x7f57dc0522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f57dc04fdf0 0x7f57dc0522b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f57f4008000 tx=0x7f57f4008040).stop 2020-07-18T03:26:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b59b0 msgr2=0x7f5800141cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b59b0 0x7f5800141cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f58000b1e30 tx=0x7f57f0007b70).stop 2020-07-18T03:26:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 shutdown_connections 2020-07-18T03:26:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f57dc04fdf0 0x7f57dc0522b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 --2- 172.21.15.47:0/445129190 >> v2:172.21.15.18:3300/0 conn(0x7f58000ad330 0x7f5800140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 --2- 172.21.15.47:0/445129190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58000b59b0 0x7f5800141cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 >> 172.21.15.47:0/445129190 conn(0x7f580005f5a0 msgr2=0x7f580014a900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 shutdown_connections 2020-07-18T03:26:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.362+0000 7f5805393700 1 -- 172.21.15.47:0/445129190 wait complete. 2020-07-18T03:26:00.375 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:26:00.375 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:00.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.488+0000 7efed248b700 1 Processor -- start 2020-07-18T03:26:00.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.488+0000 7efed248b700 1 -- start start 2020-07-18T03:26:00.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efed248b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc147a00 0x7efecc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efed248b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efecc1433e0 0x7efecc143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efed248b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc147330 con 0x7efecc0ad100 2020-07-18T03:26:00.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efed248b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0ae530 con 0x7efecc1433e0 2020-07-18T03:26:00.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efed248b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0b57e0 con 0x7efecc147a00 2020-07-18T03:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc147a00 0x7efecc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efed0a28700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efecc1433e0 0x7efecc143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc147a00 0x7efecc147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51584/0 (socket says 172.21.15.47:51584) 2020-07-18T03:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.489+0000 7efecb7fe700 1 -- 172.21.15.47:0/2623104828 learned_addr learned my addr 172.21.15.47:0/2623104828 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed0a28700 1 -- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc147a00 msgr2=0x7efecc147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:00.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed0a28700 1 --2- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc147a00 0x7efecc147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed0a28700 1 -- 172.21.15.47:0/2623104828 >> v1:172.21.15.18:6789/0 conn(0x7efecc0ad100 legacy=0x7efecc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed0a28700 1 -- 172.21.15.47:0/2623104828 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0b5a80 con 0x7efecc1433e0 2020-07-18T03:26:00.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efecaffd700 1 -- 172.21.15.47:0/2623104828 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423512659 0 0) 0x7efecc147330 con 0x7efecc0ad100 2020-07-18T03:26:00.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed0a28700 1 --2- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efecc1433e0 0x7efecc143850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efecc0b0530 tx=0x7efec40091d0).ready entity=mon.0 client_cookie=8c026a45b516208 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efecaffd700 1 -- 172.21.15.47:0/2623104828 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efec400b990 con 0x7efecc1433e0 2020-07-18T03:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efecaffd700 1 -- 172.21.15.47:0/2623104828 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec400baf0 con 0x7efecc1433e0 2020-07-18T03:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efecaffd700 1 -- 172.21.15.47:0/2623104828 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efec400bc60 con 0x7efecc1433e0 2020-07-18T03:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 -- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efecc1433e0 msgr2=0x7efecc143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 --2- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efecc1433e0 0x7efecc143850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efecc0b0530 tx=0x7efec40091d0).stop 2020-07-18T03:26:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 -- 172.21.15.47:0/2623104828 shutdown_connections 2020-07-18T03:26:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 --2- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efecc1433e0 0x7efecc143850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 --2- 172.21.15.47:0/2623104828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc147a00 0x7efecc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 -- 172.21.15.47:0/2623104828 >> 172.21.15.47:0/2623104828 conn(0x7efecc0b9340 msgr2=0x7efecc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 -- 172.21.15.47:0/2623104828 shutdown_connections 2020-07-18T03:26:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.490+0000 7efed248b700 1 -- 172.21.15.47:0/2623104828 wait complete. 2020-07-18T03:26:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.492+0000 7efed248b700 1 Processor -- start 2020-07-18T03:26:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.492+0000 7efed248b700 1 -- start start 2020-07-18T03:26:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efed248b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc0ad100 0x7efecc0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efed248b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc131530 con 0x7efecc1428c0 2020-07-18T03:26:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efed248b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc062980 con 0x7efecc147a00 2020-07-18T03:26:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efed248b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1317d0 con 0x7efecc0ad100 2020-07-18T03:26:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efecc147a00 0x7efecc1309f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40082/0 (socket says 172.21.15.47:40082) 2020-07-18T03:26:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecb7fe700 1 -- 172.21.15.47:0/705257112 learned_addr learned my addr 172.21.15.47:0/705257112 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecbfff700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc0ad100 0x7efecc0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951955896 0 0) 0x7efecc062980 con 0x7efecc147a00 2020-07-18T03:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea4003240 con 0x7efecc147a00 2020-07-18T03:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259773132 0 0) 0x7efecc131530 con 0x7efecc1428c0 2020-07-18T03:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecbfff700 1 -- 172.21.15.47:0/705257112 >> v1:172.21.15.47:6789/0 conn(0x7efecc147a00 legacy=0x7efecc1309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecbfff700 1 -- 172.21.15.47:0/705257112 >> v1:172.21.15.18:6789/0 conn(0x7efecc1428c0 legacy=0x7efecc062e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecbfff700 1 -- 172.21.15.47:0/705257112 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc131100 con 0x7efecc0ad100 2020-07-18T03:26:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.493+0000 7efecbfff700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc0ad100 0x7efecc0b1540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7efec0007000 tx=0x7efebc0078f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.494+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efebc00f470 con 0x7efecc0ad100 2020-07-18T03:26:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.494+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00f5d0 con 0x7efecc0ad100 2020-07-18T03:26:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.494+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc063530 con 0x7efecc0ad100 2020-07-18T03:26:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.494+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efebc00f930 con 0x7efecc0ad100 2020-07-18T03:26:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.494+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc064490 con 0x7efecc0ad100 2020-07-18T03:26:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.495+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc131350 con 0x7efecc0ad100 2020-07-18T03:26:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.495+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7efebc019030 con 0x7efecc0ad100 2020-07-18T03:26:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.495+0000 7efeabfff700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efea40502c0 0x7efea4052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.495+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efebc066fd0 con 0x7efecc0ad100 2020-07-18T03:26:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.495+0000 7efecb7fe700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efea40502c0 0x7efea4052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.495+0000 7efecb7fe700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efea40502c0 0x7efea4052780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7efec0003070 tx=0x7efec0006e90).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.499+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7efebc03f6c0 con 0x7efecc0ad100 2020-07-18T03:26:00.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.709+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efecc057450 con 0x7efecc0ad100 2020-07-18T03:26:00.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.710+0000 7efeabfff700 1 -- 172.21.15.47:0/705257112 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2124 (secure 0 0 0) 0x7efebc042ce0 con 0x7efecc0ad100 2020-07-18T03:26:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:00.712 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:49.581963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4387":{"gid":4387,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.47:6835/308881804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":308881804},{"type":"v1","addr":"172.21.15.47:6835","nonce":308881804}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:26:00.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efea40502c0 msgr2=0x7efea4052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efea40502c0 0x7efea4052780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7efec0003070 tx=0x7efec0006e90).stop 2020-07-18T03:26:00.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc0ad100 msgr2=0x7efecc0b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:00.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc0ad100 0x7efecc0b1540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7efec0007000 tx=0x7efebc0078f0).stop 2020-07-18T03:26:00.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 shutdown_connections 2020-07-18T03:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efecc0ad100 0x7efecc0b1540 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 --2- 172.21.15.47:0/705257112 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efea40502c0 0x7efea4052780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 >> 172.21.15.47:0/705257112 conn(0x7efecc0b9340 msgr2=0x7efecc14ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 shutdown_connections 2020-07-18T03:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:00.711+0000 7efed248b700 1 -- 172.21.15.47:0/705257112 wait complete. 2020-07-18T03:26:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2020-07-18T03:26:00.925 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:25:45.899311+0000', 'modified': '2020-07-18T03:25:49.581963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4387': {'gid': 4387, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.47:6835/308881804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 308881804}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 308881804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:00.926 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:00.926 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:01.065 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4387,"want_state":"up:active","state":"up:active","rank_uptime":12.472237221,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":12.728986925999999} 2020-07-18T03:26:01.066 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4387, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 12.472237221, "state": "up:active", "uptime": 12.728986926, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:01.066 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:26:01.067 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:26:01.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.068 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:26:01.085 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:01.085 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:01.086 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:01.086 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:01.086 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:01.087 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:01.087 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:01.087 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:01.088 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:01.088 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:01.088 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:01.089 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42830sec preferred_lft 42830sec 2020-07-18T03:26:01.089 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:26:01.089 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:01.090 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:01.090 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:01.090 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:01.090 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:01.091 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:26:01.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.092 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:26:01.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.204 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:26:01.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.239 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:26:01.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.311 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:26:01.383 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:26:01.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.384 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:26:01.453 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:26:01.453 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:01.453 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:01.454 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:01.454 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:26:01.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.455 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:01.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.586 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:01.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.620 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:01.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.742 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:26:01.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.759 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:26:01.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.813 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:26:01.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.886 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-07-18T03:26:01.967 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:26:01.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:01.968 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-07-18T03:26:02.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:02.046 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:26:02.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:02.149 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:26:02.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:02.205 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:26:02.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:02.252 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:26:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:02.303 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:26:02.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:02.344 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:26:02.428 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:26:02.428 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:26:02.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:02.429 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:02.444 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:02.538 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:02.539 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:02.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:02.573 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:02.643 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:26:02.643 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:02.685 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:02.717 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:02.717 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:02.789 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:02.790 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:26:02.788+0000 7f3fb82f11c0 -1 init, newargv = 0x55dd54dc5df0 newargc=15 2020-07-18T03:26:02.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[28698]: starting ceph client 2020-07-18T03:26:02.791 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:02.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:02.792 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:02.795 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[28698]: starting fuse 2020-07-18T03:26:02.812 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:26:02.813 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:26:02.814 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:26:02.814 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:26:02.815 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:26:02.817 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:26:02.824 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:26:02.824 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:02.825 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:26:02.826 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:26:02.827 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:26:02.828 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:26:02.828 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:02.829 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:26:02.829 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:26:02.830 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:26:02.830 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:26:02.831 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:02.832 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:26:02.832 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:26:02.833 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:26:02.833 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:26:02.834 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:26:02.834 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:26:02.835 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:26:02.836 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:26:02.837 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:02.838 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:26:02.839 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:26:02.943 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.28698.asok 2020-07-18T03:26:03.124 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28698.asok 2020-07-18T03:26:03.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:03.125 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28698.asok status 2020-07-18T03:26:03.276 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:26:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:26:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:26:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:26:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:26:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:26:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:26:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "28698", 2020-07-18T03:26:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:26:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:26:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:26:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4720, 2020-07-18T03:26:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:26:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:26:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:26:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4720 2020-07-18T03:26:03.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:03.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:03.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:03.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:26:03.280 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2689953610 2020-07-18T03:26:03.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:03.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:03.281 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:03.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:03.281 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:26:03.282 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2689953610 2020-07-18T03:26:03.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:03.283 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4720 192.168.0.1:0/2689953610", 2020-07-18T03:26:03.283 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2689953610", 2020-07-18T03:26:03.283 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:26:03.283 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 6, 2020-07-18T03:26:03.283 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 24, 2020-07-18T03:26:03.283 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:26:03.284 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:26:03.284 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:26:03.285 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:03.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.397+0000 7ff8f16bf700 1 Processor -- start 2020-07-18T03:26:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.397+0000 7ff8f16bf700 1 -- start start 2020-07-18T03:26:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8f16bf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8ec148ba0 0x7ff8ec149000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8f16bf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff8ec065120 0x7ff8ec065570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8f16bf700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec13f400 con 0x7ff8ec148490 2020-07-18T03:26:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8f16bf700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ec145db0 con 0x7ff8ec065120 2020-07-18T03:26:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8f16bf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ec060710 con 0x7ff8ec148ba0 2020-07-18T03:26:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8ea7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8ec148ba0 0x7ff8ec149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8ea7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8ec148ba0 0x7ff8ec149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59014/0 (socket says 172.21.15.47:59014) 2020-07-18T03:26:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.398+0000 7ff8ea7fc700 1 -- 172.21.15.47:0/125309026 learned_addr learned my addr 172.21.15.47:0/125309026 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8eb7fe700 1 --2- 172.21.15.47:0/125309026 >> v2:172.21.15.18:3300/0 conn(0x7ff8ec065120 0x7ff8ec065570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8ea7fc700 1 -- 172.21.15.47:0/125309026 >> v2:172.21.15.18:3300/0 conn(0x7ff8ec065120 msgr2=0x7ff8ec065570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8ea7fc700 1 --2- 172.21.15.47:0/125309026 >> v2:172.21.15.18:3300/0 conn(0x7ff8ec065120 0x7ff8ec065570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8ea7fc700 1 -- 172.21.15.47:0/125309026 >> v1:172.21.15.18:6789/0 conn(0x7ff8ec148490 legacy=0x7ff8ec05f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8ea7fc700 1 -- 172.21.15.47:0/125309026 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec1453d0 con 0x7ff8ec148ba0 2020-07-18T03:26:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8ea7fc700 1 --2- 172.21.15.47:0/125309026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ec148ba0 0x7ff8ec149000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff8ec05cad0 tx=0x7ff8dc006e90).ready entity=mon.1 client_cookie=44f91feeeb5b93c9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8e9ffb700 1 -- 172.21.15.47:0/125309026 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc009930 con 0x7ff8ec148ba0 2020-07-18T03:26:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8e9ffb700 1 -- 172.21.15.47:0/125309026 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc009a90 con 0x7ff8ec148ba0 2020-07-18T03:26:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8e9ffb700 1 -- 172.21.15.47:0/125309026 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc009e30 con 0x7ff8ec148ba0 2020-07-18T03:26:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 -- 172.21.15.47:0/125309026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ec148ba0 msgr2=0x7ff8ec149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/125309026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ec148ba0 0x7ff8ec149000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff8ec05cad0 tx=0x7ff8dc006e90).stop 2020-07-18T03:26:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 -- 172.21.15.47:0/125309026 shutdown_connections 2020-07-18T03:26:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/125309026 >> v2:172.21.15.18:3300/0 conn(0x7ff8ec065120 0x7ff8ec065570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/125309026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ec148ba0 0x7ff8ec149000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 -- 172.21.15.47:0/125309026 >> 172.21.15.47:0/125309026 conn(0x7ff8ec0b9400 msgr2=0x7ff8ec0b9810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 -- 172.21.15.47:0/125309026 shutdown_connections 2020-07-18T03:26:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.399+0000 7ff8f16bf700 1 -- 172.21.15.47:0/125309026 wait complete. 2020-07-18T03:26:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8f16bf700 1 Processor -- start 2020-07-18T03:26:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8f16bf700 1 -- start start 2020-07-18T03:26:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8f16bf700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8ec065120 0x7ff8ec064980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8f16bf700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec0b2640 con 0x7ff8ec148490 2020-07-18T03:26:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8f16bf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec0b2870 con 0x7ff8ec0b3250 2020-07-18T03:26:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8f16bf700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec0b2b10 con 0x7ff8ec065120 2020-07-18T03:26:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8eb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8ec0b3250 0x7ff8ec129f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40110/0 (socket says 172.21.15.47:40110) 2020-07-18T03:26:03.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.401+0000 7ff8eb7fe700 1 -- 172.21.15.47:0/3024993956 learned_addr learned my addr 172.21.15.47:0/3024993956 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:03.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8eaffd700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8ec065120 0x7ff8ec064980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144392738 0 0) 0x7ff8ec0b2870 con 0x7ff8ec0b3250 2020-07-18T03:26:03.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c8003240 con 0x7ff8ec0b3250 2020-07-18T03:26:03.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979522887 0 0) 0x7ff8ec0b2640 con 0x7ff8ec148490 2020-07-18T03:26:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ec0b2870 con 0x7ff8ec148490 2020-07-18T03:26:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8eaffd700 1 -- 172.21.15.47:0/3024993956 >> v1:172.21.15.47:6789/0 conn(0x7ff8ec0b3250 legacy=0x7ff8ec129f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 718600065 0 0) 0x7ff8c8003240 con 0x7ff8ec0b3250 2020-07-18T03:26:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8eaffd700 1 -- 172.21.15.47:0/3024993956 >> v1:172.21.15.18:6789/0 conn(0x7ff8ec148490 legacy=0x7ff8ec141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8eaffd700 1 -- 172.21.15.47:0/3024993956 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0b2db0 con 0x7ff8ec065120 2020-07-18T03:26:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8eaffd700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8ec065120 0x7ff8ec064980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff8e400a000 tx=0x7ff8e0007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4001810 con 0x7ff8ec065120 2020-07-18T03:26:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e0007240 con 0x7ff8ec065120 2020-07-18T03:26:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec12a650 con 0x7ff8ec065120 2020-07-18T03:26:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e00073b0 con 0x7ff8ec065120 2020-07-18T03:26:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.402+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec0b4190 con 0x7ff8ec065120 2020-07-18T03:26:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.403+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec057450 con 0x7ff8ec065120 2020-07-18T03:26:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.404+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff8e0010070 con 0x7ff8ec065120 2020-07-18T03:26:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.404+0000 7ff8d2ffd700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8c8050470 0x7ff8c8052930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.404+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff8e0066e40 con 0x7ff8ec065120 2020-07-18T03:26:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.404+0000 7ff8ea7fc700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8c8050470 0x7ff8c8052930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.404+0000 7ff8ea7fc700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8c8050470 0x7ff8c8052930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff8dc00b180 tx=0x7ff8dc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.408+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff8e003f5d0 con 0x7ff8ec065120 2020-07-18T03:26:03.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.569+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8ec068c50 con 0x7ff8ec065120 2020-07-18T03:26:03.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.571+0000 7ff8d2ffd700 1 -- 172.21.15.47:0/3024993956 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7ff8e0042bf0 con 0x7ff8ec065120 2020-07-18T03:26:03.572 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:03.573 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8c8050470 msgr2=0x7ff8c8052930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8c8050470 0x7ff8c8052930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff8dc00b180 tx=0x7ff8dc015000).stop 2020-07-18T03:26:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8ec065120 msgr2=0x7ff8ec064980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8ec065120 0x7ff8ec064980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff8e400a000 tx=0x7ff8e0007b50).stop 2020-07-18T03:26:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 shutdown_connections 2020-07-18T03:26:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8c8050470 0x7ff8c8052930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 --2- 172.21.15.47:0/3024993956 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8ec065120 0x7ff8ec064980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 >> 172.21.15.47:0/3024993956 conn(0x7ff8ec0b9400 msgr2=0x7ff8ec061bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 shutdown_connections 2020-07-18T03:26:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.572+0000 7ff8f16bf700 1 -- 172.21.15.47:0/3024993956 wait complete. 2020-07-18T03:26:03.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:03.592 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.709+0000 7f6d9ce2d700 1 Processor -- start 2020-07-18T03:26:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.709+0000 7f6d9ce2d700 1 -- start start 2020-07-18T03:26:03.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9ce2d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6d980b3a10 0x7f6d980b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9ce2d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d98147de0 con 0x7f6d980b59b0 2020-07-18T03:26:03.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9ce2d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d980ae000 con 0x7f6d980ad330 2020-07-18T03:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9ce2d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d98148010 con 0x7f6d980b3a10 2020-07-18T03:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6d980ad330 0x7f6d98143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40114/0 (socket says 172.21.15.47:40114) 2020-07-18T03:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9659c700 1 -- 172.21.15.47:0/1341188253 learned_addr learned my addr 172.21.15.47:0/1341188253 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d96d9d700 1 --2- 172.21.15.47:0/1341188253 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b3a10 0x7f6d980b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686910191 0 0) 0x7f6d980ae000 con 0x7f6d980ad330 2020-07-18T03:26:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d7c003240 con 0x7f6d980ad330 2020-07-18T03:26:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 248090941 0 0) 0x7f6d7c003240 con 0x7f6d980ad330 2020-07-18T03:26:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b3a10 msgr2=0x7f6d980b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9559a700 1 --2- 172.21.15.47:0/1341188253 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b3a10 0x7f6d980b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.710+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 >> v1:172.21.15.18:6789/0 conn(0x7f6d980b59b0 legacy=0x7f6d980b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d980ad710 con 0x7f6d980ad330 2020-07-18T03:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f6d8c0016f0 con 0x7f6d980ad330 2020-07-18T03:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 >> v1:172.21.15.47:6789/0 conn(0x7f6d980ad330 legacy=0x7f6d98143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d7c003730 0x7f6d7c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d7c0044e0 0x7f6d7c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d7c004e50 0x7f6d7c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98148010 con 0x7f6d7c003730 2020-07-18T03:26:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c004350 con 0x7f6d7c0044e0 2020-07-18T03:26:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c00bc50 con 0x7f6d7c004e50 2020-07-18T03:26:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d7c0044e0 0x7f6d7c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9659c700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d7c003730 0x7f6d7c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d96d9d700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d7c004e50 0x7f6d7c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 -- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d7c004e50 msgr2=0x7f6d7c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d7c004e50 0x7f6d7c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 -- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d7c003730 msgr2=0x7f6d7c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d7c003730 0x7f6d7c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 -- 172.21.15.47:0/1341188253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c003240 con 0x7f6d7c0044e0 2020-07-18T03:26:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d96d9d700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d7c004e50 0x7f6d7c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d96d9d700 1 -- 172.21.15.47:0/1341188253 reap_dead start 2020-07-18T03:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d7c0044e0 0x7f6d7c004960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f6d980ae4e0 tx=0x7f6d88007ef0).ready entity=mon.1 client_cookie=c004c70ff341ce6e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d8800cb90 con 0x7f6d7c0044e0 2020-07-18T03:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800ccf0 con 0x7f6d7c0044e0 2020-07-18T03:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9559a700 1 -- 172.21.15.47:0/1341188253 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d88011660 con 0x7f6d7c0044e0 2020-07-18T03:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1341188253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d7c0044e0 msgr2=0x7f6d7c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d7c0044e0 0x7f6d7c004960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f6d980ae4e0 tx=0x7f6d88007ef0).stop 2020-07-18T03:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1341188253 shutdown_connections 2020-07-18T03:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1341188253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d7c0044e0 0x7f6d7c004960 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.711+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1341188253 >> 172.21.15.47:0/1341188253 conn(0x7f6d9805f5a0 msgr2=0x7f6d9805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.712+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1341188253 shutdown_connections 2020-07-18T03:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.712+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1341188253 wait complete. 2020-07-18T03:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.713+0000 7f6d9ce2d700 1 Processor -- start 2020-07-18T03:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.713+0000 7f6d9ce2d700 1 -- start start 2020-07-18T03:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9ce2d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d980b1d60 0x7f6d9813cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9ce2d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6d980b33e0 0x7f6d9813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9ce2d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9813c240 con 0x7f6d9813d9d0 2020-07-18T03:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9ce2d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9813c4e0 con 0x7f6d980b33e0 2020-07-18T03:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9ce2d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9813c780 con 0x7f6d980b1d60 2020-07-18T03:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d96d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6d9813d9d0 0x7f6d98137970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40128/0 (socket says 172.21.15.47:40128) 2020-07-18T03:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d96d9d700 1 -- 172.21.15.47:0/1070792927 learned_addr learned my addr 172.21.15.47:0/1070792927 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9659c700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d980b1d60 0x7f6d9813cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1070792927 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b33e0 0x7f6d9813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935907738 0 0) 0x7f6d9813c240 con 0x7f6d9813d9d0 2020-07-18T03:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d6c003240 con 0x7f6d9813d9d0 2020-07-18T03:26:03.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9659c700 1 -- 172.21.15.47:0/1070792927 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b33e0 msgr2=0x7f6d9813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9659c700 1 --2- 172.21.15.47:0/1070792927 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b33e0 0x7f6d9813d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9659c700 1 -- 172.21.15.47:0/1070792927 >> v1:172.21.15.47:6789/0 conn(0x7f6d9813d9d0 legacy=0x7f6d98137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2886573210 0 0) 0x7f6d6c003240 con 0x7f6d9813d9d0 2020-07-18T03:26:03.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9659c700 1 -- 172.21.15.47:0/1070792927 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98138080 con 0x7f6d980b1d60 2020-07-18T03:26:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d9659c700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d980b1d60 0x7f6d9813cc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6d9000b7a0 tx=0x7f6d8c007780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.714+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9000b600 con 0x7f6d980b1d60 2020-07-18T03:26:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.715+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c00f820 con 0x7f6d980b1d60 2020-07-18T03:26:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.715+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d8c00fb80 con 0x7f6d980b1d60 2020-07-18T03:26:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.715+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d981382f0 con 0x7f6d980b1d60 2020-07-18T03:26:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.715+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9813af60 con 0x7f6d980b1d60 2020-07-18T03:26:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.716+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d980b0320 con 0x7f6d980b1d60 2020-07-18T03:26:03.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.716+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6d8c001190 con 0x7f6d980b1d60 2020-07-18T03:26:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.716+0000 7f6d867fc700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6d6c050360 0x7f6d6c052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.716+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6d8c067b30 con 0x7f6d980b1d60 2020-07-18T03:26:03.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.716+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6d6c050360 0x7f6d6c052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.716+0000 7f6d95d9b700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6d6c050360 0x7f6d6c052820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f6d8801b000 tx=0x7f6d88018000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.720+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6d8c0402c0 con 0x7f6d980b1d60 2020-07-18T03:26:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.876+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6d9805b120 con 0x7f6d980b1d60 2020-07-18T03:26:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.877+0000 7f6d867fc700 1 -- 172.21.15.47:0/1070792927 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6d8c0438e0 con 0x7f6d980b1d60 2020-07-18T03:26:03.879 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6d6c050360 msgr2=0x7f6d6c052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6d6c050360 0x7f6d6c052820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f6d8801b000 tx=0x7f6d88018000).stop 2020-07-18T03:26:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d980b1d60 msgr2=0x7f6d9813cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d980b1d60 0x7f6d9813cc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6d9000b7a0 tx=0x7f6d8c007780).stop 2020-07-18T03:26:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 shutdown_connections 2020-07-18T03:26:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6d6c050360 0x7f6d6c052820 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1070792927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6d980b1d60 0x7f6d9813cc20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 --2- 172.21.15.47:0/1070792927 >> v2:172.21.15.18:3300/0 conn(0x7f6d980b33e0 0x7f6d9813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 >> 172.21.15.47:0/1070792927 conn(0x7f6d9805f5a0 msgr2=0x7f6d980b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 shutdown_connections 2020-07-18T03:26:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:03.878+0000 7f6d9ce2d700 1 -- 172.21.15.47:0/1070792927 wait complete. 2020-07-18T03:26:03.891 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:04.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.006+0000 7f1f24e46700 1 Processor -- start 2020-07-18T03:26:04.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.006+0000 7f1f24e46700 1 -- start start 2020-07-18T03:26:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.007+0000 7f1f24e46700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f200ad330 0x7f1f20143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.007+0000 7f1f24e46700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f200b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.007+0000 7f1f24e46700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f20147de0 con 0x7f1f200b6300 2020-07-18T03:26:04.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.007+0000 7f1f24e46700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200ae000 con 0x7f1f200ad330 2020-07-18T03:26:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.007+0000 7f1f24e46700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20148010 con 0x7f1f200b59b0 2020-07-18T03:26:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1dd9b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f200b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1dd9b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f200b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51642/0 (socket says 172.21.15.47:51642) 2020-07-18T03:26:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1e59c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f200ad330 0x7f1f20143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1dd9b700 1 -- 172.21.15.47:0/4029038063 learned_addr learned my addr 172.21.15.47:0/4029038063 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1e59c700 1 -- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 msgr2=0x7f1f200b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1e59c700 1 --2- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f200b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1e59c700 1 -- 172.21.15.47:0/4029038063 >> v1:172.21.15.18:6789/0 conn(0x7f1f200b6300 legacy=0x7f1f200b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1e59c700 1 -- 172.21.15.47:0/4029038063 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200ad710 con 0x7f1f200ad330 2020-07-18T03:26:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1d59a700 1 -- 172.21.15.47:0/4029038063 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221013769 0 0) 0x7f1f20147de0 con 0x7f1f200b6300 2020-07-18T03:26:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1dd9b700 1 --2- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f200b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1e59c700 1 --2- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f200ad330 0x7f1f20143640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1f14002f30 tx=0x7f1f14006dd0).ready entity=mon.0 client_cookie=6454ebc5f5342865 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1d59a700 1 -- 172.21.15.47:0/4029038063 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f14009760 con 0x7f1f200ad330 2020-07-18T03:26:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1d59a700 1 -- 172.21.15.47:0/4029038063 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f140098c0 con 0x7f1f200ad330 2020-07-18T03:26:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f24e46700 1 -- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f200ad330 msgr2=0x7f1f20143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f1d59a700 1 -- 172.21.15.47:0/4029038063 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f14009c20 con 0x7f1f200ad330 2020-07-18T03:26:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f24e46700 1 --2- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f200ad330 0x7f1f20143640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f1f14002f30 tx=0x7f1f14006dd0).stop 2020-07-18T03:26:04.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f24e46700 1 -- 172.21.15.47:0/4029038063 shutdown_connections 2020-07-18T03:26:04.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.008+0000 7f1f24e46700 1 --2- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f200ad330 0x7f1f20143640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.009+0000 7f1f24e46700 1 --2- 172.21.15.47:0/4029038063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f200b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.009+0000 7f1f24e46700 1 -- 172.21.15.47:0/4029038063 >> 172.21.15.47:0/4029038063 conn(0x7f1f2005f5a0 msgr2=0x7f1f2005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.009+0000 7f1f24e46700 1 -- 172.21.15.47:0/4029038063 shutdown_connections 2020-07-18T03:26:04.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.009+0000 7f1f24e46700 1 -- 172.21.15.47:0/4029038063 wait complete. 2020-07-18T03:26:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.010+0000 7f1f24e46700 1 Processor -- start 2020-07-18T03:26:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.010+0000 7f1f24e46700 1 -- start start 2020-07-18T03:26:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f24e46700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f201476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f24e46700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f2013f040 0x7f1f2013b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f24e46700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f20143640 con 0x7f1f200b6300 2020-07-18T03:26:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f24e46700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f201438e0 con 0x7f1f2013f040 2020-07-18T03:26:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f24e46700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20143b80 con 0x7f1f200b59b0 2020-07-18T03:26:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1dd9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1f200b6300 0x7f1f20141ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40138/0 (socket says 172.21.15.47:40138) 2020-07-18T03:26:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1dd9b700 1 -- 172.21.15.47:0/3140376191 learned_addr learned my addr 172.21.15.47:0/3140376191 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1e59c700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f201476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1ed9d700 1 --2- 172.21.15.47:0/3140376191 >> v2:172.21.15.47:3300/0 conn(0x7f1f2013f040 0x7f1f2013b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426896060 0 0) 0x7f1f20143640 con 0x7f1f200b6300 2020-07-18T03:26:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef4003240 con 0x7f1f200b6300 2020-07-18T03:26:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1e59c700 1 -- 172.21.15.47:0/3140376191 >> v2:172.21.15.47:3300/0 conn(0x7f1f2013f040 msgr2=0x7f1f2013b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1e59c700 1 --2- 172.21.15.47:0/3140376191 >> v2:172.21.15.47:3300/0 conn(0x7f1f2013f040 0x7f1f2013b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1e59c700 1 -- 172.21.15.47:0/3140376191 >> v1:172.21.15.47:6789/0 conn(0x7f1f200b6300 legacy=0x7f1f20141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1e59c700 1 -- 172.21.15.47:0/3140376191 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f201422b0 con 0x7f1f200b59b0 2020-07-18T03:26:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1ed9d700 1 --2- 172.21.15.47:0/3140376191 >> v2:172.21.15.47:3300/0 conn(0x7f1f2013f040 0x7f1f2013b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.011+0000 7f1f1e59c700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f201476b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1f14002f30 tx=0x7f1f14015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.012+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f14021070 con 0x7f1f200b59b0 2020-07-18T03:26:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.012+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1401c5c0 con 0x7f1f200b59b0 2020-07-18T03:26:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.012+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f20147ba0 con 0x7f1f200b59b0 2020-07-18T03:26:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.012+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f1401c920 con 0x7f1f200b59b0 2020-07-18T03:26:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.012+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2013bd00 con 0x7f1f200b59b0 2020-07-18T03:26:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.013+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20147d30 con 0x7f1f200b59b0 2020-07-18T03:26:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.013+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1f1400f070 con 0x7f1f200b59b0 2020-07-18T03:26:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.013+0000 7f1f0e7fc700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef4050350 0x7f1ef4052810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.013+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1f14068ee0 con 0x7f1f200b59b0 2020-07-18T03:26:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.013+0000 7f1f1dd9b700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef4050350 0x7f1ef4052810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.013+0000 7f1f1dd9b700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef4050350 0x7f1ef4052810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f1f1000a000 tx=0x7f1f10006e90).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.017+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1f140415f0 con 0x7f1f200b59b0 2020-07-18T03:26:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.177+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1f200b0230 con 0x7f1f200b59b0 2020-07-18T03:26:04.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.179+0000 7f1f0e7fc700 1 -- 172.21.15.47:0/3140376191 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1f14044c10 con 0x7f1f200b59b0 2020-07-18T03:26:04.182 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef4050350 msgr2=0x7f1ef4052810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef4050350 0x7f1ef4052810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f1f1000a000 tx=0x7f1f10006e90).stop 2020-07-18T03:26:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 msgr2=0x7f1f201476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f201476b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f1f14002f30 tx=0x7f1f14015000).stop 2020-07-18T03:26:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 shutdown_connections 2020-07-18T03:26:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 --2- 172.21.15.47:0/3140376191 >> v2:172.21.15.47:3300/0 conn(0x7f1f2013f040 0x7f1f2013b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f200b59b0 0x7f1f201476b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 --2- 172.21.15.47:0/3140376191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef4050350 0x7f1ef4052810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 >> 172.21.15.47:0/3140376191 conn(0x7f1f2005f5a0 msgr2=0x7f1f200b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 shutdown_connections 2020-07-18T03:26:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.181+0000 7f1f24e46700 1 -- 172.21.15.47:0/3140376191 wait complete. 2020-07-18T03:26:04.194 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:04.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.310+0000 7ff88ffb5700 1 Processor -- start 2020-07-18T03:26:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.310+0000 7ff88ffb5700 1 -- start start 2020-07-18T03:26:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88ffb5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff888142a20 0x7ff888142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88ffb5700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8880af4b0 con 0x7ff8880b3af0 2020-07-18T03:26:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88ffb5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8880b6d80 con 0x7ff888061c60 2020-07-18T03:26:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88ffb5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8880af710 con 0x7ff888142a20 2020-07-18T03:26:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88d550700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff888061c60 0x7ff8880620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40148/0 (socket says 172.21.15.47:40148) 2020-07-18T03:26:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88d550700 1 -- 172.21.15.47:0/304477800 learned_addr learned my addr 172.21.15.47:0/304477800 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88dd51700 1 --2- 172.21.15.47:0/304477800 >> v2:172.21.15.18:3300/0 conn(0x7ff888142a20 0x7ff888142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946480446 0 0) 0x7ff8880b6d80 con 0x7ff888061c60 2020-07-18T03:26:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870003240 con 0x7ff888061c60 2020-07-18T03:26:04.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2296388580 0 0) 0x7ff870003240 con 0x7ff888061c60 2020-07-18T03:26:04.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 >> v2:172.21.15.18:3300/0 conn(0x7ff888142a20 msgr2=0x7ff888142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 --2- 172.21.15.47:0/304477800 >> v2:172.21.15.18:3300/0 conn(0x7ff888142a20 0x7ff888142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 >> v1:172.21.15.18:6789/0 conn(0x7ff8880b3af0 legacy=0x7ff8880b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880af9b0 con 0x7ff888061c60 2020-07-18T03:26:04.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7ff8780016d0 con 0x7ff888061c60 2020-07-18T03:26:04.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 >> v1:172.21.15.47:6789/0 conn(0x7ff888061c60 legacy=0x7ff8880620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.311+0000 7ff88cd4f700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff870003730 0x7ff870003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8700044e0 0x7ff870004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff870004e50 0x7ff87000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880af710 con 0x7ff870004e50 2020-07-18T03:26:04.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870004350 con 0x7ff870003730 2020-07-18T03:26:04.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000bc50 con 0x7ff8700044e0 2020-07-18T03:26:04.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff870003730 0x7ff870003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88d550700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8700044e0 0x7ff870004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88e552700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff870004e50 0x7ff87000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 -- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8700044e0 msgr2=0x7ff870004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8700044e0 0x7ff870004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 -- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff870004e50 msgr2=0x7ff87000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff870004e50 0x7ff87000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 -- 172.21.15.47:0/304477800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870003240 con 0x7ff870003730 2020-07-18T03:26:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88e552700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff870004e50 0x7ff87000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88e552700 1 -- 172.21.15.47:0/304477800 reap_dead start 2020-07-18T03:26:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88dd51700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff870003730 0x7ff870003b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7ff88400c000 tx=0x7ff884008040).ready entity=mon.1 client_cookie=6ba981695c3a1bad server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88400e970 con 0x7ff870003730 2020-07-18T03:26:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88400ead0 con 0x7ff870003730 2020-07-18T03:26:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88cd4f700 1 -- 172.21.15.47:0/304477800 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88400ee30 con 0x7ff870003730 2020-07-18T03:26:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 -- 172.21.15.47:0/304477800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff870003730 msgr2=0x7ff870003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff870003730 0x7ff870003b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7ff88400c000 tx=0x7ff884008040).stop 2020-07-18T03:26:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 -- 172.21.15.47:0/304477800 shutdown_connections 2020-07-18T03:26:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 --2- 172.21.15.47:0/304477800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff870003730 0x7ff870003b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 -- 172.21.15.47:0/304477800 >> 172.21.15.47:0/304477800 conn(0x7ff88805f5a0 msgr2=0x7ff88805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 -- 172.21.15.47:0/304477800 shutdown_connections 2020-07-18T03:26:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.312+0000 7ff88ffb5700 1 -- 172.21.15.47:0/304477800 wait complete. 2020-07-18T03:26:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.314+0000 7ff88ffb5700 1 Processor -- start 2020-07-18T03:26:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.314+0000 7ff88ffb5700 1 -- start start 2020-07-18T03:26:04.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88ffb5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8881458c0 0x7ff888063d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88ffb5700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff888135880 con 0x7ff888136180 2020-07-18T03:26:04.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88ffb5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff888142150 con 0x7ff888135da0 2020-07-18T03:26:04.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88ffb5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff888062f60 con 0x7ff8881458c0 2020-07-18T03:26:04.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88dd51700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff888135da0 0x7ff888142e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40156/0 (socket says 172.21.15.47:40156) 2020-07-18T03:26:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88dd51700 1 -- 172.21.15.47:0/1318945098 learned_addr learned my addr 172.21.15.47:0/1318945098 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88e552700 1 --2- 172.21.15.47:0/1318945098 >> v2:172.21.15.47:3300/0 conn(0x7ff8881458c0 0x7ff888063d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385151620 0 0) 0x7ff888142150 con 0x7ff888135da0 2020-07-18T03:26:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860003240 con 0x7ff888135da0 2020-07-18T03:26:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88e552700 1 -- 172.21.15.47:0/1318945098 >> v1:172.21.15.47:6789/0 conn(0x7ff888135da0 legacy=0x7ff888142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116901751 0 0) 0x7ff8880af9b0 con 0x7ff888136180 2020-07-18T03:26:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88e552700 1 -- 172.21.15.47:0/1318945098 >> v1:172.21.15.18:6789/0 conn(0x7ff888136180 legacy=0x7ff8880635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88e552700 1 -- 172.21.15.47:0/1318945098 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff888063150 con 0x7ff8881458c0 2020-07-18T03:26:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88e552700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8881458c0 0x7ff888063d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff8780014a0 tx=0x7ff880009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff880002640 con 0x7ff8881458c0 2020-07-18T03:26:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff880010040 con 0x7ff8881458c0 2020-07-18T03:26:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff888141d20 con 0x7ff8881458c0 2020-07-18T03:26:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.315+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff880002990 con 0x7ff8881458c0 2020-07-18T03:26:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.316+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff888150c80 con 0x7ff8881458c0 2020-07-18T03:26:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.316+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff88001b020 con 0x7ff8881458c0 2020-07-18T03:26:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.316+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff888142300 con 0x7ff8881458c0 2020-07-18T03:26:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.317+0000 7ff875ffb700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8600502c0 0x7ff860052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.317+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff880067780 con 0x7ff8881458c0 2020-07-18T03:26:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.317+0000 7ff88dd51700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8600502c0 0x7ff860052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.317+0000 7ff88dd51700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8600502c0 0x7ff860052780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff884006f90 tx=0x7ff88401a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.320+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff88003fe90 con 0x7ff8881458c0 2020-07-18T03:26:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.478+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8880632e0 con 0x7ff8881458c0 2020-07-18T03:26:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.478+0000 7ff875ffb700 1 -- 172.21.15.47:0/1318945098 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff8800434b0 con 0x7ff8881458c0 2020-07-18T03:26:04.481 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8600502c0 msgr2=0x7ff860052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8600502c0 0x7ff860052780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff884006f90 tx=0x7ff88401a000).stop 2020-07-18T03:26:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8881458c0 msgr2=0x7ff888063d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8881458c0 0x7ff888063d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff8780014a0 tx=0x7ff880009040).stop 2020-07-18T03:26:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 shutdown_connections 2020-07-18T03:26:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8881458c0 0x7ff888063d00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 --2- 172.21.15.47:0/1318945098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8600502c0 0x7ff860052780 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 >> 172.21.15.47:0/1318945098 conn(0x7ff88805f5a0 msgr2=0x7ff88805d270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 shutdown_connections 2020-07-18T03:26:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.480+0000 7ff88ffb5700 1 -- 172.21.15.47:0/1318945098 wait complete. 2020-07-18T03:26:04.492 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.607+0000 7f7b2e34c700 1 Processor -- start 2020-07-18T03:26:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.607+0000 7f7b2e34c700 1 -- start start 2020-07-18T03:26:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.607+0000 7f7b2e34c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 0x7f7b280b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b2e34c700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2814c440 con 0x7f7b281489f0 2020-07-18T03:26:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b2e34c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b280b5bb0 con 0x7f7b28148570 2020-07-18T03:26:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b2e34c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2814f960 con 0x7f7b280b68b0 2020-07-18T03:26:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b277fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7b28148570 0x7f7b28061aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40164/0 (socket says 172.21.15.47:40164) 2020-07-18T03:26:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b277fe700 1 -- 172.21.15.47:0/1281610620 learned_addr learned my addr 172.21.15.47:0/1281610620 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b27fff700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 0x7f7b280b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806905964 0 0) 0x7f7b280b5bb0 con 0x7f7b28148570 2020-07-18T03:26:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b14003240 con 0x7f7b28148570 2020-07-18T03:26:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1735516008 0 0) 0x7f7b14003240 con 0x7f7b28148570 2020-07-18T03:26:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 msgr2=0x7f7b280b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 0x7f7b280b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 >> v1:172.21.15.18:6789/0 conn(0x7f7b281489f0 legacy=0x7f7b280621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b280628c0 con 0x7f7b28148570 2020-07-18T03:26:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7b1c0016d0 con 0x7f7b28148570 2020-07-18T03:26:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 >> v1:172.21.15.47:6789/0 conn(0x7f7b28148570 legacy=0x7f7b28061aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b14003730 0x7f7b14003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b14004070 0x7f7b14004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 0x7f7b14004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2814f960 con 0x7f7b14003730 2020-07-18T03:26:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b140050e0 con 0x7f7b14004070 2020-07-18T03:26:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b140052a0 con 0x7f7b280b68b0 2020-07-18T03:26:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.608+0000 7f7b277fe700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b14004070 0x7f7b14004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b27fff700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b14003730 0x7f7b14003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2c8e9700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 0x7f7b14004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b277fe700 1 -- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 msgr2=0x7f7b14004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b277fe700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b280b68b0 0x7f7b14004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b277fe700 1 -- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b14003730 msgr2=0x7f7b14003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b277fe700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b14003730 0x7f7b14003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b277fe700 1 -- 172.21.15.47:0/1281610620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14003240 con 0x7f7b14004070 2020-07-18T03:26:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b27fff700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b14003730 0x7f7b14003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b277fe700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b14004070 0x7f7b14004540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f7b1c0013e0 tx=0x7f7b1c0076a0).ready entity=mon.1 client_cookie=76441dab8ae117a1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b1c0070d0 con 0x7f7b14004070 2020-07-18T03:26:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c007230 con 0x7f7b14004070 2020-07-18T03:26:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b26ffd700 1 -- 172.21.15.47:0/1281610620 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b1c0055f0 con 0x7f7b14004070 2020-07-18T03:26:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2e34c700 1 -- 172.21.15.47:0/1281610620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b14004070 msgr2=0x7f7b14004540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2e34c700 1 --2- 172.21.15.47:0/1281610620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b14004070 0x7f7b14004540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f7b1c0013e0 tx=0x7f7b1c0076a0).stop 2020-07-18T03:26:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2c8e9700 1 -- 172.21.15.47:0/1281610620 reap_dead start 2020-07-18T03:26:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2e34c700 1 -- 172.21.15.47:0/1281610620 shutdown_connections 2020-07-18T03:26:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2e34c700 1 -- 172.21.15.47:0/1281610620 >> 172.21.15.47:0/1281610620 conn(0x7f7b2805f5a0 msgr2=0x7f7b2805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2e34c700 1 -- 172.21.15.47:0/1281610620 shutdown_connections 2020-07-18T03:26:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.609+0000 7f7b2e34c700 1 -- 172.21.15.47:0/1281610620 wait complete. 2020-07-18T03:26:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 Processor -- start 2020-07-18T03:26:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 -- start start 2020-07-18T03:26:04.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b2815c880 0x7f7b280b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7b2815cc60 0x7f7b28147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b280b26c0 con 0x7f7b2815d120 2020-07-18T03:26:04.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b280b2960 con 0x7f7b2815cc60 2020-07-18T03:26:04.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2e34c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b2340 con 0x7f7b2815c880 2020-07-18T03:26:04.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.611+0000 7f7b2c8e9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7b2815d120 0x7f7b281590f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40178/0 (socket says 172.21.15.47:40178) 2020-07-18T03:26:04.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b2c8e9700 1 -- 172.21.15.47:0/4026965174 learned_addr learned my addr 172.21.15.47:0/4026965174 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:04.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b27fff700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b2815c880 0x7f7b280b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b277fe700 1 --2- 172.21.15.47:0/4026965174 >> v2:172.21.15.18:3300/0 conn(0x7f7b2815cc60 0x7f7b28147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000834884 0 0) 0x7f7b280b26c0 con 0x7f7b2815d120 2020-07-18T03:26:04.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b00003240 con 0x7f7b2815d120 2020-07-18T03:26:04.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1234343240 0 0) 0x7f7b00003240 con 0x7f7b2815d120 2020-07-18T03:26:04.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b280b26c0 con 0x7f7b2815d120 2020-07-18T03:26:04.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7b20001450 con 0x7f7b2815d120 2020-07-18T03:26:04.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b277fe700 1 -- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b2815c880 msgr2=0x7f7b280b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b277fe700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b2815c880 0x7f7b280b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b277fe700 1 -- 172.21.15.47:0/4026965174 >> v1:172.21.15.47:6789/0 conn(0x7f7b2815d120 legacy=0x7f7b281590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b277fe700 1 -- 172.21.15.47:0/4026965174 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2814a560 con 0x7f7b2815cc60 2020-07-18T03:26:04.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b27fff700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b2815c880 0x7f7b280b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:04.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b277fe700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2815cc60 0x7f7b28147300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f7b1c0013e0 tx=0x7f7b1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b1c013640 con 0x7f7b2815cc60 2020-07-18T03:26:04.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0137a0 con 0x7f7b2815cc60 2020-07-18T03:26:04.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b1c013b00 con 0x7f7b2815cc60 2020-07-18T03:26:04.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2814a7b0 con 0x7f7b2815cc60 2020-07-18T03:26:04.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.612+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b281479b0 con 0x7f7b2815cc60 2020-07-18T03:26:04.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.613+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b280b24b0 con 0x7f7b2815cc60 2020-07-18T03:26:04.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.614+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7b1c01d070 con 0x7f7b2815cc60 2020-07-18T03:26:04.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.614+0000 7f7b07fff700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b00050640 0x7f7b00052b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.614+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7b1c0693f0 con 0x7f7b2815cc60 2020-07-18T03:26:04.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.614+0000 7f7b27fff700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b00050640 0x7f7b00052b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.614+0000 7f7b27fff700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b00050640 0x7f7b00052b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7b18008000 tx=0x7f7b18008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.618+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7b1c041b80 con 0x7f7b2815cc60 2020-07-18T03:26:04.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.774+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b2805b120 con 0x7f7b2815cc60 2020-07-18T03:26:04.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.775+0000 7f7b07fff700 1 -- 172.21.15.47:0/4026965174 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7b1c005e60 con 0x7f7b2815cc60 2020-07-18T03:26:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b00050640 msgr2=0x7f7b00052b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b00050640 0x7f7b00052b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7b18008000 tx=0x7f7b18008040).stop 2020-07-18T03:26:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2815cc60 msgr2=0x7f7b28147300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2815cc60 0x7f7b28147300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f7b1c0013e0 tx=0x7f7b1c015000).stop 2020-07-18T03:26:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 shutdown_connections 2020-07-18T03:26:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b00050640 0x7f7b00052b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b2815c880 0x7f7b280b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 --2- 172.21.15.47:0/4026965174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2815cc60 0x7f7b28147300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 >> 172.21.15.47:0/4026965174 conn(0x7f7b2805f5a0 msgr2=0x7f7b2805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 shutdown_connections 2020-07-18T03:26:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.776+0000 7f7b2e34c700 1 -- 172.21.15.47:0/4026965174 wait complete. 2020-07-18T03:26:04.787 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:04.789 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:26:04.789 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.901+0000 7f5a6ff7d700 1 Processor -- start 2020-07-18T03:26:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.901+0000 7f5a6ff7d700 1 -- start start 2020-07-18T03:26:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6ff7d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a68147a00 0x7f5a68147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6ff7d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a68147330 con 0x7f5a681433e0 2020-07-18T03:26:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6ff7d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680b57e0 con 0x7f5a680ad100 2020-07-18T03:26:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6ff7d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680ae530 con 0x7f5a68147a00 2020-07-18T03:26:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6d518700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a68147a00 0x7f5a68147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6dd19700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5a680ad100 0x7f5a680b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40182/0 (socket says 172.21.15.47:40182) 2020-07-18T03:26:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6dd19700 1 -- 172.21.15.47:0/2034418777 learned_addr learned my addr 172.21.15.47:0/2034418777 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6d518700 1 -- 172.21.15.47:0/2034418777 >> v1:172.21.15.47:6789/0 conn(0x7f5a680ad100 legacy=0x7f5a680b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6d518700 1 -- 172.21.15.47:0/2034418777 >> v1:172.21.15.18:6789/0 conn(0x7f5a681433e0 legacy=0x7f5a68143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6cd17700 1 -- 172.21.15.47:0/2034418777 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443568354 0 0) 0x7f5a680b57e0 con 0x7f5a680ad100 2020-07-18T03:26:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.902+0000 7f5a6d518700 1 -- 172.21.15.47:0/2034418777 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b5a80 con 0x7f5a68147a00 2020-07-18T03:26:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6d518700 1 --2- 172.21.15.47:0/2034418777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a68147a00 0x7f5a68147e40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5a6805ca30 tx=0x7f5a640091d0).ready entity=mon.1 client_cookie=5cb64b3815310d88 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6cd17700 1 -- 172.21.15.47:0/2034418777 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6400ba10 con 0x7f5a68147a00 2020-07-18T03:26:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6cd17700 1 -- 172.21.15.47:0/2034418777 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6400bb70 con 0x7f5a68147a00 2020-07-18T03:26:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6cd17700 1 -- 172.21.15.47:0/2034418777 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6400bce0 con 0x7f5a68147a00 2020-07-18T03:26:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/2034418777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a68147a00 msgr2=0x7f5a68147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/2034418777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a68147a00 0x7f5a68147e40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5a6805ca30 tx=0x7f5a640091d0).stop 2020-07-18T03:26:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/2034418777 shutdown_connections 2020-07-18T03:26:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/2034418777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a68147a00 0x7f5a68147e40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/2034418777 >> 172.21.15.47:0/2034418777 conn(0x7f5a680b9340 msgr2=0x7f5a680b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/2034418777 shutdown_connections 2020-07-18T03:26:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.903+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/2034418777 wait complete. 2020-07-18T03:26:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.904+0000 7f5a6ff7d700 1 Processor -- start 2020-07-18T03:26:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.904+0000 7f5a6ff7d700 1 -- start start 2020-07-18T03:26:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6ff7d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a680ad100 0x7f5a680b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6ff7d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5a681433e0 0x7f5a68139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6ff7d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a681574f0 con 0x7f5a680636e0 2020-07-18T03:26:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6ff7d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a68061c60 con 0x7f5a681433e0 2020-07-18T03:26:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6ff7d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680669a0 con 0x7f5a680ad100 2020-07-18T03:26:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a680ad100 0x7f5a680b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a680ad100 0x7f5a680b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59098/0 (socket says 172.21.15.47:59098) 2020-07-18T03:26:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 -- 172.21.15.47:0/1224549702 learned_addr learned my addr 172.21.15.47:0/1224549702 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6d518700 1 --2- 172.21.15.47:0/1224549702 >> v2:172.21.15.18:3300/0 conn(0x7f5a681433e0 0x7f5a68139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 -- 172.21.15.47:0/1224549702 >> v2:172.21.15.18:3300/0 conn(0x7f5a681433e0 msgr2=0x7f5a68139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 --2- 172.21.15.47:0/1224549702 >> v2:172.21.15.18:3300/0 conn(0x7f5a681433e0 0x7f5a68139f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 -- 172.21.15.47:0/1224549702 >> v1:172.21.15.18:6789/0 conn(0x7f5a680636e0 legacy=0x7f5a68145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.905+0000 7f5a6dd19700 1 -- 172.21.15.47:0/1224549702 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b2f20 con 0x7f5a680ad100 2020-07-18T03:26:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.906+0000 7f5a6dd19700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a680ad100 0x7f5a680b2a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5a680653a0 tx=0x7f5a580079c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.906+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a580072e0 con 0x7f5a680ad100 2020-07-18T03:26:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.906+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58007440 con 0x7f5a680ad100 2020-07-18T03:26:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.906+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a68157130 con 0x7f5a680ad100 2020-07-18T03:26:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.906+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a580057e0 con 0x7f5a680ad100 2020-07-18T03:26:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.906+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a680b3d00 con 0x7f5a680ad100 2020-07-18T03:26:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.907+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a68066b70 con 0x7f5a680ad100 2020-07-18T03:26:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.907+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5a58010070 con 0x7f5a680ad100 2020-07-18T03:26:04.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.907+0000 7f5a55ffb700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4004ff60 0x7f5a40052420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:04.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.907+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5a580672e0 con 0x7f5a680ad100 2020-07-18T03:26:04.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.907+0000 7f5a6d518700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4004ff60 0x7f5a40052420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:04.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.908+0000 7f5a6d518700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4004ff60 0x7f5a40052420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f5a6401a000 tx=0x7f5a64016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:04.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:04.911+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5a5803fa70 con 0x7f5a680ad100 2020-07-18T03:26:05.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.124+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a68157320 con 0x7f5a680ad100 2020-07-18T03:26:05.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.125+0000 7f5a55ffb700 1 -- 172.21.15.47:0/1224549702 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2124 (secure 0 0 0) 0x7f5a58043090 con 0x7f5a680ad100 2020-07-18T03:26:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:05.127 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:49.581963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4387":{"gid":4387,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.47:6835/308881804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":308881804},{"type":"v1","addr":"172.21.15.47:6835","nonce":308881804}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:26:05.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4004ff60 msgr2=0x7f5a40052420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:05.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4004ff60 0x7f5a40052420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f5a6401a000 tx=0x7f5a64016000).stop 2020-07-18T03:26:05.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a680ad100 msgr2=0x7f5a680b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:05.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a680ad100 0x7f5a680b2a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5a680653a0 tx=0x7f5a580079c0).stop 2020-07-18T03:26:05.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 shutdown_connections 2020-07-18T03:26:05.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5a4004ff60 0x7f5a40052420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:05.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/1224549702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a680ad100 0x7f5a680b2a30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:05.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 --2- 172.21.15.47:0/1224549702 >> v2:172.21.15.18:3300/0 conn(0x7f5a681433e0 0x7f5a68139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:05.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 >> 172.21.15.47:0/1224549702 conn(0x7f5a680b9340 msgr2=0x7f5a680b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:05.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 shutdown_connections 2020-07-18T03:26:05.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:05.126+0000 7f5a6ff7d700 1 -- 172.21.15.47:0/1224549702 wait complete. 2020-07-18T03:26:05.131 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2020-07-18T03:26:05.341 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:25:45.899311+0000', 'modified': '2020-07-18T03:25:49.581963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4387': {'gid': 4387, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.47:6835/308881804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 308881804}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 308881804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:05.342 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:05.343 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:05.479 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4387,"want_state":"up:active","state":"up:active","rank_uptime":16.887087755,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":17.143840176000001} 2020-07-18T03:26:05.480 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4387, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 16.887087755, "state": "up:active", "uptime": 17.143840176, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:05.480 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:26:05.481 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:26:05.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.481 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:26:05.496 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:05.498 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:05.498 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42825sec preferred_lft 42825sec 2020-07-18T03:26:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:26:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:05.500 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:05.501 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:05.501 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether ca:7a:6c:3b:58:ba brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::63:49ff:fee4:50d6/64 scope link 2020-07-18T03:26:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:05.503 INFO:teuthology.orchestra.run.smithi047.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:26:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether ca:7a:6c:3b:58:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:26:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::c87a:6cff:fe3b:58ba/64 scope link 2020-07-18T03:26:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:05.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.505 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:26:05.567 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:26:05.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.569 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:26:05.624 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:26:05.625 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:05.625 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:05.625 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:05.626 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:26:05.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.626 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:05.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.698 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:05.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.772 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:05.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.844 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:26:05.897 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-07-18T03:26:05.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.899 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:26:05.952 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:26:05.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:05.953 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:26:06.007 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:26:06.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.008 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:26:06.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.080 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1' 2020-07-18T03:26:06.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.152 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:26:06.234 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:06.235 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:06.235 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:06.236 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.236 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:06.236 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.237 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:06.237 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:3b:3e:0e:06:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:26:06.237 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:26:06.238 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.238 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::d83b:3eff:fe0e:6a3/64 scope link 2020-07-18T03:26:06.239 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.248 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:26:06.325 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.327 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:3b:3e:0e:06:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:26:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:26:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::d83b:3eff:fe0e:6a3/64 scope link 2020-07-18T03:26:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:06.334 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-07-18T03:26:06.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.335 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1' 2020-07-18T03:26:06.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.408 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:26:06.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.503 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:26:06.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.568 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:26:06.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.628 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:26:06.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.677 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 up' 2020-07-18T03:26:06.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:06.723 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.1 master ceph-brx' 2020-07-18T03:26:06.838 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:26:06.839 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:26:06.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:06.840 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:06.856 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:06.928 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:06.928 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:07.014 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:07.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:07.015 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:07.068 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:26:07.069 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:26:07.069 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:07.111 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:07.145 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:07.146 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:07.211 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:26:07.209+0000 7f35aa7ba1c0 -1 init, newargv = 0x55e40ff49dd0 newargc=15 2020-07-18T03:26:07.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[29502]: starting ceph client 2020-07-18T03:26:07.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[29502]: starting fuse 2020-07-18T03:26:07.222 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:07.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:07.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:07.224 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:07.277 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:26:07.278 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:26:07.278 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:26:07.279 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:26:07.279 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:26:07.279 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:26:07.280 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:26:07.280 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:26:07.280 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:07.281 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:26:07.281 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:26:07.281 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:26:07.282 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:26:07.282 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:07.282 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:26:07.283 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:26:07.283 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:26:07.284 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:26:07.284 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:07.284 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:26:07.285 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:26:07.285 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:26:07.285 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:26:07.286 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:26:07.287 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:26:07.287 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:26:07.288 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:26:07.288 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:07.289 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:26:07.289 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:26:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.29502.asok 2020-07-18T03:26:07.589 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29502.asok 2020-07-18T03:26:07.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:07.590 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29502.asok status 2020-07-18T03:26:07.741 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:26:07.742 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:26:07.742 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:26:07.742 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:26:07.743 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:26:07.743 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:26:07.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:26:07.743 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "29502", 2020-07-18T03:26:07.744 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:26:07.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:07.744 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:26:07.744 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:26:07.744 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4771, 2020-07-18T03:26:07.745 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:26:07.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:26:07.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:26:07.745 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4771 2020-07-18T03:26:07.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:07.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:07.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:07.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:26:07.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1307110712 2020-07-18T03:26:07.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:07.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:07.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:07.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:07.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:26:07.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1307110712 2020-07-18T03:26:07.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:07.748 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4771 192.168.0.2:0/1307110712", 2020-07-18T03:26:07.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/1307110712", 2020-07-18T03:26:07.749 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:26:07.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 6, 2020-07-18T03:26:07.750 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 24, 2020-07-18T03:26:07.750 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:26:07.750 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:26:07.750 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:26:07.753 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:07.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.866+0000 7f4d097ac700 1 Processor -- start 2020-07-18T03:26:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.866+0000 7f4d097ac700 1 -- start start 2020-07-18T03:26:07.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d097ac700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d040ad100 0x7f4d040b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:07.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d097ac700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d04147330 con 0x7f4d04147a00 2020-07-18T03:26:07.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d097ac700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040603c0 con 0x7f4d041443f0 2020-07-18T03:26:07.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d097ac700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040b57e0 con 0x7f4d040ad100 2020-07-18T03:26:07.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d037fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d041443f0 0x7f4d0405fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40212/0 (socket says 172.21.15.47:40212) 2020-07-18T03:26:07.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d037fe700 1 -- 172.21.15.47:0/791459756 learned_addr learned my addr 172.21.15.47:0/791459756 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:07.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d02ffd700 1 --2- 172.21.15.47:0/791459756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d040ad100 0x7f4d040b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:07.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d01ffb700 1 -- 172.21.15.47:0/791459756 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525957973 0 0) 0x7f4d040603c0 con 0x7f4d041443f0 2020-07-18T03:26:07.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.867+0000 7f4d01ffb700 1 -- 172.21.15.47:0/791459756 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf0003240 con 0x7f4d041443f0 2020-07-18T03:26:07.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d01ffb700 1 -- 172.21.15.47:0/791459756 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 598896577 0 0) 0x7f4cf0003240 con 0x7f4d041443f0 2020-07-18T03:26:07.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d02ffd700 1 -- 172.21.15.47:0/791459756 >> v1:172.21.15.47:6789/0 conn(0x7f4d041443f0 legacy=0x7f4d0405fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:07.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d02ffd700 1 -- 172.21.15.47:0/791459756 >> v1:172.21.15.18:6789/0 conn(0x7f4d04147a00 legacy=0x7f4d0405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:07.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d02ffd700 1 -- 172.21.15.47:0/791459756 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040b5a80 con 0x7f4d040ad100 2020-07-18T03:26:07.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d02ffd700 1 --2- 172.21.15.47:0/791459756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d040ad100 0x7f4d040b4c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4cf40010d0 tx=0x7f4cf8006dd0).ready entity=mon.2 client_cookie=10c2b0fa78d7aac1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:07.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d01ffb700 1 -- 172.21.15.47:0/791459756 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cfc001a50 con 0x7f4d040ad100 2020-07-18T03:26:07.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d01ffb700 1 -- 172.21.15.47:0/791459756 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf800d570 con 0x7f4d040ad100 2020-07-18T03:26:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d01ffb700 1 -- 172.21.15.47:0/791459756 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cf800d6e0 con 0x7f4d040ad100 2020-07-18T03:26:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 -- 172.21.15.47:0/791459756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d040ad100 msgr2=0x7f4d040b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 --2- 172.21.15.47:0/791459756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d040ad100 0x7f4d040b4c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4cf40010d0 tx=0x7f4cf8006dd0).stop 2020-07-18T03:26:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 -- 172.21.15.47:0/791459756 shutdown_connections 2020-07-18T03:26:07.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 --2- 172.21.15.47:0/791459756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d040ad100 0x7f4d040b4c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:07.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 -- 172.21.15.47:0/791459756 >> 172.21.15.47:0/791459756 conn(0x7f4d040b9340 msgr2=0x7f4d040b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:07.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 -- 172.21.15.47:0/791459756 shutdown_connections 2020-07-18T03:26:07.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.868+0000 7f4d097ac700 1 -- 172.21.15.47:0/791459756 wait complete. 2020-07-18T03:26:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 Processor -- start 2020-07-18T03:26:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 -- start start 2020-07-18T03:26:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d041443f0 0x7f4d04063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4d040621b0 0x7f4d040b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d04063ef0 con 0x7f4d04147a00 2020-07-18T03:26:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d04064190 con 0x7f4d040621b0 2020-07-18T03:26:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.870+0000 7f4d097ac700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04064430 con 0x7f4d041443f0 2020-07-18T03:26:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d027fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d04147a00 0x7f4d04141b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40216/0 (socket says 172.21.15.47:40216) 2020-07-18T03:26:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d027fc700 1 -- 172.21.15.47:0/1438741937 learned_addr learned my addr 172.21.15.47:0/1438741937 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d02ffd700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d041443f0 0x7f4d04063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d037fe700 1 --2- 172.21.15.47:0/1438741937 >> v2:172.21.15.18:3300/0 conn(0x7f4d040621b0 0x7f4d040b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716993247 0 0) 0x7f4d04063ef0 con 0x7f4d04147a00 2020-07-18T03:26:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce0003240 con 0x7f4d04147a00 2020-07-18T03:26:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d02ffd700 1 -- 172.21.15.47:0/1438741937 >> v2:172.21.15.18:3300/0 conn(0x7f4d040621b0 msgr2=0x7f4d040b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3166218016 0 0) 0x7f4ce0003240 con 0x7f4d04147a00 2020-07-18T03:26:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d02ffd700 1 --2- 172.21.15.47:0/1438741937 >> v2:172.21.15.18:3300/0 conn(0x7f4d040621b0 0x7f4d040b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d02ffd700 1 -- 172.21.15.47:0/1438741937 >> v1:172.21.15.47:6789/0 conn(0x7f4d04147a00 legacy=0x7f4d04141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d02ffd700 1 -- 172.21.15.47:0/1438741937 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04142240 con 0x7f4d041443f0 2020-07-18T03:26:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d02ffd700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d041443f0 0x7f4d04063900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f4cf4001cf0 tx=0x7f4cf8006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cf4001b50 con 0x7f4d041443f0 2020-07-18T03:26:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf8013d80 con 0x7f4d041443f0 2020-07-18T03:26:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cf801e5f0 con 0x7f4d041443f0 2020-07-18T03:26:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.871+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d040648d0 con 0x7f4d041443f0 2020-07-18T03:26:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.872+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d040b1ff0 con 0x7f4d041443f0 2020-07-18T03:26:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.872+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040b7150 con 0x7f4d041443f0 2020-07-18T03:26:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.873+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4cf801c070 con 0x7f4d041443f0 2020-07-18T03:26:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.873+0000 7f4ceaffd700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ce0050310 0x7f4ce00527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.873+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4cf8068780 con 0x7f4d041443f0 2020-07-18T03:26:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.873+0000 7f4d027fc700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ce0050310 0x7f4ce00527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.873+0000 7f4d027fc700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ce0050310 0x7f4ce00527d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4cf40013f0 tx=0x7f4cf40087e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:07.877+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4cf8040e90 con 0x7f4d041443f0 2020-07-18T03:26:08.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.034+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d04057450 con 0x7f4d041443f0 2020-07-18T03:26:08.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.035+0000 7f4ceaffd700 1 -- 172.21.15.47:0/1438741937 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f4cf80444b0 con 0x7f4d041443f0 2020-07-18T03:26:08.038 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:08.038 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:08.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ce0050310 msgr2=0x7f4ce00527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ce0050310 0x7f4ce00527d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4cf40013f0 tx=0x7f4cf40087e0).stop 2020-07-18T03:26:08.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d041443f0 msgr2=0x7f4d04063900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d041443f0 0x7f4d04063900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f4cf4001cf0 tx=0x7f4cf8006dd0).stop 2020-07-18T03:26:08.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 shutdown_connections 2020-07-18T03:26:08.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 --2- 172.21.15.47:0/1438741937 >> v2:172.21.15.18:3300/0 conn(0x7f4d040621b0 0x7f4d040b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4d041443f0 0x7f4d04063900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 --2- 172.21.15.47:0/1438741937 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ce0050310 0x7f4ce00527d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 >> 172.21.15.47:0/1438741937 conn(0x7f4d040b9340 msgr2=0x7f4d041469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 shutdown_connections 2020-07-18T03:26:08.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.037+0000 7f4d097ac700 1 -- 172.21.15.47:0/1438741937 wait complete. 2020-07-18T03:26:08.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:08.051 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.167+0000 7f98225ee700 1 Processor -- start 2020-07-18T03:26:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.167+0000 7f98225ee700 1 -- start start 2020-07-18T03:26:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f98225ee700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 0x7f981c143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f98225ee700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f981c0b59b0 0x7f981c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f98225ee700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c147de0 con 0x7f981c0b6300 2020-07-18T03:26:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f98225ee700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c0ae000 con 0x7f981c0b59b0 2020-07-18T03:26:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f98225ee700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c148010 con 0x7f981c0ad330 2020-07-18T03:26:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f9820b8b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f981c0b6300 0x7f981c0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40224/0 (socket says 172.21.15.47:40224) 2020-07-18T03:26:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f9820b8b700 1 -- 172.21.15.47:0/1082135063 learned_addr learned my addr 172.21.15.47:0/1082135063 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f981bfff700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 0x7f981c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f981b7fe700 1 --2- 172.21.15.47:0/1082135063 >> v2:172.21.15.18:3300/0 conn(0x7f981c0b59b0 0x7f981c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085960401 0 0) 0x7f981c147de0 con 0x7f981c0b6300 2020-07-18T03:26:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9808003240 con 0x7f981c0b6300 2020-07-18T03:26:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.168+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3483575691 0 0) 0x7f9808003240 con 0x7f981c0b6300 2020-07-18T03:26:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 msgr2=0x7f981c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 0x7f981c143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 >> v2:172.21.15.18:3300/0 conn(0x7f981c0b59b0 msgr2=0x7f981c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 --2- 172.21.15.47:0/1082135063 >> v2:172.21.15.18:3300/0 conn(0x7f981c0b59b0 0x7f981c0b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0ad710 con 0x7f981c0b6300 2020-07-18T03:26:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f9814001a50 con 0x7f981c0b6300 2020-07-18T03:26:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 >> v1:172.21.15.47:6789/0 conn(0x7f981c0b6300 legacy=0x7f981c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9808003730 0x7f9808003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98080044e0 0x7f9808004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 0x7f9808004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0ae000 con 0x7f9808003730 2020-07-18T03:26:08.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c148010 con 0x7f98080044e0 2020-07-18T03:26:08.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9808004270 con 0x7f981c0ad330 2020-07-18T03:26:08.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98080044e0 0x7f9808004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f9820b8b700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 0x7f9808004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981bfff700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9808003730 0x7f9808003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 -- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 msgr2=0x7f9808004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:08.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f981c0ad330 0x7f9808004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 -- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9808003730 msgr2=0x7f9808003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:08.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9808003730 0x7f9808003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 -- 172.21.15.47:0/1082135063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808003240 con 0x7f98080044e0 2020-07-18T03:26:08.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981b7fe700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98080044e0 0x7f9808004960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f981000c000 tx=0x7f9810008040).ready entity=mon.1 client_cookie=7361f02df838b5e5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:08.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981000eb60 con 0x7f98080044e0 2020-07-18T03:26:08.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981000ecc0 con 0x7f98080044e0 2020-07-18T03:26:08.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f981affd700 1 -- 172.21.15.47:0/1082135063 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98100125e0 con 0x7f98080044e0 2020-07-18T03:26:08.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f98225ee700 1 -- 172.21.15.47:0/1082135063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98080044e0 msgr2=0x7f9808004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f98225ee700 1 --2- 172.21.15.47:0/1082135063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98080044e0 0x7f9808004960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f981000c000 tx=0x7f9810008040).stop 2020-07-18T03:26:08.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f9820b8b700 1 -- 172.21.15.47:0/1082135063 reap_dead start 2020-07-18T03:26:08.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f98225ee700 1 -- 172.21.15.47:0/1082135063 shutdown_connections 2020-07-18T03:26:08.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f98225ee700 1 -- 172.21.15.47:0/1082135063 >> 172.21.15.47:0/1082135063 conn(0x7f981c05f5a0 msgr2=0x7f981c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:08.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f98225ee700 1 -- 172.21.15.47:0/1082135063 shutdown_connections 2020-07-18T03:26:08.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.169+0000 7f98225ee700 1 -- 172.21.15.47:0/1082135063 wait complete. 2020-07-18T03:26:08.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.171+0000 7f98225ee700 1 Processor -- start 2020-07-18T03:26:08.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.171+0000 7f98225ee700 1 -- start start 2020-07-18T03:26:08.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f981c13b130 0x7f981c1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f981c13e9b0 0x7f981c159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c1437c0 con 0x7f981c13b510 2020-07-18T03:26:08.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c144690 con 0x7f981c13b130 2020-07-18T03:26:08.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c159e30 con 0x7f981c13e9b0 2020-07-18T03:26:08.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f981c13e9b0 0x7f981c159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f9820b8b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f981c13b510 0x7f981c061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40236/0 (socket says 172.21.15.47:40236) 2020-07-18T03:26:08.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f981c13e9b0 0x7f981c159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59146/0 (socket says 172.21.15.47:59146) 2020-07-18T03:26:08.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f9820b8b700 1 -- 172.21.15.47:0/292508073 learned_addr learned my addr 172.21.15.47:0/292508073 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:08.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981bfff700 1 --2- 172.21.15.47:0/292508073 >> v2:172.21.15.18:3300/0 conn(0x7f981c13b130 0x7f981c1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 -- 172.21.15.47:0/292508073 >> v2:172.21.15.18:3300/0 conn(0x7f981c13b130 msgr2=0x7f981c1441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 --2- 172.21.15.47:0/292508073 >> v2:172.21.15.18:3300/0 conn(0x7f981c13b130 0x7f981c1441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 -- 172.21.15.47:0/292508073 >> v1:172.21.15.47:6789/0 conn(0x7f981c13b510 legacy=0x7f981c061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784782099 0 0) 0x7f981c1437c0 con 0x7f981c13b510 2020-07-18T03:26:08.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 -- 172.21.15.47:0/292508073 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c143a60 con 0x7f981c13e9b0 2020-07-18T03:26:08.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f981b7fe700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c13e9b0 0x7f981c159940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f981000c000 tx=0x7f9810019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:08.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981001c850 con 0x7f981c13e9b0 2020-07-18T03:26:08.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981001c9b0 con 0x7f981c13e9b0 2020-07-18T03:26:08.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981001cd10 con 0x7f981c13e9b0 2020-07-18T03:26:08.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c143d00 con 0x7f981c13e9b0 2020-07-18T03:26:08.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.172+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c13ced0 con 0x7f981c13e9b0 2020-07-18T03:26:08.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.174+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c143e90 con 0x7f981c13e9b0 2020-07-18T03:26:08.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.174+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9810025470 con 0x7f981c13e9b0 2020-07-18T03:26:08.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.174+0000 7f97fbfff700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f404fd80 0x7f97f4052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.174+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f98100696f0 con 0x7f981c13e9b0 2020-07-18T03:26:08.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.174+0000 7f981bfff700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f404fd80 0x7f97f4052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.174+0000 7f981bfff700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f404fd80 0x7f97f4052240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f980c008000 tx=0x7f980c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:08.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.177+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9810041da0 con 0x7f981c13e9b0 2020-07-18T03:26:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.337+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f981c116540 con 0x7f981c13e9b0 2020-07-18T03:26:08.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.337+0000 7f97fbfff700 1 -- 172.21.15.47:0/292508073 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f98100453c0 con 0x7f981c13e9b0 2020-07-18T03:26:08.339 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:08.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f404fd80 msgr2=0x7f97f4052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f404fd80 0x7f97f4052240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f980c008000 tx=0x7f980c008040).stop 2020-07-18T03:26:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c13e9b0 msgr2=0x7f981c159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c13e9b0 0x7f981c159940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f981000c000 tx=0x7f9810019000).stop 2020-07-18T03:26:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 shutdown_connections 2020-07-18T03:26:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f404fd80 0x7f97f4052240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 --2- 172.21.15.47:0/292508073 >> v2:172.21.15.18:3300/0 conn(0x7f981c13b130 0x7f981c1441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 --2- 172.21.15.47:0/292508073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c13e9b0 0x7f981c159940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 >> 172.21.15.47:0/292508073 conn(0x7f981c05f5a0 msgr2=0x7f981c0b8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 shutdown_connections 2020-07-18T03:26:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.338+0000 7f98225ee700 1 -- 172.21.15.47:0/292508073 wait complete. 2020-07-18T03:26:08.351 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:08.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.466+0000 7f4930ed7700 1 Processor -- start 2020-07-18T03:26:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.466+0000 7f4930ed7700 1 -- start start 2020-07-18T03:26:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f492c142430 0x7f492c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c061db0 con 0x7f492c141fb0 2020-07-18T03:26:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c062050 con 0x7f492c144e70 2020-07-18T03:26:08.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c145250 con 0x7f492c142430 2020-07-18T03:26:08.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f492c142430 0x7f492c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f492c142430 0x7f492c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59156/0 (socket says 172.21.15.47:59156) 2020-07-18T03:26:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f492c144e70 0x7f492c0b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40240/0 (socket says 172.21.15.47:40240) 2020-07-18T03:26:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 -- 172.21.15.47:0/3486840085 learned_addr learned my addr 172.21.15.47:0/3486840085 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 -- 172.21.15.47:0/3486840085 >> v1:172.21.15.47:6789/0 conn(0x7f492c144e70 legacy=0x7f492c0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 -- 172.21.15.47:0/3486840085 >> v1:172.21.15.18:6789/0 conn(0x7f492c141fb0 legacy=0x7f492c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 -- 172.21.15.47:0/3486840085 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0622f0 con 0x7f492c142430 2020-07-18T03:26:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492ad9d700 1 --2- 172.21.15.47:0/3486840085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c142430 0x7f492c0b0bc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f492c062670 tx=0x7f4924009270).ready entity=mon.1 client_cookie=33b0ebcff9d05272 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492959a700 1 -- 172.21.15.47:0/3486840085 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492400bbc0 con 0x7f492c142430 2020-07-18T03:26:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492959a700 1 -- 172.21.15.47:0/3486840085 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400bd20 con 0x7f492c142430 2020-07-18T03:26:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f492959a700 1 -- 172.21.15.47:0/3486840085 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492400be90 con 0x7f492c142430 2020-07-18T03:26:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 -- 172.21.15.47:0/3486840085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c142430 msgr2=0x7f492c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 --2- 172.21.15.47:0/3486840085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c142430 0x7f492c0b0bc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f492c062670 tx=0x7f4924009270).stop 2020-07-18T03:26:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 -- 172.21.15.47:0/3486840085 shutdown_connections 2020-07-18T03:26:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 --2- 172.21.15.47:0/3486840085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f492c142430 0x7f492c0b0bc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.467+0000 7f4930ed7700 1 -- 172.21.15.47:0/3486840085 >> 172.21.15.47:0/3486840085 conn(0x7f492c05f5a0 msgr2=0x7f492c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.468+0000 7f4930ed7700 1 -- 172.21.15.47:0/3486840085 shutdown_connections 2020-07-18T03:26:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.468+0000 7f4930ed7700 1 -- 172.21.15.47:0/3486840085 wait complete. 2020-07-18T03:26:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.469+0000 7f4930ed7700 1 Processor -- start 2020-07-18T03:26:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.469+0000 7f4930ed7700 1 -- start start 2020-07-18T03:26:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f4930ed7700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f492c144e70 0x7f492c13cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f4930ed7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 0x7f492c13ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f4930ed7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c13d640 con 0x7f492c141fb0 2020-07-18T03:26:08.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f4930ed7700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c13d420 con 0x7f492c144e70 2020-07-18T03:26:08.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f4930ed7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c13d8e0 con 0x7f492c13a7e0 2020-07-18T03:26:08.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f492a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f492c141fb0 0x7f492c143ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40246/0 (socket says 172.21.15.47:40246) 2020-07-18T03:26:08.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f492a59c700 1 -- 172.21.15.47:0/1769826043 learned_addr learned my addr 172.21.15.47:0/1769826043 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:08.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894345676 0 0) 0x7f492c13d640 con 0x7f492c141fb0 2020-07-18T03:26:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4900003240 con 0x7f492c141fb0 2020-07-18T03:26:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f4929d9b700 1 --2- 172.21.15.47:0/1769826043 >> v2:172.21.15.18:3300/0 conn(0x7f492c144e70 0x7f492c13cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f492ad9d700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 0x7f492c13ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1519568704 0 0) 0x7f4900003240 con 0x7f492c141fb0 2020-07-18T03:26:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f492c13d640 con 0x7f492c141fb0 2020-07-18T03:26:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4920001960 con 0x7f492c141fb0 2020-07-18T03:26:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 msgr2=0x7f492c13ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 0x7f492c13ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 >> v2:172.21.15.18:3300/0 conn(0x7f492c144e70 msgr2=0x7f492c13cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 --2- 172.21.15.47:0/1769826043 >> v2:172.21.15.18:3300/0 conn(0x7f492c144e70 0x7f492c13cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 >> v1:172.21.15.47:6789/0 conn(0x7f492c141fb0 legacy=0x7f492c143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4900003800 0x7f4900003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49000044e0 0x7f4900004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 0x7f4900004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c13d420 con 0x7f4900003800 2020-07-18T03:26:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c13d8e0 con 0x7f49000044e0 2020-07-18T03:26:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.470+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4900005300 con 0x7f492c13a7e0 2020-07-18T03:26:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49000044e0 0x7f4900004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f492a59c700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4900003800 0x7f4900003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f492ad9d700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 0x7f4900004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 -- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 msgr2=0x7f4900004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f492c13a7e0 0x7f4900004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 -- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4900003800 msgr2=0x7f4900003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4900003800 0x7f4900003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c13db80 con 0x7f49000044e0 2020-07-18T03:26:08.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c13e020 con 0x7f49000044e0 2020-07-18T03:26:08.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c13ae10 con 0x7f49000044e0 2020-07-18T03:26:08.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f4929d9b700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49000044e0 0x7f4900004920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f491c00d000 tx=0x7f491c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:08.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491c011660 con 0x7f49000044e0 2020-07-18T03:26:08.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c0117c0 con 0x7f49000044e0 2020-07-18T03:26:08.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.471+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491c011930 con 0x7f49000044e0 2020-07-18T03:26:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.472+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c13dd10 con 0x7f49000044e0 2020-07-18T03:26:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.472+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f491c011a90 con 0x7f49000044e0 2020-07-18T03:26:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.472+0000 7f491a7fc700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4900058250 0x7f490005a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.473+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f491c067910 con 0x7f49000044e0 2020-07-18T03:26:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.473+0000 7f492a59c700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4900058250 0x7f490005a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:08.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.473+0000 7f492a59c700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4900058250 0x7f490005a710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f492000d000 tx=0x7f49200079a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:08.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.476+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f491c0400a0 con 0x7f49000044e0 2020-07-18T03:26:08.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.634+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f492c145d20 con 0x7f49000044e0 2020-07-18T03:26:08.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.634+0000 7f491a7fc700 1 -- 172.21.15.47:0/1769826043 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f491c0436c0 con 0x7f49000044e0 2020-07-18T03:26:08.637 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:08.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4900058250 msgr2=0x7f490005a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4900058250 0x7f490005a710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f492000d000 tx=0x7f49200079a0).stop 2020-07-18T03:26:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49000044e0 msgr2=0x7f4900004920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49000044e0 0x7f4900004920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f491c00d000 tx=0x7f491c007b50).stop 2020-07-18T03:26:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f492ad9d700 1 -- 172.21.15.47:0/1769826043 reap_dead start 2020-07-18T03:26:08.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 shutdown_connections 2020-07-18T03:26:08.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 --2- 172.21.15.47:0/1769826043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49000044e0 0x7f4900004920 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 >> 172.21.15.47:0/1769826043 conn(0x7f492c05f5a0 msgr2=0x7f492c144600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 shutdown_connections 2020-07-18T03:26:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:08.636+0000 7f4930ed7700 1 -- 172.21.15.47:0/1769826043 wait complete. 2020-07-18T03:26:08.654 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:09.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.046+0000 7f3d3cd90700 1 Processor -- start 2020-07-18T03:26:09.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.046+0000 7f3d3cd90700 1 -- start start 2020-07-18T03:26:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3cd90700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380ad330 0x7f3d38143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3cd90700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3d380b59b0 0x7f3d380b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3cd90700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3d380b6300 0x7f3d380b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3cd90700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d38147de0 con 0x7f3d380b6300 2020-07-18T03:26:09.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3cd90700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d380ae000 con 0x7f3d380b59b0 2020-07-18T03:26:09.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3cd90700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d38148010 con 0x7f3d380ad330 2020-07-18T03:26:09.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3d380b59b0 0x7f3d380b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3d380b59b0 0x7f3d380b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59174/0 (socket says 172.21.15.47:59174) 2020-07-18T03:26:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 -- 172.21.15.47:0/471053409 learned_addr learned my addr 172.21.15.47:0/471053409 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d36d9d700 1 --2- 172.21.15.47:0/471053409 >> v2:172.21.15.18:3300/0 conn(0x7f3d380b6300 0x7f3d380b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3659c700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380ad330 0x7f3d38143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 -- 172.21.15.47:0/471053409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380ad330 msgr2=0x7f3d38143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380ad330 0x7f3d38143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 -- 172.21.15.47:0/471053409 >> v2:172.21.15.18:3300/0 conn(0x7f3d380b6300 msgr2=0x7f3d380b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/471053409 >> v2:172.21.15.18:3300/0 conn(0x7f3d380b6300 0x7f3d380b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d35d9b700 1 -- 172.21.15.47:0/471053409 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d380ad710 con 0x7f3d380b59b0 2020-07-18T03:26:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.047+0000 7f3d3659c700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380ad330 0x7f3d38143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d380b59b0 0x7f3d380b5e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f3d28002f30 tx=0x7f3d28006e90).ready entity=mon.1 client_cookie=3201096a7c1eeee0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3559a700 1 -- 172.21.15.47:0/471053409 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d28009850 con 0x7f3d380b59b0 2020-07-18T03:26:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3559a700 1 -- 172.21.15.47:0/471053409 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d280099b0 con 0x7f3d380b59b0 2020-07-18T03:26:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3559a700 1 -- 172.21.15.47:0/471053409 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d28009d10 con 0x7f3d380b59b0 2020-07-18T03:26:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 -- 172.21.15.47:0/471053409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d380b59b0 msgr2=0x7f3d380b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d380b59b0 0x7f3d380b5e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f3d28002f30 tx=0x7f3d28006e90).stop 2020-07-18T03:26:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 -- 172.21.15.47:0/471053409 shutdown_connections 2020-07-18T03:26:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 --2- 172.21.15.47:0/471053409 >> v2:172.21.15.18:3300/0 conn(0x7f3d380b6300 0x7f3d380b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380ad330 0x7f3d38143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 --2- 172.21.15.47:0/471053409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d380b59b0 0x7f3d380b5e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 -- 172.21.15.47:0/471053409 >> 172.21.15.47:0/471053409 conn(0x7f3d3805f5a0 msgr2=0x7f3d3805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 -- 172.21.15.47:0/471053409 shutdown_connections 2020-07-18T03:26:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.048+0000 7f3d3cd90700 1 -- 172.21.15.47:0/471053409 wait complete. 2020-07-18T03:26:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.049+0000 7f3d3cd90700 1 Processor -- start 2020-07-18T03:26:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.049+0000 7f3d3cd90700 1 -- start start 2020-07-18T03:26:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d3cd90700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3d380ad330 0x7f3d38156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d3cd90700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380b6300 0x7f3d38061d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d3cd90700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d38156710 con 0x7f3d380667e0 2020-07-18T03:26:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d3cd90700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d381569b0 con 0x7f3d380ad330 2020-07-18T03:26:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d3cd90700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d38157340 con 0x7f3d380b6300 2020-07-18T03:26:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d36d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3d380667e0 0x7f3d3813da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40268/0 (socket says 172.21.15.47:40268) 2020-07-18T03:26:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d36d9d700 1 -- 172.21.15.47:0/788777453 learned_addr learned my addr 172.21.15.47:0/788777453 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380b6300 0x7f3d38061d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d3659c700 1 --2- 172.21.15.47:0/788777453 >> v2:172.21.15.18:3300/0 conn(0x7f3d380ad330 0x7f3d38156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549799463 0 0) 0x7f3d38156710 con 0x7f3d380667e0 2020-07-18T03:26:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.050+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c003240 con 0x7f3d380667e0 2020-07-18T03:26:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1977489462 0 0) 0x7f3d0c003240 con 0x7f3d380667e0 2020-07-18T03:26:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3d38156710 con 0x7f3d380667e0 2020-07-18T03:26:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f3d30009ae0 con 0x7f3d380667e0 2020-07-18T03:26:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380b6300 msgr2=0x7f3d38061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380b6300 0x7f3d38061d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 >> v2:172.21.15.18:3300/0 conn(0x7f3d380ad330 msgr2=0x7f3d38156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 --2- 172.21.15.47:0/788777453 >> v2:172.21.15.18:3300/0 conn(0x7f3d380ad330 0x7f3d38156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 >> v1:172.21.15.47:6789/0 conn(0x7f3d380667e0 legacy=0x7f3d3813da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c003800 0x7f3d0c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d0c0044e0 0x7f3d0c004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d0c004e10 0x7f3d0c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d381569b0 con 0x7f3d0c0044e0 2020-07-18T03:26:09.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d38157340 con 0x7f3d0c003800 2020-07-18T03:26:09.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c005370 con 0x7f3d0c004e10 2020-07-18T03:26:09.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d380b6300 0x7f3d38061d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:09.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3659c700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c003800 0x7f3d0c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d36d9d700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d0c004e10 0x7f3d0c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d0c0044e0 0x7f3d0c004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3659c700 1 -- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d0c004e10 msgr2=0x7f3d0c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3659c700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3d0c004e10 0x7f3d0c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3659c700 1 -- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d0c0044e0 msgr2=0x7f3d0c004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3659c700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d0c0044e0 0x7f3d0c004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3659c700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d380626b0 con 0x7f3d0c003800 2020-07-18T03:26:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3813e170 con 0x7f3d0c003800 2020-07-18T03:26:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.051+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d38064d90 con 0x7f3d0c003800 2020-07-18T03:26:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.052+0000 7f3d3659c700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c003800 0x7f3d0c003c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f3d0c0054b0 tx=0x7f3d2c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.052+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d2c00eb60 con 0x7f3d0c003800 2020-07-18T03:26:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.052+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c00ecc0 con 0x7f3d0c003800 2020-07-18T03:26:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.052+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d2c006020 con 0x7f3d0c003800 2020-07-18T03:26:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.053+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d38156570 con 0x7f3d0c003800 2020-07-18T03:26:09.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.053+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3d2c006180 con 0x7f3d0c003800 2020-07-18T03:26:09.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.053+0000 7f3d267fc700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d0c058700 0x7f3d0c05abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.053+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3d2c0681b0 con 0x7f3d0c003800 2020-07-18T03:26:09.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.053+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d0c058700 0x7f3d0c05abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.053+0000 7f3d35d9b700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d0c058700 0x7f3d0c05abc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3d28016000 tx=0x7f3d28015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.056+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3d38156570 con 0x7f3d0c003800 2020-07-18T03:26:09.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.211+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d38156570 con 0x7f3d0c003800 2020-07-18T03:26:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.212+0000 7f3d267fc700 1 -- 172.21.15.47:0/788777453 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3d38156570 con 0x7f3d0c003800 2020-07-18T03:26:09.214 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d0c058700 msgr2=0x7f3d0c05abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d0c058700 0x7f3d0c05abc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3d28016000 tx=0x7f3d28015000).stop 2020-07-18T03:26:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c003800 msgr2=0x7f3d0c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 --2- 172.21.15.47:0/788777453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c003800 0x7f3d0c003c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f3d0c0054b0 tx=0x7f3d2c008040).stop 2020-07-18T03:26:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d36d9d700 1 -- 172.21.15.47:0/788777453 reap_dead start 2020-07-18T03:26:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d36d9d700 1 -- 172.21.15.47:0/788777453 reap_dead start 2020-07-18T03:26:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 shutdown_connections 2020-07-18T03:26:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 >> 172.21.15.47:0/788777453 conn(0x7f3d3805f5a0 msgr2=0x7f3d38148210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 shutdown_connections 2020-07-18T03:26:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.213+0000 7f3d3cd90700 1 -- 172.21.15.47:0/788777453 wait complete. 2020-07-18T03:26:09.225 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.340+0000 7fe605419700 1 Processor -- start 2020-07-18T03:26:09.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.340+0000 7fe605419700 1 -- start start 2020-07-18T03:26:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe605419700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe600142a20 0x7fe600142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe605419700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6000b3af0 0x7fe6000b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe605419700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6000af4b0 con 0x7fe600061c60 2020-07-18T03:26:09.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe605419700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6000b6d80 con 0x7fe6000b3af0 2020-07-18T03:26:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe605419700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000af710 con 0x7fe600142a20 2020-07-18T03:26:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe5ff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6000b3af0 0x7fe6000b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe5ff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6000b3af0 0x7fe6000b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59192/0 (socket says 172.21.15.47:59192) 2020-07-18T03:26:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe5ff7fe700 1 -- 172.21.15.47:0/1167889033 learned_addr learned my addr 172.21.15.47:0/1167889033 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.341+0000 7fe5feffd700 1 --2- 172.21.15.47:0/1167889033 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe600142a20 0x7fe600142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5ff7fe700 1 -- 172.21.15.47:0/1167889033 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe600142a20 msgr2=0x7fe600142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5ff7fe700 1 --2- 172.21.15.47:0/1167889033 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe600142a20 0x7fe600142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5ff7fe700 1 -- 172.21.15.47:0/1167889033 >> v1:172.21.15.18:6789/0 conn(0x7fe600061c60 legacy=0x7fe6000620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5ff7fe700 1 -- 172.21.15.47:0/1167889033 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000af9b0 con 0x7fe6000b3af0 2020-07-18T03:26:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5ff7fe700 1 --2- 172.21.15.47:0/1167889033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6000b3af0 0x7fe6000b3f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fe6000afd30 tx=0x7fe5f8009270).ready entity=mon.1 client_cookie=7ccbd53661084d82 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5fdffb700 1 -- 172.21.15.47:0/1167889033 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f800bbc0 con 0x7fe6000b3af0 2020-07-18T03:26:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5fdffb700 1 -- 172.21.15.47:0/1167889033 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f800bd20 con 0x7fe6000b3af0 2020-07-18T03:26:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe5fdffb700 1 -- 172.21.15.47:0/1167889033 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f8011400 con 0x7fe6000b3af0 2020-07-18T03:26:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 -- 172.21.15.47:0/1167889033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6000b3af0 msgr2=0x7fe6000b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 --2- 172.21.15.47:0/1167889033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6000b3af0 0x7fe6000b3f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fe6000afd30 tx=0x7fe5f8009270).stop 2020-07-18T03:26:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 -- 172.21.15.47:0/1167889033 shutdown_connections 2020-07-18T03:26:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 --2- 172.21.15.47:0/1167889033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6000b3af0 0x7fe6000b3f60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 --2- 172.21.15.47:0/1167889033 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe600142a20 0x7fe600142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 -- 172.21.15.47:0/1167889033 >> 172.21.15.47:0/1167889033 conn(0x7fe60005f5a0 msgr2=0x7fe60005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 -- 172.21.15.47:0/1167889033 shutdown_connections 2020-07-18T03:26:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.342+0000 7fe605419700 1 -- 172.21.15.47:0/1167889033 wait complete. 2020-07-18T03:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe605419700 1 Processor -- start 2020-07-18T03:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe605419700 1 -- start start 2020-07-18T03:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe605419700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe600061c60 0x7fe60013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe605419700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6001460a0 con 0x7fe600142a20 2020-07-18T03:26:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe605419700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe600146340 con 0x7fe6000661e0 2020-07-18T03:26:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe605419700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6000666d0 con 0x7fe600061c60 2020-07-18T03:26:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe5ff7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe6000661e0 0x7fe60013f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40288/0 (socket says 172.21.15.47:40288) 2020-07-18T03:26:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.344+0000 7fe5ff7fe700 1 -- 172.21.15.47:0/3771737740 learned_addr learned my addr 172.21.15.47:0/3771737740 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5feffd700 1 --2- 172.21.15.47:0/3771737740 >> v2:172.21.15.18:3300/0 conn(0x7fe600061c60 0x7fe60013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494978389 0 0) 0x7fe600146340 con 0x7fe6000661e0 2020-07-18T03:26:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5dc003240 con 0x7fe6000661e0 2020-07-18T03:26:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5feffd700 1 -- 172.21.15.47:0/3771737740 >> v1:172.21.15.47:6789/0 conn(0x7fe6000661e0 legacy=0x7fe60013f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5feffd700 1 -- 172.21.15.47:0/3771737740 >> v1:172.21.15.18:6789/0 conn(0x7fe600142a20 legacy=0x7fe600063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5feffd700 1 -- 172.21.15.47:0/3771737740 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6001433e0 con 0x7fe600061c60 2020-07-18T03:26:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5feffd700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe600061c60 0x7fe60013cdf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe5f400b000 tx=0x7fe5f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f4011790 con 0x7fe600061c60 2020-07-18T03:26:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f40118f0 con 0x7fe600061c60 2020-07-18T03:26:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe600066b70 con 0x7fe600061c60 2020-07-18T03:26:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f4011c50 con 0x7fe600061c60 2020-07-18T03:26:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.345+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe600063990 con 0x7fe600061c60 2020-07-18T03:26:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.346+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6000b6b60 con 0x7fe600061c60 2020-07-18T03:26:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.347+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe5f4021070 con 0x7fe600061c60 2020-07-18T03:26:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.347+0000 7fe5e6ffd700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe5dc050360 0x7fe5dc052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.347+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe5f4067e60 con 0x7fe600061c60 2020-07-18T03:26:09.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.347+0000 7fe5fe7fc700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe5dc050360 0x7fe5dc052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.347+0000 7fe5fe7fc700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe5dc050360 0x7fe5dc052820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fe600142300 tx=0x7fe5f00078f0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.351+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe5f40405f0 con 0x7fe600061c60 2020-07-18T03:26:09.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.509+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe60005b120 con 0x7fe600061c60 2020-07-18T03:26:09.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.510+0000 7fe5e6ffd700 1 -- 172.21.15.47:0/3771737740 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe5f4015020 con 0x7fe600061c60 2020-07-18T03:26:09.512 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:09.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe5dc050360 msgr2=0x7fe5dc052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe5dc050360 0x7fe5dc052820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fe600142300 tx=0x7fe5f00078f0).stop 2020-07-18T03:26:09.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe600061c60 msgr2=0x7fe60013cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe600061c60 0x7fe60013cdf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe5f400b000 tx=0x7fe5f4008040).stop 2020-07-18T03:26:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 shutdown_connections 2020-07-18T03:26:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe5dc050360 0x7fe5dc052820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 --2- 172.21.15.47:0/3771737740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe600061c60 0x7fe60013cdf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 >> 172.21.15.47:0/3771737740 conn(0x7fe60005f5a0 msgr2=0x7fe6000b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 shutdown_connections 2020-07-18T03:26:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.511+0000 7fe605419700 1 -- 172.21.15.47:0/3771737740 wait complete. 2020-07-18T03:26:09.523 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:26:09.524 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.637+0000 7fb98339e700 1 Processor -- start 2020-07-18T03:26:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.638+0000 7fb98339e700 1 -- start start 2020-07-18T03:26:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.638+0000 7fb98339e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb97c0b0c40 0x7fb97c0b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.638+0000 7fb98339e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0b3130 con 0x7fb97c1437b0 2020-07-18T03:26:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.638+0000 7fb98339e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0b33d0 con 0x7fb97c0b2290 2020-07-18T03:26:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c143c40 con 0x7fb97c0b0c40 2020-07-18T03:26:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb982b9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb97c0b0c40 0x7fb97c0b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98239c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb97c0b2290 0x7fb97c0b6530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40292/0 (socket says 172.21.15.47:40292) 2020-07-18T03:26:09.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98239c700 1 -- 172.21.15.47:0/3400760145 learned_addr learned my addr 172.21.15.47:0/3400760145 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:09.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb982b9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb97c0b0c40 0x7fb97c0b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59206/0 (socket says 172.21.15.47:59206) 2020-07-18T03:26:09.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb982b9d700 1 -- 172.21.15.47:0/3400760145 >> v1:172.21.15.47:6789/0 conn(0x7fb97c0b2290 legacy=0x7fb97c0b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb982b9d700 1 -- 172.21.15.47:0/3400760145 >> v1:172.21.15.18:6789/0 conn(0x7fb97c1437b0 legacy=0x7fb97c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb982b9d700 1 -- 172.21.15.47:0/3400760145 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0b2670 con 0x7fb97c0b0c40 2020-07-18T03:26:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb982b9d700 1 --2- 172.21.15.47:0/3400760145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c0b0c40 0x7fb97c0b10f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fb9740032b0 tx=0x7fb9740091d0).ready entity=mon.1 client_cookie=e127459eadfdf467 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98139a700 1 -- 172.21.15.47:0/3400760145 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97400bc40 con 0x7fb97c0b0c40 2020-07-18T03:26:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98139a700 1 -- 172.21.15.47:0/3400760145 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97400bda0 con 0x7fb97c0b0c40 2020-07-18T03:26:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98139a700 1 -- 172.21.15.47:0/3400760145 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb974010660 con 0x7fb97c0b0c40 2020-07-18T03:26:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 -- 172.21.15.47:0/3400760145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c0b0c40 msgr2=0x7fb97c0b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 --2- 172.21.15.47:0/3400760145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c0b0c40 0x7fb97c0b10f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fb9740032b0 tx=0x7fb9740091d0).stop 2020-07-18T03:26:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 -- 172.21.15.47:0/3400760145 shutdown_connections 2020-07-18T03:26:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 --2- 172.21.15.47:0/3400760145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c0b0c40 0x7fb97c0b10f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 -- 172.21.15.47:0/3400760145 >> 172.21.15.47:0/3400760145 conn(0x7fb97c0b9e50 msgr2=0x7fb97c0b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 -- 172.21.15.47:0/3400760145 shutdown_connections 2020-07-18T03:26:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.639+0000 7fb98339e700 1 -- 172.21.15.47:0/3400760145 wait complete. 2020-07-18T03:26:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.641+0000 7fb98339e700 1 Processor -- start 2020-07-18T03:26:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.641+0000 7fb98339e700 1 -- start start 2020-07-18T03:26:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.641+0000 7fb98339e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb97c135190 0x7fb97c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.641+0000 7fb98339e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c137b20 con 0x7fb97c1437b0 2020-07-18T03:26:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.641+0000 7fb98339e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c1514f0 con 0x7fb97c0b2290 2020-07-18T03:26:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.641+0000 7fb98339e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c146580 con 0x7fb97c135190 2020-07-18T03:26:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb98239c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb97c0b2290 0x7fb97c137f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40296/0 (socket says 172.21.15.47:40296) 2020-07-18T03:26:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb98239c700 1 -- 172.21.15.47:0/3204643476 learned_addr learned my addr 172.21.15.47:0/3204643476 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:09.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb982b9d700 1 --2- 172.21.15.47:0/3204643476 >> v2:172.21.15.47:3300/0 conn(0x7fb97c135190 0x7fb97c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb982b9d700 1 -- 172.21.15.47:0/3204643476 >> v1:172.21.15.47:6789/0 conn(0x7fb97c0b2290 legacy=0x7fb97c137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765913686 0 0) 0x7fb97c137b20 con 0x7fb97c1437b0 2020-07-18T03:26:09.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb982b9d700 1 -- 172.21.15.47:0/3204643476 >> v1:172.21.15.18:6789/0 conn(0x7fb97c1437b0 legacy=0x7fb97c134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb982b9d700 1 -- 172.21.15.47:0/3204643476 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c1462e0 con 0x7fb97c135190 2020-07-18T03:26:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb982b9d700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c135190 0x7fb97c15b890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fb96c001110 tx=0x7fb974018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb974018780 con 0x7fb97c135190 2020-07-18T03:26:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9740188e0 con 0x7fb97c135190 2020-07-18T03:26:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c152960 con 0x7fb97c135190 2020-07-18T03:26:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb974018c40 con 0x7fb97c135190 2020-07-18T03:26:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.642+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c145d60 con 0x7fb97c135190 2020-07-18T03:26:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.643+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c143430 con 0x7fb97c135190 2020-07-18T03:26:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.643+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb97402b070 con 0x7fb97c135190 2020-07-18T03:26:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.644+0000 7fb96a7fc700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb95404fea0 0x7fb954052360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.644+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb97406b300 con 0x7fb97c135190 2020-07-18T03:26:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.644+0000 7fb98239c700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb95404fea0 0x7fb954052360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.644+0000 7fb98239c700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb95404fea0 0x7fb954052360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fb96c0010d0 tx=0x7fb96c007800).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.647+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb974043ad0 con 0x7fb97c135190 2020-07-18T03:26:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.854+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb97c151680 con 0x7fb97c135190 2020-07-18T03:26:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.854+0000 7fb96a7fc700 1 -- 172.21.15.47:0/3204643476 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2124 (secure 0 0 0) 0x7fb9740470b0 con 0x7fb97c135190 2020-07-18T03:26:09.857 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:09.857 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:49.581963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4387":{"gid":4387,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.47:6835/308881804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":308881804},{"type":"v1","addr":"172.21.15.47:6835","nonce":308881804}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:26:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb95404fea0 msgr2=0x7fb954052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb95404fea0 0x7fb954052360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fb96c0010d0 tx=0x7fb96c007800).stop 2020-07-18T03:26:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c135190 msgr2=0x7fb97c15b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c135190 0x7fb97c15b890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fb96c001110 tx=0x7fb974018000).stop 2020-07-18T03:26:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 shutdown_connections 2020-07-18T03:26:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb97c135190 0x7fb97c15b890 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 --2- 172.21.15.47:0/3204643476 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb95404fea0 0x7fb954052360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 >> 172.21.15.47:0/3204643476 conn(0x7fb97c0b9e50 msgr2=0x7fb97c0ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 shutdown_connections 2020-07-18T03:26:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:09.856+0000 7fb98339e700 1 -- 172.21.15.47:0/3204643476 wait complete. 2020-07-18T03:26:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2020-07-18T03:26:10.069 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:25:45.899311+0000', 'modified': '2020-07-18T03:25:49.581963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4387': {'gid': 4387, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.47:6835/308881804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 308881804}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 308881804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:10.070 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:10.071 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:10.214 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4387,"want_state":"up:active","state":"up:active","rank_uptime":21.618044222999998,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":21.874795784} 2020-07-18T03:26:10.215 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4387, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 21.618044223, "state": "up:active", "uptime": 21.874795784, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:10.216 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:26:10.216 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2020-07-18T03:26:10.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.217 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:26:10.235 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:10.236 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:10.236 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:10.236 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:10.236 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:10.236 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:10.237 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:10.237 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:10.237 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:10.237 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:10.237 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:10.238 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42820sec preferred_lft 42820sec 2020-07-18T03:26:10.238 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:26:10.238 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:10.238 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:10.238 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:10.239 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:10.239 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:10.239 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:26:10.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.240 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:26:10.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.362 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:26:10.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.401 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:26:10.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.475 INFO:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:26:10.549 INFO:teuthology.orchestra.run.smithi018.stdout:1 2020-07-18T03:26:10.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.550 INFO:teuthology.orchestra.run.smithi018:> route 2020-07-18T03:26:10.615 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2020-07-18T03:26:10.615 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:10.615 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:10.616 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:10.616 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:26:10.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.617 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:10.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.840 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:10.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.873 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:10.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:10.999 INFO:teuthology.orchestra.run.smithi018:> ip netns list 2020-07-18T03:26:11.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.014 INFO:teuthology.orchestra.run.smithi018:> ip netns list-id 2020-07-18T03:26:11.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.069 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2' 2020-07-18T03:26:11.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.142 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 0' 2020-07-18T03:26:11.215 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.1/16 2020-07-18T03:26:11.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.216 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.0' 2020-07-18T03:26:11.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.311 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:26:11.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.454 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up' 2020-07-18T03:26:11.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.526 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up' 2020-07-18T03:26:11.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.591 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254' 2020-07-18T03:26:11.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.649 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:26:11.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:11.742 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:26:11.847 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2020-07-18T03:26:11.848 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@smithi018.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2020-07-18T03:26:11.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:11.849 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:26:11.880 INFO:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2020-07-18T03:26:11.970 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:11.971 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:12.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:12.046 INFO:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2020-07-18T03:26:12.101 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:26:12.101 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:26:12.150 INFO:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2020-07-18T03:26:12.183 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:12.184 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:12.256 INFO:teuthology.orchestra.run.smithi018.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:12.257 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:12.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:12.258 INFO:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2020-07-18T03:26:12.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi018.stderr:2020-07-18T03:26:12.284+0000 7fd6fe8f81c0 -1 init, newargv = 0x55deb3c9b2e0 newargc=15 2020-07-18T03:26:12.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi018.stderr:ceph-fuse[26469]: starting ceph client 2020-07-18T03:26:12.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi018.stderr:ceph-fuse[26469]: starting fuse 2020-07-18T03:26:12.304 INFO:teuthology.orchestra.run.smithi018.stdout:47 2020-07-18T03:26:12.304 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:26:12.305 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:26:12.305 INFO:teuthology.orchestra.run.smithi018:> import glob 2020-07-18T03:26:12.306 INFO:teuthology.orchestra.run.smithi018:> import re 2020-07-18T03:26:12.306 INFO:teuthology.orchestra.run.smithi018:> import os 2020-07-18T03:26:12.306 INFO:teuthology.orchestra.run.smithi018:> import subprocess 2020-07-18T03:26:12.307 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:12.307 INFO:teuthology.orchestra.run.smithi018:> def _find_admin_socket(client_name): 2020-07-18T03:26:12.307 INFO:teuthology.orchestra.run.smithi018:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2020-07-18T03:26:12.308 INFO:teuthology.orchestra.run.smithi018:> files = glob.glob(asok_path) 2020-07-18T03:26:12.308 INFO:teuthology.orchestra.run.smithi018:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2020-07-18T03:26:12.309 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:12.309 INFO:teuthology.orchestra.run.smithi018:> # Given a non-glob path, it better be there 2020-07-18T03:26:12.309 INFO:teuthology.orchestra.run.smithi018:> if "*" not in asok_path: 2020-07-18T03:26:12.310 INFO:teuthology.orchestra.run.smithi018:> assert(len(files) == 1) 2020-07-18T03:26:12.310 INFO:teuthology.orchestra.run.smithi018:> return files[0] 2020-07-18T03:26:12.310 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:12.311 INFO:teuthology.orchestra.run.smithi018:> for f in files: 2020-07-18T03:26:12.311 INFO:teuthology.orchestra.run.smithi018:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:26:12.311 INFO:teuthology.orchestra.run.smithi018:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:26:12.312 INFO:teuthology.orchestra.run.smithi018:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:26:12.312 INFO:teuthology.orchestra.run.smithi018:> contents = proc_f.read() 2020-07-18T03:26:12.312 INFO:teuthology.orchestra.run.smithi018:> if mountpoint in contents: 2020-07-18T03:26:12.313 INFO:teuthology.orchestra.run.smithi018:> return f 2020-07-18T03:26:12.320 INFO:teuthology.orchestra.run.smithi018:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:26:12.320 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:12.321 INFO:teuthology.orchestra.run.smithi018:> print(_find_admin_socket("client.2")) 2020-07-18T03:26:12.321 INFO:teuthology.orchestra.run.smithi018:> ' 2020-07-18T03:26:12.437 INFO:teuthology.orchestra.run.smithi018.stdout:/var/run/ceph/ceph-client.2.26469.asok 2020-07-18T03:26:12.618 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.26469.asok 2020-07-18T03:26:12.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:12.619 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.26469.asok status 2020-07-18T03:26:12.772 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2020-07-18T03:26:12.773 INFO:teuthology.orchestra.run.smithi018.stdout: "metadata": { 2020-07-18T03:26:12.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:26:12.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:26:12.773 INFO:teuthology.orchestra.run.smithi018.stdout: "entity_id": "2", 2020-07-18T03:26:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": "smithi018", 2020-07-18T03:26:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2020-07-18T03:26:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: "pid": "26469", 2020-07-18T03:26:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: "root": "/" 2020-07-18T03:26:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_count": 0, 2020-07-18T03:26:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_pinned_count": 0, 2020-07-18T03:26:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4695, 2020-07-18T03:26:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "inst": { 2020-07-18T03:26:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "name": { 2020-07-18T03:26:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "client", 2020-07-18T03:26:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "num": 4695 2020-07-18T03:26:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2020-07-18T03:26:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2020-07-18T03:26:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:26:12.777 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 430614419 2020-07-18T03:26:12.777 INFO:teuthology.orchestra.run.smithi018.stdout: } 2020-07-18T03:26:12.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:12.777 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2020-07-18T03:26:12.777 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2020-07-18T03:26:12.778 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:26:12.778 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 430614419 2020-07-18T03:26:12.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:12.778 INFO:teuthology.orchestra.run.smithi018.stdout: "inst_str": "client.4695 192.168.0.1:0/430614419", 2020-07-18T03:26:12.778 INFO:teuthology.orchestra.run.smithi018.stdout: "addr_str": "192.168.0.1:0/430614419", 2020-07-18T03:26:12.779 INFO:teuthology.orchestra.run.smithi018.stdout: "inode_count": 1, 2020-07-18T03:26:12.779 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_epoch": 6, 2020-07-18T03:26:12.779 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch": 24, 2020-07-18T03:26:12.779 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:26:12.779 INFO:teuthology.orchestra.run.smithi018.stdout: "blacklisted": false 2020-07-18T03:26:12.779 INFO:teuthology.orchestra.run.smithi018.stdout:} 2020-07-18T03:26:12.781 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.908+0000 7f6a47fdc700 1 Processor -- start 2020-07-18T03:26:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.908+0000 7f6a47fdc700 1 -- start start 2020-07-18T03:26:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a47fdc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400ad100 0x7f6a400b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a47fdc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a40147a00 0x7f6a40147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a47fdc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a40147330 con 0x7f6a4005f5a0 2020-07-18T03:26:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a47fdc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a400ae530 con 0x7f6a40147a00 2020-07-18T03:26:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a47fdc700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400b57e0 con 0x7f6a400ad100 2020-07-18T03:26:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a45577700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a40147a00 0x7f6a40147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a46579700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6a4005f5a0 0x7f6a4005fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40306/0 (socket says 172.21.15.47:40306) 2020-07-18T03:26:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.909+0000 7f6a46579700 1 -- 172.21.15.47:0/2028125940 learned_addr learned my addr 172.21.15.47:0/2028125940 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a45d78700 1 --2- 172.21.15.47:0/2028125940 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400ad100 0x7f6a400b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a44d76700 1 -- 172.21.15.47:0/2028125940 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517421114 0 0) 0x7f6a40147330 con 0x7f6a4005f5a0 2020-07-18T03:26:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a44d76700 1 -- 172.21.15.47:0/2028125940 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a28003240 con 0x7f6a4005f5a0 2020-07-18T03:26:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a45577700 1 -- 172.21.15.47:0/2028125940 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400ad100 msgr2=0x7f6a400b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a45577700 1 --2- 172.21.15.47:0/2028125940 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400ad100 0x7f6a400b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a45577700 1 -- 172.21.15.47:0/2028125940 >> v1:172.21.15.47:6789/0 conn(0x7f6a4005f5a0 legacy=0x7f6a4005fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a45577700 1 -- 172.21.15.47:0/2028125940 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400b5a80 con 0x7f6a40147a00 2020-07-18T03:26:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a45577700 1 --2- 172.21.15.47:0/2028125940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a40147a00 0x7f6a40147e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f6a3c002f30 tx=0x7f6a3c006dd0).ready entity=mon.1 client_cookie=2c3b1c0f8872bba6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a44d76700 1 -- 172.21.15.47:0/2028125940 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3c009760 con 0x7f6a40147a00 2020-07-18T03:26:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a44d76700 1 -- 172.21.15.47:0/2028125940 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0098c0 con 0x7f6a40147a00 2020-07-18T03:26:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a44d76700 1 -- 172.21.15.47:0/2028125940 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3c009c20 con 0x7f6a40147a00 2020-07-18T03:26:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2028125940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a40147a00 msgr2=0x7f6a40147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2028125940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a40147a00 0x7f6a40147e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f6a3c002f30 tx=0x7f6a3c006dd0).stop 2020-07-18T03:26:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2028125940 shutdown_connections 2020-07-18T03:26:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2028125940 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400ad100 0x7f6a400b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2028125940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a40147a00 0x7f6a40147e40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2028125940 >> 172.21.15.47:0/2028125940 conn(0x7f6a400b9340 msgr2=0x7f6a400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2028125940 shutdown_connections 2020-07-18T03:26:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.910+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2028125940 wait complete. 2020-07-18T03:26:12.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.912+0000 7f6a47fdc700 1 Processor -- start 2020-07-18T03:26:12.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.912+0000 7f6a47fdc700 1 -- start start 2020-07-18T03:26:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.912+0000 7f6a47fdc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6a4005f5a0 0x7f6a40159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a47fdc700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a40159a10 con 0x7f6a401440b0 2020-07-18T03:26:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a47fdc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a40149fd0 con 0x7f6a400ad100 2020-07-18T03:26:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a47fdc700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4014a270 con 0x7f6a4005f5a0 2020-07-18T03:26:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a45577700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6a400ad100 0x7f6a400660c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40310/0 (socket says 172.21.15.47:40310) 2020-07-18T03:26:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a45577700 1 -- 172.21.15.47:0/2317110514 learned_addr learned my addr 172.21.15.47:0/2317110514 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a45d78700 1 --2- 172.21.15.47:0/2317110514 >> v2:172.21.15.18:3300/0 conn(0x7f6a4005f5a0 0x7f6a40159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242219088 0 0) 0x7f6a40149fd0 con 0x7f6a400ad100 2020-07-18T03:26:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a1c003240 con 0x7f6a400ad100 2020-07-18T03:26:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3686211748 0 0) 0x7f6a1c003240 con 0x7f6a400ad100 2020-07-18T03:26:12.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a40149fd0 con 0x7f6a400ad100 2020-07-18T03:26:12.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f6a3c009760 con 0x7f6a400ad100 2020-07-18T03:26:12.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 >> v2:172.21.15.18:3300/0 conn(0x7f6a4005f5a0 msgr2=0x7f6a40159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 --2- 172.21.15.47:0/2317110514 >> v2:172.21.15.18:3300/0 conn(0x7f6a4005f5a0 0x7f6a40159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:12.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 >> v1:172.21.15.47:6789/0 conn(0x7f6a400ad100 legacy=0x7f6a400660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 >> v1:172.21.15.18:6789/0 conn(0x7f6a401440b0 legacy=0x7f6a4014a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a1c003800 0x7f6a1c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a1c0044e0 0x7f6a1c004960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a1c004e50 0x7f6a1c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4014a270 con 0x7f6a1c004e50 2020-07-18T03:26:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c004250 con 0x7f6a1c003800 2020-07-18T03:26:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c00bc20 con 0x7f6a1c0044e0 2020-07-18T03:26:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.913+0000 7f6a45d78700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a1c003800 0x7f6a1c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a46579700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a1c004e50 0x7f6a1c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45577700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a1c0044e0 0x7f6a1c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45d78700 1 -- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a1c0044e0 msgr2=0x7f6a1c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45d78700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a1c0044e0 0x7f6a1c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45d78700 1 -- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a1c004e50 msgr2=0x7f6a1c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:12.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45d78700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a1c004e50 0x7f6a1c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:12.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45d78700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4014a510 con 0x7f6a1c003800 2020-07-18T03:26:12.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a46579700 1 -- 172.21.15.47:0/2317110514 reap_dead start 2020-07-18T03:26:12.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4014a9b0 con 0x7f6a1c003800 2020-07-18T03:26:12.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a45d78700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a1c003800 0x7f6a1c003be0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f6a1c0054b0 tx=0x7f6a34008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:12.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3400e970 con 0x7f6a1c003800 2020-07-18T03:26:12.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a40159760 con 0x7f6a1c003800 2020-07-18T03:26:12.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3400ead0 con 0x7f6a1c003800 2020-07-18T03:26:12.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.914+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3400ee30 con 0x7f6a1c003800 2020-07-18T03:26:12.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.915+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a400afdf0 con 0x7f6a1c003800 2020-07-18T03:26:12.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.915+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6a34014430 con 0x7f6a1c003800 2020-07-18T03:26:12.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.915+0000 7f6a31ffb700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c053da0 0x7f6a1c056260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:12.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.916+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6a34067d90 con 0x7f6a1c003800 2020-07-18T03:26:12.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.916+0000 7f6a45577700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c053da0 0x7f6a1c056260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:12.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.916+0000 7f6a45577700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c053da0 0x7f6a1c056260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6a3c01b000 tx=0x7f6a3c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:12.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:12.919+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6a340404c0 con 0x7f6a1c003800 2020-07-18T03:26:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.075+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a40148550 con 0x7f6a1c003800 2020-07-18T03:26:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.076+0000 7f6a31ffb700 1 -- 172.21.15.47:0/2317110514 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f6a34043ae0 con 0x7f6a1c003800 2020-07-18T03:26:13.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:13.078 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c053da0 msgr2=0x7f6a1c056260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c053da0 0x7f6a1c056260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6a3c01b000 tx=0x7f6a3c015000).stop 2020-07-18T03:26:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a1c003800 msgr2=0x7f6a1c003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a1c003800 0x7f6a1c003be0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f6a1c0054b0 tx=0x7f6a34008040).stop 2020-07-18T03:26:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 shutdown_connections 2020-07-18T03:26:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c053da0 0x7f6a1c056260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 --2- 172.21.15.47:0/2317110514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a1c003800 0x7f6a1c003be0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 >> 172.21.15.47:0/2317110514 conn(0x7f6a400b9340 msgr2=0x7f6a4005fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 shutdown_connections 2020-07-18T03:26:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.077+0000 7f6a47fdc700 1 -- 172.21.15.47:0/2317110514 wait complete. 2020-07-18T03:26:13.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:13.090 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:13.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 Processor -- start 2020-07-18T03:26:13.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 -- start start 2020-07-18T03:26:13.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe0540b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0540b6300 0x7fe0540b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe054147de0 con 0x7fe0540ad330 2020-07-18T03:26:13.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0540ae000 con 0x7fe0540b59b0 2020-07-18T03:26:13.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05ab02700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe054148010 con 0x7fe0540b6300 2020-07-18T03:26:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.208+0000 7fe05909f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0540b6300 0x7fe0540b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0540b6300 0x7fe0540b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59236/0 (socket says 172.21.15.47:59236) 2020-07-18T03:26:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 -- 172.21.15.47:0/456954055 learned_addr learned my addr 172.21.15.47:0/456954055 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe053fff700 1 --2- 172.21.15.47:0/456954055 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe0540b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 -- 172.21.15.47:0/456954055 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 msgr2=0x7fe0540b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:13.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 --2- 172.21.15.47:0/456954055 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe0540b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 -- 172.21.15.47:0/456954055 >> v1:172.21.15.18:6789/0 conn(0x7fe0540ad330 legacy=0x7fe054143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 -- 172.21.15.47:0/456954055 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540ad710 con 0x7fe0540b6300 2020-07-18T03:26:13.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe053fff700 1 --2- 172.21.15.47:0/456954055 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe0540b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:13.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05909f700 1 --2- 172.21.15.47:0/456954055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540b6300 0x7fe0540b3a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fe04c0032b0 tx=0x7fe04c009270).ready entity=mon.1 client_cookie=800a12b4aa7e148e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe0537fe700 1 -- 172.21.15.47:0/456954055 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe04c00bc00 con 0x7fe0540b6300 2020-07-18T03:26:13.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe0537fe700 1 -- 172.21.15.47:0/456954055 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c00bd60 con 0x7fe0540b6300 2020-07-18T03:26:13.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe0537fe700 1 -- 172.21.15.47:0/456954055 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe04c011400 con 0x7fe0540b6300 2020-07-18T03:26:13.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 -- 172.21.15.47:0/456954055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540b6300 msgr2=0x7fe0540b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 --2- 172.21.15.47:0/456954055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540b6300 0x7fe0540b3a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fe04c0032b0 tx=0x7fe04c009270).stop 2020-07-18T03:26:13.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 -- 172.21.15.47:0/456954055 shutdown_connections 2020-07-18T03:26:13.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 --2- 172.21.15.47:0/456954055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540b6300 0x7fe0540b3a10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 --2- 172.21.15.47:0/456954055 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe0540b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 -- 172.21.15.47:0/456954055 >> 172.21.15.47:0/456954055 conn(0x7fe05405f5a0 msgr2=0x7fe05405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 -- 172.21.15.47:0/456954055 shutdown_connections 2020-07-18T03:26:13.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.209+0000 7fe05ab02700 1 -- 172.21.15.47:0/456954055 wait complete. 2020-07-18T03:26:13.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.211+0000 7fe05ab02700 1 Processor -- start 2020-07-18T03:26:13.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.211+0000 7fe05ab02700 1 -- start start 2020-07-18T03:26:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0540ad330 0x7fe05413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe054142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe054139b00 con 0x7fe054146090 2020-07-18T03:26:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05413abf0 con 0x7fe0540b59b0 2020-07-18T03:26:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe054139da0 con 0x7fe0540ad330 2020-07-18T03:26:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0540ad330 0x7fe05413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0540ad330 0x7fe05413ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59240/0 (socket says 172.21.15.47:59240) 2020-07-18T03:26:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 -- 172.21.15.47:0/2530971240 learned_addr learned my addr 172.21.15.47:0/2530971240 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe053fff700 1 --2- 172.21.15.47:0/2530971240 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe054142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 -- 172.21.15.47:0/2530971240 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 msgr2=0x7fe054142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 --2- 172.21.15.47:0/2530971240 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe054142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 -- 172.21.15.47:0/2530971240 >> v1:172.21.15.18:6789/0 conn(0x7fe054146090 legacy=0x7fe05413a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 -- 172.21.15.47:0/2530971240 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05413a040 con 0x7fe0540ad330 2020-07-18T03:26:13.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05889e700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540ad330 0x7fe05413ad60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fe0540ae4e0 tx=0x7fe044007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe04400e4a0 con 0x7fe0540ad330 2020-07-18T03:26:13.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04400e600 con 0x7fe0540ad330 2020-07-18T03:26:13.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe04400e960 con 0x7fe0540ad330 2020-07-18T03:26:13.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe054142c50 con 0x7fe0540ad330 2020-07-18T03:26:13.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.212+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05412e6b0 con 0x7fe0540ad330 2020-07-18T03:26:13.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.213+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05405c7c0 con 0x7fe0540ad330 2020-07-18T03:26:13.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.213+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe044019030 con 0x7fe0540ad330 2020-07-18T03:26:13.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.214+0000 7fe050ff9700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe02c04ffb0 0x7fe02c052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.214+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe0440670e0 con 0x7fe0540ad330 2020-07-18T03:26:13.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.214+0000 7fe053fff700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe02c04ffb0 0x7fe02c052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.214+0000 7fe053fff700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe02c04ffb0 0x7fe02c052470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe04800c000 tx=0x7fe048008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.217+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe04403f870 con 0x7fe0540ad330 2020-07-18T03:26:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.376+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe054062090 con 0x7fe0540ad330 2020-07-18T03:26:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.376+0000 7fe050ff9700 1 -- 172.21.15.47:0/2530971240 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe0440170b0 con 0x7fe0540ad330 2020-07-18T03:26:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe02c04ffb0 msgr2=0x7fe02c052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe02c04ffb0 0x7fe02c052470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe04800c000 tx=0x7fe048008040).stop 2020-07-18T03:26:13.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540ad330 msgr2=0x7fe05413ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540ad330 0x7fe05413ad60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fe0540ae4e0 tx=0x7fe044007c30).stop 2020-07-18T03:26:13.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 shutdown_connections 2020-07-18T03:26:13.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe02c04ffb0 0x7fe02c052470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 --2- 172.21.15.47:0/2530971240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0540ad330 0x7fe05413ad60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 --2- 172.21.15.47:0/2530971240 >> v2:172.21.15.18:3300/0 conn(0x7fe0540b59b0 0x7fe054142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 >> 172.21.15.47:0/2530971240 conn(0x7fe05405f5a0 msgr2=0x7fe0540622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.377+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 shutdown_connections 2020-07-18T03:26:13.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.378+0000 7fe05ab02700 1 -- 172.21.15.47:0/2530971240 wait complete. 2020-07-18T03:26:13.388 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:13.390 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.506+0000 7f5322496700 1 Processor -- start 2020-07-18T03:26:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.506+0000 7f5322496700 1 -- start start 2020-07-18T03:26:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5322496700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c0619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5322496700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f531c145690 0x7f531c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5322496700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c149120 con 0x7f531c13eda0 2020-07-18T03:26:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5322496700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c144d60 con 0x7f531c145690 2020-07-18T03:26:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5322496700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c144f50 con 0x7f531c148d40 2020-07-18T03:26:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5321494700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c0619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5320c93700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f531c145690 0x7f531c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5321c95700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f531c13eda0 0x7f531c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59102/0 (socket says 172.21.15.47:59102) 2020-07-18T03:26:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.507+0000 7f5321c95700 1 -- 172.21.15.47:0/3313829763 learned_addr learned my addr 172.21.15.47:0/3313829763 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5320c93700 1 -- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 msgr2=0x7f531c0619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5320c93700 1 --2- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c0619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5320c93700 1 -- 172.21.15.47:0/3313829763 >> v1:172.21.15.18:6789/0 conn(0x7f531c13eda0 legacy=0x7f531c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5320c93700 1 -- 172.21.15.47:0/3313829763 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c1451f0 con 0x7f531c145690 2020-07-18T03:26:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f530bfff700 1 -- 172.21.15.47:0/3313829763 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318503867 0 0) 0x7f531c149120 con 0x7f531c13eda0 2020-07-18T03:26:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5320c93700 1 --2- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f531c145690 0x7f531c13e8b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5318002f30 tx=0x7f5318006dd0).ready entity=mon.0 client_cookie=317b6c69f6284ab2 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f530bfff700 1 -- 172.21.15.47:0/3313829763 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5318009760 con 0x7f531c145690 2020-07-18T03:26:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f530bfff700 1 -- 172.21.15.47:0/3313829763 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53180098c0 con 0x7f531c145690 2020-07-18T03:26:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f530bfff700 1 -- 172.21.15.47:0/3313829763 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5318009c20 con 0x7f531c145690 2020-07-18T03:26:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 -- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f531c145690 msgr2=0x7f531c13e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 --2- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f531c145690 0x7f531c13e8b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5318002f30 tx=0x7f5318006dd0).stop 2020-07-18T03:26:13.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 -- 172.21.15.47:0/3313829763 shutdown_connections 2020-07-18T03:26:13.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 --2- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c0619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 --2- 172.21.15.47:0/3313829763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f531c145690 0x7f531c13e8b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 -- 172.21.15.47:0/3313829763 >> 172.21.15.47:0/3313829763 conn(0x7f531c0b91d0 msgr2=0x7f531c0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 -- 172.21.15.47:0/3313829763 shutdown_connections 2020-07-18T03:26:13.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.508+0000 7f5322496700 1 -- 172.21.15.47:0/3313829763 wait complete. 2020-07-18T03:26:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.510+0000 7f5322496700 1 Processor -- start 2020-07-18T03:26:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.510+0000 7f5322496700 1 -- start start 2020-07-18T03:26:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5322496700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c13cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5322496700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c140070 con 0x7f531c13c3d0 2020-07-18T03:26:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5322496700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c1442c0 con 0x7f531c13eda0 2020-07-18T03:26:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5322496700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c13f820 con 0x7f531c148d40 2020-07-18T03:26:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5321494700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f531c13eda0 0x7f531c1407a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40340/0 (socket says 172.21.15.47:40340) 2020-07-18T03:26:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5321494700 1 -- 172.21.15.47:0/2705603827 learned_addr learned my addr 172.21.15.47:0/2705603827 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5320c93700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c13cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667904995 0 0) 0x7f531c1442c0 con 0x7f531c13eda0 2020-07-18T03:26:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f8003240 con 0x7f531c13eda0 2020-07-18T03:26:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922056011 0 0) 0x7f531c140070 con 0x7f531c13c3d0 2020-07-18T03:26:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f531c1442c0 con 0x7f531c13c3d0 2020-07-18T03:26:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5320c93700 1 -- 172.21.15.47:0/2705603827 >> v1:172.21.15.47:6789/0 conn(0x7f531c13eda0 legacy=0x7f531c1407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5320c93700 1 -- 172.21.15.47:0/2705603827 >> v1:172.21.15.18:6789/0 conn(0x7f531c13c3d0 legacy=0x7f531c0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5320c93700 1 -- 172.21.15.47:0/2705603827 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c13f580 con 0x7f531c148d40 2020-07-18T03:26:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.511+0000 7f5320c93700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c13cbc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5318002f30 tx=0x7f5318015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.512+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5318021070 con 0x7f531c148d40 2020-07-18T03:26:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.512+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531801c5c0 con 0x7f531c148d40 2020-07-18T03:26:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.512+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c13fcf0 con 0x7f531c148d40 2020-07-18T03:26:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.512+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f531801c920 con 0x7f531c148d40 2020-07-18T03:26:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.512+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c0ae8e0 con 0x7f531c148d40 2020-07-18T03:26:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.513+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c13fee0 con 0x7f531c148d40 2020-07-18T03:26:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.513+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f531800f070 con 0x7f531c148d40 2020-07-18T03:26:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.513+0000 7f53097fa700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f52f80504d0 0x7f52f8052990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.513+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f53180693a0 con 0x7f531c148d40 2020-07-18T03:26:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.513+0000 7f5321494700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f52f80504d0 0x7f52f8052990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.514+0000 7f5321494700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f52f80504d0 0x7f52f8052990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f530c00a000 tx=0x7f530c006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.517+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5318041a10 con 0x7f531c148d40 2020-07-18T03:26:13.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.674+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f531c144ba0 con 0x7f531c148d40 2020-07-18T03:26:13.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.674+0000 7f53097fa700 1 -- 172.21.15.47:0/2705603827 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5318045030 con 0x7f531c148d40 2020-07-18T03:26:13.676 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:13.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f52f80504d0 msgr2=0x7f52f8052990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f52f80504d0 0x7f52f8052990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f530c00a000 tx=0x7f530c006dd0).stop 2020-07-18T03:26:13.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 msgr2=0x7f531c13cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c13cbc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5318002f30 tx=0x7f5318015000).stop 2020-07-18T03:26:13.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 shutdown_connections 2020-07-18T03:26:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f52f80504d0 0x7f52f8052990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 --2- 172.21.15.47:0/2705603827 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f531c148d40 0x7f531c13cbc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 >> 172.21.15.47:0/2705603827 conn(0x7f531c0b91d0 msgr2=0x7f531c065020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 shutdown_connections 2020-07-18T03:26:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.675+0000 7f5322496700 1 -- 172.21.15.47:0/2705603827 wait complete. 2020-07-18T03:26:13.686 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:13.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 Processor -- start 2020-07-18T03:26:13.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 -- start start 2020-07-18T03:26:13.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc5a80ad330 0x7fc5a8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 0x7fc5a80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a8147de0 con 0x7fc5a80b6300 2020-07-18T03:26:13.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80ae000 con 0x7fc5a80ad330 2020-07-18T03:26:13.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.804+0000 7fc5ae94e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8148010 con 0x7fc5a80b59b0 2020-07-18T03:26:13.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 0x7fc5a80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a7fff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc5a80ad330 0x7fc5a8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 0x7fc5a80b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51862/0 (socket says 172.21.15.47:51862) 2020-07-18T03:26:13.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a77fe700 1 -- 172.21.15.47:0/2532887766 learned_addr learned my addr 172.21.15.47:0/2532887766 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:13.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5aceeb700 1 --1- 172.21.15.47:0/2532887766 >> v1:172.21.15.18:6789/0 conn(0x7fc5a80b6300 0x7fc5a80b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59116/0 (socket says 172.21.15.47:59116) 2020-07-18T03:26:13.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a7fff700 1 -- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 msgr2=0x7fc5a80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:13.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a7fff700 1 --2- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 0x7fc5a80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a7fff700 1 -- 172.21.15.47:0/2532887766 >> v1:172.21.15.18:6789/0 conn(0x7fc5a80b6300 legacy=0x7fc5a80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a6ffd700 1 -- 172.21.15.47:0/2532887766 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930391613 0 0) 0x7fc5a8147de0 con 0x7fc5a80b6300 2020-07-18T03:26:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a7fff700 1 -- 172.21.15.47:0/2532887766 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80ad710 con 0x7fc5a80ad330 2020-07-18T03:26:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a77fe700 1 --2- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 0x7fc5a80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a7fff700 1 --2- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a80ad330 0x7fc5a8143640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fc598002f30 tx=0x7fc598006dd0).ready entity=mon.0 client_cookie=799dca65e630787a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.805+0000 7fc5a6ffd700 1 -- 172.21.15.47:0/2532887766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc598009760 con 0x7fc5a80ad330 2020-07-18T03:26:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5a6ffd700 1 -- 172.21.15.47:0/2532887766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5980098c0 con 0x7fc5a80ad330 2020-07-18T03:26:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5a6ffd700 1 -- 172.21.15.47:0/2532887766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc598009c20 con 0x7fc5a80ad330 2020-07-18T03:26:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a80ad330 msgr2=0x7fc5a8143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a80ad330 0x7fc5a8143640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fc598002f30 tx=0x7fc598006dd0).stop 2020-07-18T03:26:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2532887766 shutdown_connections 2020-07-18T03:26:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a80ad330 0x7fc5a8143640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2532887766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5a80b59b0 0x7fc5a80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2532887766 >> 172.21.15.47:0/2532887766 conn(0x7fc5a805f5a0 msgr2=0x7fc5a805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2532887766 shutdown_connections 2020-07-18T03:26:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.806+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2532887766 wait complete. 2020-07-18T03:26:13.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.807+0000 7fc5ae94e700 1 Processor -- start 2020-07-18T03:26:13.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.807+0000 7fc5ae94e700 1 -- start start 2020-07-18T03:26:13.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5ae94e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5a80b6300 0x7fc5a8063ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5ae94e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc5a8066570 0x7fc5a8065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5ae94e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a8065590 con 0x7fc5a80b59b0 2020-07-18T03:26:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5ae94e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80653b0 con 0x7fc5a8066570 2020-07-18T03:26:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5ae94e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a8065830 con 0x7fc5a80b6300 2020-07-18T03:26:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5a80b6300 0x7fc5a8063ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5a80b6300 0x7fc5a8063ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59268/0 (socket says 172.21.15.47:59268) 2020-07-18T03:26:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 -- 172.21.15.47:0/2053498399 learned_addr learned my addr 172.21.15.47:0/2053498399 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5aceeb700 1 --2- 172.21.15.47:0/2053498399 >> v2:172.21.15.18:3300/0 conn(0x7fc5a8066570 0x7fc5a8065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 -- 172.21.15.47:0/2053498399 >> v2:172.21.15.18:3300/0 conn(0x7fc5a8066570 msgr2=0x7fc5a8065cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 --2- 172.21.15.47:0/2053498399 >> v2:172.21.15.18:3300/0 conn(0x7fc5a8066570 0x7fc5a8065cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 -- 172.21.15.47:0/2053498399 >> v1:172.21.15.47:6789/0 conn(0x7fc5a80b59b0 legacy=0x7fc5a813ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415953415 0 0) 0x7fc5a8065590 con 0x7fc5a80b59b0 2020-07-18T03:26:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 -- 172.21.15.47:0/2053498399 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a813bf70 con 0x7fc5a80b6300 2020-07-18T03:26:13.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.808+0000 7fc5a77fe700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5a80b6300 0x7fc5a8063ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fc59c00c000 tx=0x7fc59c006e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.809+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc59c00cbb0 con 0x7fc5a80b6300 2020-07-18T03:26:13.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.809+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00cd10 con 0x7fc5a80b6300 2020-07-18T03:26:13.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.809+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80661c0 con 0x7fc5a80b6300 2020-07-18T03:26:13.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.809+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc59c0125e0 con 0x7fc5a80b6300 2020-07-18T03:26:13.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.809+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a81418c0 con 0x7fc5a80b6300 2020-07-18T03:26:13.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.810+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a80b0230 con 0x7fc5a80b6300 2020-07-18T03:26:13.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.810+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc59c012790 con 0x7fc5a80b6300 2020-07-18T03:26:13.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.810+0000 7fc587fff700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc58004fdd0 0x7fc580052290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:13.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.810+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc59c0680d0 con 0x7fc5a80b6300 2020-07-18T03:26:13.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.810+0000 7fc5a7fff700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc58004fdd0 0x7fc580052290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:13.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.811+0000 7fc5a7fff700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc58004fdd0 0x7fc580052290 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc598002ef0 tx=0x7fc598015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:13.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.814+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc59c040860 con 0x7fc5a80b6300 2020-07-18T03:26:13.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.969+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5a805b120 con 0x7fc5a80b6300 2020-07-18T03:26:13.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.970+0000 7fc587fff700 1 -- 172.21.15.47:0/2053498399 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc59c043e80 con 0x7fc5a80b6300 2020-07-18T03:26:13.971 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:13.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc58004fdd0 msgr2=0x7fc580052290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc58004fdd0 0x7fc580052290 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc598002ef0 tx=0x7fc598015000).stop 2020-07-18T03:26:13.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5a80b6300 msgr2=0x7fc5a8063ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:13.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5a80b6300 0x7fc5a8063ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fc59c00c000 tx=0x7fc59c006e90).stop 2020-07-18T03:26:13.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 shutdown_connections 2020-07-18T03:26:13.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2053498399 >> v2:172.21.15.18:3300/0 conn(0x7fc5a8066570 0x7fc5a8065cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc58004fdd0 0x7fc580052290 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 --2- 172.21.15.47:0/2053498399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5a80b6300 0x7fc5a8063ad0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:13.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 >> 172.21.15.47:0/2053498399 conn(0x7fc5a805f5a0 msgr2=0x7fc5a80af300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:13.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 shutdown_connections 2020-07-18T03:26:13.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:13.971+0000 7fc5ae94e700 1 -- 172.21.15.47:0/2053498399 wait complete. 2020-07-18T03:26:13.982 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 Processor -- start 2020-07-18T03:26:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 -- start start 2020-07-18T03:26:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5ee00ad0d0 0x7f5ee0143c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee005f760 0x7f5ee0140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee00ad570 con 0x7f5ee0141290 2020-07-18T03:26:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee00ad810 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.100+0000 7f5edffff700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00adab0 con 0x7f5ee005f760 2020-07-18T03:26:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5ede7fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee005f760 0x7f5ee0140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edeffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5ee00ad0d0 0x7f5ee0143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edf7fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f5ee0141290 0x7f5ee0144e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59132/0 (socket says 172.21.15.47:59132) 2020-07-18T03:26:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edeffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5ee00ad0d0 0x7f5ee0143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:44954/0 (socket says 172.21.15.47:44954) 2020-07-18T03:26:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edf7fe700 1 -- 172.21.15.47:0/906643316 learned_addr learned my addr 172.21.15.47:0/906643316 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5ede7fc700 1 -- 172.21.15.47:0/906643316 >> v2:172.21.15.18:3300/0 conn(0x7f5ee00ad0d0 msgr2=0x7f5ee0143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5ede7fc700 1 --2- 172.21.15.47:0/906643316 >> v2:172.21.15.18:3300/0 conn(0x7f5ee00ad0d0 0x7f5ee0143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5ede7fc700 1 -- 172.21.15.47:0/906643316 >> v1:172.21.15.18:6789/0 conn(0x7f5ee0141290 legacy=0x7f5ee0144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5ede7fc700 1 -- 172.21.15.47:0/906643316 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00606c0 con 0x7f5ee005f760 2020-07-18T03:26:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5ede7fc700 1 --2- 172.21.15.47:0/906643316 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee005f760 0x7f5ee0140da0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5ed00032b0 tx=0x7f5ed00091d0).ready entity=mon.2 client_cookie=8ec269f1c06b9249 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5eddffb700 1 -- 172.21.15.47:0/906643316 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ed000b9a0 con 0x7f5ee005f760 2020-07-18T03:26:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5eddffb700 1 -- 172.21.15.47:0/906643316 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed000bb00 con 0x7f5ee005f760 2020-07-18T03:26:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5eddffb700 1 -- 172.21.15.47:0/906643316 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ed000bc70 con 0x7f5ee005f760 2020-07-18T03:26:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edffff700 1 -- 172.21.15.47:0/906643316 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee005f760 msgr2=0x7f5ee0140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edffff700 1 --2- 172.21.15.47:0/906643316 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee005f760 0x7f5ee0140da0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5ed00032b0 tx=0x7f5ed00091d0).stop 2020-07-18T03:26:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edffff700 1 -- 172.21.15.47:0/906643316 shutdown_connections 2020-07-18T03:26:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edffff700 1 --2- 172.21.15.47:0/906643316 >> v2:172.21.15.18:3300/0 conn(0x7f5ee00ad0d0 0x7f5ee0143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edffff700 1 --2- 172.21.15.47:0/906643316 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee005f760 0x7f5ee0140da0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.101+0000 7f5edffff700 1 -- 172.21.15.47:0/906643316 >> 172.21.15.47:0/906643316 conn(0x7f5ee00b91d0 msgr2=0x7f5ee00b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.102+0000 7f5edffff700 1 -- 172.21.15.47:0/906643316 shutdown_connections 2020-07-18T03:26:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.102+0000 7f5edffff700 1 -- 172.21.15.47:0/906643316 wait complete. 2020-07-18T03:26:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.103+0000 7f5edffff700 1 Processor -- start 2020-07-18T03:26:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.103+0000 7f5edffff700 1 -- start start 2020-07-18T03:26:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee00ad0d0 0x7f5ee0149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edffff700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee00b4ce0 con 0x7f5ee0141290 2020-07-18T03:26:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee00b53b0 con 0x7f5ee013faf0 2020-07-18T03:26:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edffff700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0152bc0 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edf7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5ee013faf0 0x7f5ee0148010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40372/0 (socket says 172.21.15.47:40372) 2020-07-18T03:26:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edf7fe700 1 -- 172.21.15.47:0/2938877313 learned_addr learned my addr 172.21.15.47:0/2938877313 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edeffd700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee00ad0d0 0x7f5ee0149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511707393 0 0) 0x7f5ee00b53b0 con 0x7f5ee013faf0 2020-07-18T03:26:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ebc003240 con 0x7f5ee013faf0 2020-07-18T03:26:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624031088 0 0) 0x7f5ee00b4ce0 con 0x7f5ee0141290 2020-07-18T03:26:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee00b53b0 con 0x7f5ee0141290 2020-07-18T03:26:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edeffd700 1 -- 172.21.15.47:0/2938877313 >> v1:172.21.15.47:6789/0 conn(0x7f5ee013faf0 legacy=0x7f5ee0148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edeffd700 1 -- 172.21.15.47:0/2938877313 >> v1:172.21.15.18:6789/0 conn(0x7f5ee0141290 legacy=0x7f5ee0146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edeffd700 1 -- 172.21.15.47:0/2938877313 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00b4f10 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.104+0000 7f5edeffd700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee00ad0d0 0x7f5ee0149ac0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5ec800c000 tx=0x7f5ec8006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.105+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ec800ca10 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.105+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee00b14f0 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.105+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec800cb70 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.105+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ec800be70 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.105+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee0149740 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.106+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee0152d00 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.106+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5ec80123f0 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.106+0000 7f5ed6ffd700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5ebc050450 0x7f5ebc052910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.106+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5ec8067f20 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.106+0000 7f5ede7fc700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5ebc050450 0x7f5ebc052910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.107+0000 7f5ede7fc700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5ebc050450 0x7f5ebc052910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f5ed0006dc0 tx=0x7f5ed0016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.110+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5ec80405d0 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.270+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ee00b50a0 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.271+0000 7f5ed6ffd700 1 -- 172.21.15.47:0/2938877313 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5ec8043bf0 con 0x7f5ee00ad0d0 2020-07-18T03:26:14.272 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.272+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5ebc050450 msgr2=0x7f5ebc052910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.272+0000 7f5edffff700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5ebc050450 0x7f5ebc052910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f5ed0006dc0 tx=0x7f5ed0016000).stop 2020-07-18T03:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee00ad0d0 msgr2=0x7f5ee0149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee00ad0d0 0x7f5ee0149ac0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5ec800c000 tx=0x7f5ec8006dd0).stop 2020-07-18T03:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 shutdown_connections 2020-07-18T03:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5ebc050450 0x7f5ebc052910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 --2- 172.21.15.47:0/2938877313 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ee00ad0d0 0x7f5ee0149ac0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 >> 172.21.15.47:0/2938877313 conn(0x7f5ee00b91d0 msgr2=0x7f5ee0064b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 shutdown_connections 2020-07-18T03:26:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.273+0000 7f5edffff700 1 -- 172.21.15.47:0/2938877313 wait complete. 2020-07-18T03:26:14.284 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:26:14.285 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:14.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.400+0000 7fcdda885700 1 Processor -- start 2020-07-18T03:26:14.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.400+0000 7fcdda885700 1 -- start start 2020-07-18T03:26:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.400+0000 7fcdda885700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdd4147a00 0x7fcdd4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd4147330 con 0x7fcdd40ad100 2020-07-18T03:26:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd40b57e0 con 0x7fcdd41433e0 2020-07-18T03:26:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40ae530 con 0x7fcdd4147a00 2020-07-18T03:26:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdd4147a00 0x7fcdd4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd8e22700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcdd41433e0 0x7fcdd4143820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40376/0 (socket says 172.21.15.47:40376) 2020-07-18T03:26:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd8e22700 1 -- 172.21.15.47:0/1384842905 learned_addr learned my addr 172.21.15.47:0/1384842905 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd37fe700 1 -- 172.21.15.47:0/1384842905 >> v1:172.21.15.47:6789/0 conn(0x7fcdd41433e0 legacy=0x7fcdd4143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd37fe700 1 -- 172.21.15.47:0/1384842905 >> v1:172.21.15.18:6789/0 conn(0x7fcdd40ad100 legacy=0x7fcdd40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd37fe700 1 -- 172.21.15.47:0/1384842905 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40b5a80 con 0x7fcdd4147a00 2020-07-18T03:26:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd37fe700 1 --2- 172.21.15.47:0/1384842905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd4147a00 0x7fcdd4147e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fcdd405ca30 tx=0x7fcdc8006dd0).ready entity=mon.1 client_cookie=992940e46224184 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd2ffd700 1 -- 172.21.15.47:0/1384842905 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdc80097b0 con 0x7fcdd4147a00 2020-07-18T03:26:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd2ffd700 1 -- 172.21.15.47:0/1384842905 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc8009910 con 0x7fcdd4147a00 2020-07-18T03:26:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdd2ffd700 1 -- 172.21.15.47:0/1384842905 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdc8009c70 con 0x7fcdd4147a00 2020-07-18T03:26:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- 172.21.15.47:0/1384842905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd4147a00 msgr2=0x7fcdd4147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 --2- 172.21.15.47:0/1384842905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd4147a00 0x7fcdd4147e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fcdd405ca30 tx=0x7fcdc8006dd0).stop 2020-07-18T03:26:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- 172.21.15.47:0/1384842905 shutdown_connections 2020-07-18T03:26:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 --2- 172.21.15.47:0/1384842905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd4147a00 0x7fcdd4147e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- 172.21.15.47:0/1384842905 >> 172.21.15.47:0/1384842905 conn(0x7fcdd40b9340 msgr2=0x7fcdd40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- 172.21.15.47:0/1384842905 shutdown_connections 2020-07-18T03:26:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.401+0000 7fcdda885700 1 -- 172.21.15.47:0/1384842905 wait complete. 2020-07-18T03:26:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.403+0000 7fcdda885700 1 Processor -- start 2020-07-18T03:26:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.403+0000 7fcdda885700 1 -- start start 2020-07-18T03:26:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdda885700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcdd40ad100 0x7fcdd41562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdda885700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fcdd41433e0 0x7fcdd40660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdda885700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd40b7eb0 con 0x7fcdd40b7770 2020-07-18T03:26:14.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdda885700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40b7cd0 con 0x7fcdd41433e0 2020-07-18T03:26:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdda885700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40b1580 con 0x7fcdd40ad100 2020-07-18T03:26:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd8e22700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcdd40b7770 0x7fcdd4138ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40384/0 (socket says 172.21.15.47:40384) 2020-07-18T03:26:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd8e22700 1 -- 172.21.15.47:0/3328792253 learned_addr learned my addr 172.21.15.47:0/3328792253 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd37fe700 1 --2- 172.21.15.47:0/3328792253 >> v2:172.21.15.18:3300/0 conn(0x7fcdd41433e0 0x7fcdd40660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd3fff700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcdd40ad100 0x7fcdd41562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339242065 0 0) 0x7fcdd40b7eb0 con 0x7fcdd40b7770 2020-07-18T03:26:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac003240 con 0x7fcdd40b7770 2020-07-18T03:26:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd37fe700 1 -- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcdd40ad100 msgr2=0x7fcdd41562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd37fe700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcdd40ad100 0x7fcdd41562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd37fe700 1 -- 172.21.15.47:0/3328792253 >> v1:172.21.15.47:6789/0 conn(0x7fcdd40b7770 legacy=0x7fcdd4138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 913541243 0 0) 0x7fcdac003240 con 0x7fcdd40b7770 2020-07-18T03:26:14.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd37fe700 1 -- 172.21.15.47:0/3328792253 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40b8150 con 0x7fcdd41433e0 2020-07-18T03:26:14.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.404+0000 7fcdd3fff700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcdd40ad100 0x7fcdd41562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:14.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.405+0000 7fcdd37fe700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcdd41433e0 0x7fcdd40660a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fcdcc001d40 tx=0x7fcdc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:14.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.405+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdcc001ba0 con 0x7fcdd41433e0 2020-07-18T03:26:14.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.405+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc801e460 con 0x7fcdd41433e0 2020-07-18T03:26:14.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.405+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd40b85f0 con 0x7fcdd41433e0 2020-07-18T03:26:14.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.405+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdc801e5d0 con 0x7fcdd41433e0 2020-07-18T03:26:14.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.405+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd40b8d50 con 0x7fcdd41433e0 2020-07-18T03:26:14.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.406+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd40b82e0 con 0x7fcdd41433e0 2020-07-18T03:26:14.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.406+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fcdc801a040 con 0x7fcdd41433e0 2020-07-18T03:26:14.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.406+0000 7fcdb3fff700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcdac050310 0x7fcdac0527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:14.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.406+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcdc8068d10 con 0x7fcdd41433e0 2020-07-18T03:26:14.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.406+0000 7fcdd3fff700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcdac050310 0x7fcdac0527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:14.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.407+0000 7fcdd3fff700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcdac050310 0x7fcdac0527d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fcdc400d000 tx=0x7fcdc4007ac0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:14.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.410+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fcdc8041470 con 0x7fcdd41433e0 2020-07-18T03:26:14.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.620+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdd4057450 con 0x7fcdd41433e0 2020-07-18T03:26:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.621+0000 7fcdb3fff700 1 -- 172.21.15.47:0/3328792253 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2124 (secure 0 0 0) 0x7fcdc8023020 con 0x7fcdd41433e0 2020-07-18T03:26:14.622 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:14.623 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:49.581963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4387":{"gid":4387,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.47:6835/308881804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":308881804},{"type":"v1","addr":"172.21.15.47:6835","nonce":308881804}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:26:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcdac050310 msgr2=0x7fcdac0527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcdac050310 0x7fcdac0527d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fcdc400d000 tx=0x7fcdc4007ac0).stop 2020-07-18T03:26:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcdd41433e0 msgr2=0x7fcdd40660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcdd41433e0 0x7fcdd40660a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fcdcc001d40 tx=0x7fcdc8015000).stop 2020-07-18T03:26:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 shutdown_connections 2020-07-18T03:26:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcdac050310 0x7fcdac0527d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcdd40ad100 0x7fcdd41562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 --2- 172.21.15.47:0/3328792253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcdd41433e0 0x7fcdd40660a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.622+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 >> 172.21.15.47:0/3328792253 conn(0x7fcdd40b9340 msgr2=0x7fcdd41469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.623+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 shutdown_connections 2020-07-18T03:26:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:14.623+0000 7fcdda885700 1 -- 172.21.15.47:0/3328792253 wait complete. 2020-07-18T03:26:14.627 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2020-07-18T03:26:14.836 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:25:45.899311+0000', 'modified': '2020-07-18T03:25:49.581963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4387': {'gid': 4387, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.47:6835/308881804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 308881804}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 308881804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:14.836 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:14.837 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:14.974 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4387,"want_state":"up:active","state":"up:active","rank_uptime":26.382251628999999,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":26.639003735999999} 2020-07-18T03:26:14.975 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4387, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 26.382251629, "state": "up:active", "uptime": 26.639003736, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:14.975 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:26:14.976 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2020-07-18T03:26:14.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:14.977 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:26:14.995 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:14.997 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:14.997 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42815sec preferred_lft 42815sec 2020-07-18T03:26:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:26:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:14.998 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:14.999 INFO:teuthology.orchestra.run.smithi018.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 36:1b:a5:09:a1:58 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::804f:27ff:feee:118a/64 scope link 2020-07-18T03:26:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.000 INFO:teuthology.orchestra.run.smithi018.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:26:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 36:1b:a5:09:a1:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2020-07-18T03:26:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::341b:a5ff:fe09:a158/64 scope link 2020-07-18T03:26:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.002 INFO:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:26:15.069 INFO:teuthology.orchestra.run.smithi018.stdout:1 2020-07-18T03:26:15.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.071 INFO:teuthology.orchestra.run.smithi018:> route 2020-07-18T03:26:15.129 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2020-07-18T03:26:15.130 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:15.130 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:15.130 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:15.131 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:26:15.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.132 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:15.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.205 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:15.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.280 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:15.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.355 INFO:teuthology.orchestra.run.smithi018:> ip netns list 2020-07-18T03:26:15.409 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 0) 2020-07-18T03:26:15.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.410 INFO:teuthology.orchestra.run.smithi018:> ip netns list-id 2020-07-18T03:26:15.464 INFO:teuthology.orchestra.run.smithi018.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.2) 2020-07-18T03:26:15.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.465 INFO:teuthology.orchestra.run.smithi018:> ip netns list-id 2020-07-18T03:26:15.519 INFO:teuthology.orchestra.run.smithi018.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.2) 2020-07-18T03:26:15.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.520 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3' 2020-07-18T03:26:15.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.593 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1' 2020-07-18T03:26:15.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.665 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr' 2020-07-18T03:26:15.746 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.747 INFO:teuthology.orchestra.run.smithi018.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 86:df:f7:d2:11:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:26:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:26:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::84df:f7ff:fed2:11a3/64 scope link 2020-07-18T03:26:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.757 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr' 2020-07-18T03:26:15.837 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:15.837 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:15.838 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:15.838 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.838 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:15.839 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.839 INFO:teuthology.orchestra.run.smithi018.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:15.839 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 86:df:f7:d2:11:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:26:15.840 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:26:15.840 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.840 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::84df:f7ff:fed2:11a3/64 scope link 2020-07-18T03:26:15.841 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:15.849 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2020-07-18T03:26:15.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.851 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1' 2020-07-18T03:26:15.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:15.923 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:26:16.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:16.015 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up' 2020-07-18T03:26:16.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:16.070 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up' 2020-07-18T03:26:16.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:16.122 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254' 2020-07-18T03:26:16.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:16.173 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set brx.1 up' 2020-07-18T03:26:16.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:16.213 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set dev brx.1 master ceph-brx' 2020-07-18T03:26:16.315 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2020-07-18T03:26:16.315 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.3 at ubuntu@smithi018.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3... 2020-07-18T03:26:16.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:16.316 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:26:16.332 INFO:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2020-07-18T03:26:16.403 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:16.403 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:16.476 INFO:teuthology.orchestra.run.smithi018.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:16.477 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:16.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:16.478 INFO:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2020-07-18T03:26:16.532 INFO:teuthology.orchestra.run.smithi018.stdout:47 2020-07-18T03:26:16.533 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:26:16.534 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.3 /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:26:16.575 INFO:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2020-07-18T03:26:16.607 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:16.608 INFO:teuthology.orchestra.run.smithi018:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:16.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi018.stderr:2020-07-18T03:26:16.669+0000 7efe1fe8d1c0 -1 init, newargv = 0x56468f5fd860 newargc=15 2020-07-18T03:26:16.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi018.stderr:ceph-fuse[26993]: starting ceph client 2020-07-18T03:26:16.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi018.stderr:ceph-fuse[26993]: starting fuse 2020-07-18T03:26:16.684 INFO:teuthology.orchestra.run.smithi018.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:16.685 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:16.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:16.686 INFO:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2020-07-18T03:26:16.740 INFO:teuthology.orchestra.run.smithi018.stdout:47 2020-07-18T03:26:16.740 INFO:teuthology.orchestra.run.smithi018.stdout:48 2020-07-18T03:26:16.741 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:26:16.741 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:26:16.742 INFO:teuthology.orchestra.run.smithi018:> import glob 2020-07-18T03:26:16.742 INFO:teuthology.orchestra.run.smithi018:> import re 2020-07-18T03:26:16.742 INFO:teuthology.orchestra.run.smithi018:> import os 2020-07-18T03:26:16.743 INFO:teuthology.orchestra.run.smithi018:> import subprocess 2020-07-18T03:26:16.743 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:16.743 INFO:teuthology.orchestra.run.smithi018:> def _find_admin_socket(client_name): 2020-07-18T03:26:16.744 INFO:teuthology.orchestra.run.smithi018:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2020-07-18T03:26:16.744 INFO:teuthology.orchestra.run.smithi018:> files = glob.glob(asok_path) 2020-07-18T03:26:16.745 INFO:teuthology.orchestra.run.smithi018:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2020-07-18T03:26:16.745 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:16.745 INFO:teuthology.orchestra.run.smithi018:> # Given a non-glob path, it better be there 2020-07-18T03:26:16.746 INFO:teuthology.orchestra.run.smithi018:> if "*" not in asok_path: 2020-07-18T03:26:16.746 INFO:teuthology.orchestra.run.smithi018:> assert(len(files) == 1) 2020-07-18T03:26:16.746 INFO:teuthology.orchestra.run.smithi018:> return files[0] 2020-07-18T03:26:16.747 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:16.747 INFO:teuthology.orchestra.run.smithi018:> for f in files: 2020-07-18T03:26:16.747 INFO:teuthology.orchestra.run.smithi018:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:26:16.748 INFO:teuthology.orchestra.run.smithi018:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:26:16.748 INFO:teuthology.orchestra.run.smithi018:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:26:16.748 INFO:teuthology.orchestra.run.smithi018:> contents = proc_f.read() 2020-07-18T03:26:16.749 INFO:teuthology.orchestra.run.smithi018:> if mountpoint in contents: 2020-07-18T03:26:16.749 INFO:teuthology.orchestra.run.smithi018:> return f 2020-07-18T03:26:16.750 INFO:teuthology.orchestra.run.smithi018:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:26:16.750 INFO:teuthology.orchestra.run.smithi018:> 2020-07-18T03:26:16.751 INFO:teuthology.orchestra.run.smithi018:> print(_find_admin_socket("client.3")) 2020-07-18T03:26:16.751 INFO:teuthology.orchestra.run.smithi018:> ' 2020-07-18T03:26:16.872 INFO:teuthology.orchestra.run.smithi018.stdout:/var/run/ceph/ceph-client.3.26993.asok 2020-07-18T03:26:17.053 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.26993.asok 2020-07-18T03:26:17.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.054 INFO:teuthology.orchestra.run.smithi018:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.26993.asok status 2020-07-18T03:26:17.207 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2020-07-18T03:26:17.207 INFO:teuthology.orchestra.run.smithi018.stdout: "metadata": { 2020-07-18T03:26:17.207 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:26:17.207 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:26:17.208 INFO:teuthology.orchestra.run.smithi018.stdout: "entity_id": "3", 2020-07-18T03:26:17.208 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": "smithi018", 2020-07-18T03:26:17.208 INFO:teuthology.orchestra.run.smithi018.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2020-07-18T03:26:17.208 INFO:teuthology.orchestra.run.smithi018.stdout: "pid": "26993", 2020-07-18T03:26:17.208 INFO:teuthology.orchestra.run.smithi018.stdout: "root": "/" 2020-07-18T03:26:17.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:17.209 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_count": 0, 2020-07-18T03:26:17.209 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_pinned_count": 0, 2020-07-18T03:26:17.209 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4490, 2020-07-18T03:26:17.209 INFO:teuthology.orchestra.run.smithi018.stdout: "inst": { 2020-07-18T03:26:17.210 INFO:teuthology.orchestra.run.smithi018.stdout: "name": { 2020-07-18T03:26:17.210 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "client", 2020-07-18T03:26:17.210 INFO:teuthology.orchestra.run.smithi018.stdout: "num": 4490 2020-07-18T03:26:17.210 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:17.210 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2020-07-18T03:26:17.211 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2020-07-18T03:26:17.211 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:26:17.211 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 1218280113 2020-07-18T03:26:17.211 INFO:teuthology.orchestra.run.smithi018.stdout: } 2020-07-18T03:26:17.211 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:17.212 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2020-07-18T03:26:17.212 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2020-07-18T03:26:17.212 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:26:17.212 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 1218280113 2020-07-18T03:26:17.212 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2020-07-18T03:26:17.212 INFO:teuthology.orchestra.run.smithi018.stdout: "inst_str": "client.4490 192.168.0.2:0/1218280113", 2020-07-18T03:26:17.213 INFO:teuthology.orchestra.run.smithi018.stdout: "addr_str": "192.168.0.2:0/1218280113", 2020-07-18T03:26:17.213 INFO:teuthology.orchestra.run.smithi018.stdout: "inode_count": 1, 2020-07-18T03:26:17.213 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_epoch": 6, 2020-07-18T03:26:17.213 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch": 24, 2020-07-18T03:26:17.213 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:26:17.214 INFO:teuthology.orchestra.run.smithi018.stdout: "blacklisted": false 2020-07-18T03:26:17.214 INFO:teuthology.orchestra.run.smithi018.stdout:} 2020-07-18T03:26:17.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.215 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:26:17.216 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:17.235 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:26:17.235 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:26:17.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.236 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:17.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.315 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:26:17.315 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:17.373 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:26:17.373 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:26:17.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.374 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:17.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.453 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:26:17.454 INFO:teuthology.orchestra.run.smithi018:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:26:17.470 INFO:teuthology.orchestra.run.smithi018.stdout:fuseblk 2020-07-18T03:26:17.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-07-18T03:26:17.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.472 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:26:17.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.548 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:26:17.549 INFO:teuthology.orchestra.run.smithi018:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:26:17.603 INFO:teuthology.orchestra.run.smithi018.stdout:fuseblk 2020-07-18T03:26:17.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2020-07-18T03:26:17.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:17.604 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:26:17.679 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-07-18T03:26:17.793 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:17.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.919+0000 7f98236a0700 1 Processor -- start 2020-07-18T03:26:17.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.920+0000 7f98236a0700 1 -- start start 2020-07-18T03:26:17.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.920+0000 7f98236a0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f981c0b5370 0x7f981c05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:17.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.920+0000 7f98236a0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c0b3ac0 con 0x7f981c0ad9e0 2020-07-18T03:26:17.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.920+0000 7f98236a0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c0b3d20 con 0x7f981c0b41c0 2020-07-18T03:26:17.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.920+0000 7f98236a0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c0b5a10 con 0x7f981c0b5370 2020-07-18T03:26:17.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f982143c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f981c0b5370 0x7f981c05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:17.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f9820c3b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f981c0b41c0 0x7f981c0ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40388/0 (socket says 172.21.15.47:40388) 2020-07-18T03:26:17.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f9820c3b700 1 -- 172.21.15.47:0/4282623093 learned_addr learned my addr 172.21.15.47:0/4282623093 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:17.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f982143c700 1 -- 172.21.15.47:0/4282623093 >> v1:172.21.15.47:6789/0 conn(0x7f981c0b41c0 legacy=0x7f981c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:17.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f982143c700 1 -- 172.21.15.47:0/4282623093 >> v1:172.21.15.18:6789/0 conn(0x7f981c0ad9e0 legacy=0x7f981c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:17.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f982143c700 1 -- 172.21.15.47:0/4282623093 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0b5750 con 0x7f981c0b5370 2020-07-18T03:26:17.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f980ffff700 1 -- 172.21.15.47:0/4282623093 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598649557 0 0) 0x7f981c0b3d20 con 0x7f981c0b41c0 2020-07-18T03:26:17.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f982143c700 1 --2- 172.21.15.47:0/4282623093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c0b5370 0x7f981c05fff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f981c067280 tx=0x7f9810006dd0).ready entity=mon.1 client_cookie=3df464f1f84dc6cd server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:17.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f980ffff700 1 -- 172.21.15.47:0/4282623093 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9810009770 con 0x7f981c0b5370 2020-07-18T03:26:17.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f980ffff700 1 -- 172.21.15.47:0/4282623093 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98100098d0 con 0x7f981c0b5370 2020-07-18T03:26:17.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f980ffff700 1 -- 172.21.15.47:0/4282623093 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9810009c30 con 0x7f981c0b5370 2020-07-18T03:26:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f98236a0700 1 -- 172.21.15.47:0/4282623093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c0b5370 msgr2=0x7f981c05fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f98236a0700 1 --2- 172.21.15.47:0/4282623093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c0b5370 0x7f981c05fff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f981c067280 tx=0x7f9810006dd0).stop 2020-07-18T03:26:17.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f98236a0700 1 -- 172.21.15.47:0/4282623093 shutdown_connections 2020-07-18T03:26:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f98236a0700 1 --2- 172.21.15.47:0/4282623093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f981c0b5370 0x7f981c05fff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.921+0000 7f98236a0700 1 -- 172.21.15.47:0/4282623093 >> 172.21.15.47:0/4282623093 conn(0x7f981c0b9340 msgr2=0x7f981c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.922+0000 7f98236a0700 1 -- 172.21.15.47:0/4282623093 shutdown_connections 2020-07-18T03:26:17.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.922+0000 7f98236a0700 1 -- 172.21.15.47:0/4282623093 wait complete. 2020-07-18T03:26:17.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.923+0000 7f98236a0700 1 Processor -- start 2020-07-18T03:26:17.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.923+0000 7f98236a0700 1 -- start start 2020-07-18T03:26:17.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f98236a0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f981c062d50 0x7f981c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:17.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f98236a0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c061910 con 0x7f981c0ad9e0 2020-07-18T03:26:17.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f98236a0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c13f790 con 0x7f981c0b41c0 2020-07-18T03:26:17.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f98236a0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c1448e0 con 0x7f981c062d50 2020-07-18T03:26:17.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f9820c3b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f981c0b41c0 0x7f981c0653f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40398/0 (socket says 172.21.15.47:40398) 2020-07-18T03:26:17.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f9820c3b700 1 -- 172.21.15.47:0/4206686140 learned_addr learned my addr 172.21.15.47:0/4206686140 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:17.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f9821c3d700 1 --2- 172.21.15.47:0/4206686140 >> v2:172.21.15.18:3300/0 conn(0x7f981c062d50 0x7f981c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:17.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055545235 0 0) 0x7f9818001030 con 0x7f981c0b41c0 2020-07-18T03:26:17.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f4003240 con 0x7f981c0b41c0 2020-07-18T03:26:17.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888704454 0 0) 0x7f981c061910 con 0x7f981c0ad9e0 2020-07-18T03:26:17.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.924+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f4003560 con 0x7f981c0ad9e0 2020-07-18T03:26:17.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f9821c3d700 1 -- 172.21.15.47:0/4206686140 >> v1:172.21.15.47:6789/0 conn(0x7f981c0b41c0 legacy=0x7f981c0653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:17.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3711064761 0 0) 0x7f97f4003240 con 0x7f981c0b41c0 2020-07-18T03:26:17.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f9821c3d700 1 -- 172.21.15.47:0/4206686140 >> v1:172.21.15.18:6789/0 conn(0x7f981c0ad9e0 legacy=0x7f981c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:17.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f9821c3d700 1 -- 172.21.15.47:0/4206686140 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c144b80 con 0x7f981c062d50 2020-07-18T03:26:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f9821c3d700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f981c062d50 0x7f981c1555e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f9818001b80 tx=0x7f9814007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98180019e0 con 0x7f981c062d50 2020-07-18T03:26:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9814006a80 con 0x7f981c062d50 2020-07-18T03:26:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9814006bf0 con 0x7f981c062d50 2020-07-18T03:26:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c1445d0 con 0x7f981c062d50 2020-07-18T03:26:17.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.925+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c0665c0 con 0x7f981c062d50 2020-07-18T03:26:17.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.926+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c0ba220 con 0x7f981c062d50 2020-07-18T03:26:17.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.926+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9814010030 con 0x7f981c062d50 2020-07-18T03:26:17.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.926+0000 7f980d7fa700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f40504c0 0x7f97f4052980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.926+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9814067600 con 0x7f981c062d50 2020-07-18T03:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.927+0000 7f982143c700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f40504c0 0x7f97f4052980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.927+0000 7f982143c700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f40504c0 0x7f97f4052980 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f9810003070 tx=0x7f9810015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:17.930+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f981403fd30 con 0x7f981c062d50 2020-07-18T03:26:18.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.089+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f981c057450 con 0x7f981c062d50 2020-07-18T03:26:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.090+0000 7f980d7fa700 1 -- 172.21.15.47:0/4206686140 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f9814043350 con 0x7f981c062d50 2020-07-18T03:26:18.091 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:18.092 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f40504c0 msgr2=0x7f97f4052980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f40504c0 0x7f97f4052980 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f9810003070 tx=0x7f9810015000).stop 2020-07-18T03:26:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f981c062d50 msgr2=0x7f981c1555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f981c062d50 0x7f981c1555e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f9818001b80 tx=0x7f9814007e40).stop 2020-07-18T03:26:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 shutdown_connections 2020-07-18T03:26:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f981c062d50 0x7f981c1555e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 --2- 172.21.15.47:0/4206686140 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97f40504c0 0x7f97f4052980 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.091+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 >> 172.21.15.47:0/4206686140 conn(0x7f981c0b9340 msgr2=0x7f981c0b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.092+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 shutdown_connections 2020-07-18T03:26:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.092+0000 7f98236a0700 1 -- 172.21.15.47:0/4206686140 wait complete. 2020-07-18T03:26:18.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:18.104 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:18.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.220+0000 7fdb496e4700 1 Processor -- start 2020-07-18T03:26:18.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.220+0000 7fdb496e4700 1 -- start start 2020-07-18T03:26:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb496e4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdb440ad020 0x7fdb440b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb496e4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb441478f0 0x7fdb44144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb496e4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb44147d80 0x7fdb44144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb496e4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4414b520 con 0x7fdb440ad020 2020-07-18T03:26:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb496e4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb441471b0 con 0x7fdb441478f0 2020-07-18T03:26:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb496e4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb44147450 con 0x7fdb44147d80 2020-07-18T03:26:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb441478f0 0x7fdb44144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb441478f0 0x7fdb44144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59314/0 (socket says 172.21.15.47:59314) 2020-07-18T03:26:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 -- 172.21.15.47:0/1936265368 learned_addr learned my addr 172.21.15.47:0/1936265368 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:18.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb42ffd700 1 --2- 172.21.15.47:0/1936265368 >> v2:172.21.15.18:3300/0 conn(0x7fdb440ad020 0x7fdb440b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb437fe700 1 --2- 172.21.15.47:0/1936265368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb44147d80 0x7fdb44144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 -- 172.21.15.47:0/1936265368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb44147d80 msgr2=0x7fdb44144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:18.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 --2- 172.21.15.47:0/1936265368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb44147d80 0x7fdb44144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 -- 172.21.15.47:0/1936265368 >> v2:172.21.15.18:3300/0 conn(0x7fdb440ad020 msgr2=0x7fdb440b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 --2- 172.21.15.47:0/1936265368 >> v2:172.21.15.18:3300/0 conn(0x7fdb440ad020 0x7fdb440b9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 -- 172.21.15.47:0/1936265368 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440ba390 con 0x7fdb441478f0 2020-07-18T03:26:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb42ffd700 1 --2- 172.21.15.47:0/1936265368 >> v2:172.21.15.18:3300/0 conn(0x7fdb440ad020 0x7fdb440b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.221+0000 7fdb427fc700 1 --2- 172.21.15.47:0/1936265368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb441478f0 0x7fdb44144230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fdb34002f30 tx=0x7fdb34006e30).ready entity=mon.1 client_cookie=262c187819452524 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb41ffb700 1 -- 172.21.15.47:0/1936265368 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb340098c0 con 0x7fdb441478f0 2020-07-18T03:26:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb41ffb700 1 -- 172.21.15.47:0/1936265368 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34009a20 con 0x7fdb441478f0 2020-07-18T03:26:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb41ffb700 1 -- 172.21.15.47:0/1936265368 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb34009d80 con 0x7fdb441478f0 2020-07-18T03:26:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 -- 172.21.15.47:0/1936265368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb441478f0 msgr2=0x7fdb44144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 --2- 172.21.15.47:0/1936265368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb441478f0 0x7fdb44144230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fdb34002f30 tx=0x7fdb34006e30).stop 2020-07-18T03:26:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 -- 172.21.15.47:0/1936265368 shutdown_connections 2020-07-18T03:26:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 --2- 172.21.15.47:0/1936265368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb44147d80 0x7fdb44144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 --2- 172.21.15.47:0/1936265368 >> v2:172.21.15.18:3300/0 conn(0x7fdb440ad020 0x7fdb440b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 --2- 172.21.15.47:0/1936265368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb441478f0 0x7fdb44144230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 -- 172.21.15.47:0/1936265368 >> 172.21.15.47:0/1936265368 conn(0x7fdb4405f5a0 msgr2=0x7fdb4405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 -- 172.21.15.47:0/1936265368 shutdown_connections 2020-07-18T03:26:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.222+0000 7fdb496e4700 1 -- 172.21.15.47:0/1936265368 wait complete. 2020-07-18T03:26:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.223+0000 7fdb496e4700 1 Processor -- start 2020-07-18T03:26:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.223+0000 7fdb496e4700 1 -- start start 2020-07-18T03:26:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb496e4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb440ad020 0x7fdb441453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb496e4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdb44147d80 0x7fdb44155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb496e4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb440b4ff0 0x7fdb440b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb496e4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb44159610 con 0x7fdb44147d80 2020-07-18T03:26:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb496e4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb44155230 con 0x7fdb440ad020 2020-07-18T03:26:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb496e4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb441554d0 con 0x7fdb440b4ff0 2020-07-18T03:26:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb440ad020 0x7fdb441453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb440ad020 0x7fdb441453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59320/0 (socket says 172.21.15.47:59320) 2020-07-18T03:26:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 -- 172.21.15.47:0/4243231441 learned_addr learned my addr 172.21.15.47:0/4243231441 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb437fe700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb440b4ff0 0x7fdb440b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb427fc700 1 --2- 172.21.15.47:0/4243231441 >> v2:172.21.15.18:3300/0 conn(0x7fdb44147d80 0x7fdb44155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 -- 172.21.15.47:0/4243231441 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb440b4ff0 msgr2=0x7fdb440b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb440b4ff0 0x7fdb440b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 -- 172.21.15.47:0/4243231441 >> v2:172.21.15.18:3300/0 conn(0x7fdb44147d80 msgr2=0x7fdb44155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.224+0000 7fdb42ffd700 1 --2- 172.21.15.47:0/4243231441 >> v2:172.21.15.18:3300/0 conn(0x7fdb44147d80 0x7fdb44155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb42ffd700 1 -- 172.21.15.47:0/4243231441 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb441458b0 con 0x7fdb440ad020 2020-07-18T03:26:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb427fc700 1 --2- 172.21.15.47:0/4243231441 >> v2:172.21.15.18:3300/0 conn(0x7fdb44147d80 0x7fdb44155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb42ffd700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb440ad020 0x7fdb441453c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fdb441464d0 tx=0x7fdb2c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb2c009980 con 0x7fdb440ad020 2020-07-18T03:26:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c009ae0 con 0x7fdb440ad020 2020-07-18T03:26:18.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb44155e60 con 0x7fdb440ad020 2020-07-18T03:26:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb2c009e40 con 0x7fdb440ad020 2020-07-18T03:26:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.225+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440b7220 con 0x7fdb440ad020 2020-07-18T03:26:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.226+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb44155640 con 0x7fdb440ad020 2020-07-18T03:26:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.226+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fdb2c013480 con 0x7fdb440ad020 2020-07-18T03:26:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.226+0000 7fdb3affd700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdb20050090 0x7fdb20052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.227+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdb2c068470 con 0x7fdb440ad020 2020-07-18T03:26:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.227+0000 7fdb427fc700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdb20050090 0x7fdb20052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.227+0000 7fdb427fc700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdb20050090 0x7fdb20052550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdb34017000 tx=0x7fdb34015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.230+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fdb2c040ba0 con 0x7fdb440ad020 2020-07-18T03:26:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.387+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdb440b31a0 con 0x7fdb440ad020 2020-07-18T03:26:18.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.387+0000 7fdb3affd700 1 -- 172.21.15.47:0/4243231441 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdb2c0441c0 con 0x7fdb440ad020 2020-07-18T03:26:18.389 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:18.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdb20050090 msgr2=0x7fdb20052550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdb20050090 0x7fdb20052550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdb34017000 tx=0x7fdb34015000).stop 2020-07-18T03:26:18.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb440ad020 msgr2=0x7fdb441453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb440ad020 0x7fdb441453c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fdb441464d0 tx=0x7fdb2c008040).stop 2020-07-18T03:26:18.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 shutdown_connections 2020-07-18T03:26:18.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdb20050090 0x7fdb20052550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdb440b4ff0 0x7fdb440b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 --2- 172.21.15.47:0/4243231441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb440ad020 0x7fdb441453c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 --2- 172.21.15.47:0/4243231441 >> v2:172.21.15.18:3300/0 conn(0x7fdb44147d80 0x7fdb44155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 >> 172.21.15.47:0/4243231441 conn(0x7fdb4405f5a0 msgr2=0x7fdb441465b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 shutdown_connections 2020-07-18T03:26:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.389+0000 7fdb496e4700 1 -- 172.21.15.47:0/4243231441 wait complete. 2020-07-18T03:26:18.400 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.517+0000 7f40a3322700 1 Processor -- start 2020-07-18T03:26:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.517+0000 7f40a3322700 1 -- start start 2020-07-18T03:26:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a3322700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409c0ad330 0x7f409c143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a3322700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f409c0b59b0 0x7f409c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a3322700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c147de0 con 0x7f409c0b6300 2020-07-18T03:26:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a3322700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c0ae000 con 0x7f409c0b59b0 2020-07-18T03:26:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a3322700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c148010 con 0x7f409c0ad330 2020-07-18T03:26:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409c0ad330 0x7f409c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409c0ad330 0x7f409c143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59328/0 (socket says 172.21.15.47:59328) 2020-07-18T03:26:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 -- 172.21.15.47:0/2759829806 learned_addr learned my addr 172.21.15.47:0/2759829806 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a08bd700 1 --2- 172.21.15.47:0/2759829806 >> v2:172.21.15.18:3300/0 conn(0x7f409c0b59b0 0x7f409c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 -- 172.21.15.47:0/2759829806 >> v2:172.21.15.18:3300/0 conn(0x7f409c0b59b0 msgr2=0x7f409c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:18.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 --2- 172.21.15.47:0/2759829806 >> v2:172.21.15.18:3300/0 conn(0x7f409c0b59b0 0x7f409c0b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 -- 172.21.15.47:0/2759829806 >> v1:172.21.15.18:6789/0 conn(0x7f409c0b6300 legacy=0x7f409c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.518+0000 7f40a10be700 1 -- 172.21.15.47:0/2759829806 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0ad710 con 0x7f409c0ad330 2020-07-18T03:26:18.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a10be700 1 --2- 172.21.15.47:0/2759829806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c0ad330 0x7f409c143640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f408c002f30 tx=0x7f408c006e90).ready entity=mon.1 client_cookie=6015682cba808d2a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f408bfff700 1 -- 172.21.15.47:0/2759829806 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c009850 con 0x7f409c0ad330 2020-07-18T03:26:18.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f408bfff700 1 -- 172.21.15.47:0/2759829806 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c0099b0 con 0x7f409c0ad330 2020-07-18T03:26:18.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f408bfff700 1 -- 172.21.15.47:0/2759829806 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c009d10 con 0x7f409c0ad330 2020-07-18T03:26:18.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 -- 172.21.15.47:0/2759829806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c0ad330 msgr2=0x7f409c143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 --2- 172.21.15.47:0/2759829806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c0ad330 0x7f409c143640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f408c002f30 tx=0x7f408c006e90).stop 2020-07-18T03:26:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 -- 172.21.15.47:0/2759829806 shutdown_connections 2020-07-18T03:26:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 --2- 172.21.15.47:0/2759829806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c0ad330 0x7f409c143640 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 --2- 172.21.15.47:0/2759829806 >> v2:172.21.15.18:3300/0 conn(0x7f409c0b59b0 0x7f409c0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 -- 172.21.15.47:0/2759829806 >> 172.21.15.47:0/2759829806 conn(0x7f409c05f5a0 msgr2=0x7f409c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 -- 172.21.15.47:0/2759829806 shutdown_connections 2020-07-18T03:26:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.519+0000 7f40a3322700 1 -- 172.21.15.47:0/2759829806 wait complete. 2020-07-18T03:26:18.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.520+0000 7f40a3322700 1 Processor -- start 2020-07-18T03:26:18.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.520+0000 7f40a3322700 1 -- start start 2020-07-18T03:26:18.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.521+0000 7f40a3322700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 0x7f409c065500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.521+0000 7f40a3322700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c155830 con 0x7f409c0b59b0 2020-07-18T03:26:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.521+0000 7f40a3322700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c155ad0 con 0x7f409c1551e0 2020-07-18T03:26:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.521+0000 7f40a3322700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c155d70 con 0x7f409c0b6300 2020-07-18T03:26:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.521+0000 7f40a18bf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f409c1551e0 0x7f409c0b9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40422/0 (socket says 172.21.15.47:40422) 2020-07-18T03:26:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.521+0000 7f40a18bf700 1 -- 172.21.15.47:0/3075209930 learned_addr learned my addr 172.21.15.47:0/3075209930 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:18.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389635640 0 0) 0x7f409c155ad0 con 0x7f409c1551e0 2020-07-18T03:26:18.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4074003240 con 0x7f409c1551e0 2020-07-18T03:26:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 0x7f409c065500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3350627682 0 0) 0x7f4074003240 con 0x7f409c1551e0 2020-07-18T03:26:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f409c155ad0 con 0x7f409c1551e0 2020-07-18T03:26:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4094001cc0 con 0x7f409c1551e0 2020-07-18T03:26:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 msgr2=0x7f409c065500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 0x7f409c065500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 >> v1:172.21.15.47:6789/0 conn(0x7f409c1551e0 legacy=0x7f409c0b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 >> v1:172.21.15.18:6789/0 conn(0x7f409c0b59b0 legacy=0x7f409c159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 0x7f4074003800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074003d10 0x7f40740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40740049d0 0x7f4074004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c155d70 con 0x7f40740049d0 2020-07-18T03:26:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4074004320 con 0x7f4074003d10 2020-07-18T03:26:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407400b760 con 0x7f409c0b6300 2020-07-18T03:26:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074003d10 0x7f40740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a10be700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 0x7f4074003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a18bf700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40740049d0 0x7f4074004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 -- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 msgr2=0x7f4074003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c0b6300 0x7f4074003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 -- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40740049d0 msgr2=0x7f4074004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40740049d0 0x7f4074004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0b9fa0 con 0x7f4074003d10 2020-07-18T03:26:18.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c113ad0 con 0x7f4074003d10 2020-07-18T03:26:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a18bf700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40740049d0 0x7f4074004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c065740 con 0x7f4074003d10 2020-07-18T03:26:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.522+0000 7f40a08bd700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074003d10 0x7f40740044e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f409800b000 tx=0x7f4098008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.523+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409800e9d0 con 0x7f4074003d10 2020-07-18T03:26:18.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.523+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409800eb30 con 0x7f4074003d10 2020-07-18T03:26:18.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.523+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409800ee90 con 0x7f4074003d10 2020-07-18T03:26:18.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.523+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c05b120 con 0x7f4074003d10 2020-07-18T03:26:18.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.524+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4098014470 con 0x7f4074003d10 2020-07-18T03:26:18.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.524+0000 7f40897fa700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4074058570 0x7f407405aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.524+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4098068290 con 0x7f4074003d10 2020-07-18T03:26:18.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.524+0000 7f40a10be700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4074058570 0x7f407405aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.524+0000 7f40a10be700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4074058570 0x7f407405aa30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f408c01c000 tx=0x7f408c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.527+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f40980409c0 con 0x7f4074003d10 2020-07-18T03:26:18.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.687+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f409c0690d0 con 0x7f4074003d10 2020-07-18T03:26:18.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.687+0000 7f40897fa700 1 -- 172.21.15.47:0/3075209930 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4098018020 con 0x7f4074003d10 2020-07-18T03:26:18.689 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.688+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4074058570 msgr2=0x7f407405aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.688+0000 7f40a3322700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4074058570 0x7f407405aa30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f408c01c000 tx=0x7f408c015000).stop 2020-07-18T03:26:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.688+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074003d10 msgr2=0x7f40740044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.688+0000 7f40a3322700 1 --2- 172.21.15.47:0/3075209930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074003d10 0x7f40740044e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f409800b000 tx=0x7f4098008040).stop 2020-07-18T03:26:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.688+0000 7f40a18bf700 1 -- 172.21.15.47:0/3075209930 reap_dead start 2020-07-18T03:26:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.688+0000 7f40a18bf700 1 -- 172.21.15.47:0/3075209930 reap_dead start 2020-07-18T03:26:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.689+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 shutdown_connections 2020-07-18T03:26:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.689+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 >> 172.21.15.47:0/3075209930 conn(0x7f409c05f5a0 msgr2=0x7f409c141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.689+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 shutdown_connections 2020-07-18T03:26:18.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.689+0000 7f40a3322700 1 -- 172.21.15.47:0/3075209930 wait complete. 2020-07-18T03:26:18.701 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:18.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 Processor -- start 2020-07-18T03:26:18.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 -- start start 2020-07-18T03:26:18.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 0x7fb598142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 0x7fb5980620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980af4b0 con 0x7fb598146ac0 2020-07-18T03:26:18.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5980b6d80 con 0x7fb598142a20 2020-07-18T03:26:18.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.822+0000 7fb59ef86700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5980af710 con 0x7fb598061c60 2020-07-18T03:26:18.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59cd22700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 0x7fb598142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 0x7fb5980620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 0x7fb5980620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:51946/0 (socket says 172.21.15.47:51946) 2020-07-18T03:26:18.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59d523700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fb598146ac0 0x7fb598146f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59204/0 (socket says 172.21.15.47:59204) 2020-07-18T03:26:18.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 -- 172.21.15.47:0/2202664428 learned_addr learned my addr 172.21.15.47:0/2202664428 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:18.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59cd22700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 0x7fb598142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:45026/0 (socket says 172.21.15.47:45026) 2020-07-18T03:26:18.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 -- 172.21.15.47:0/2202664428 >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 msgr2=0x7fb598142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:18.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 --2- 172.21.15.47:0/2202664428 >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 0x7fb598142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 -- 172.21.15.47:0/2202664428 >> v1:172.21.15.18:6789/0 conn(0x7fb598146ac0 legacy=0x7fb598146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 -- 172.21.15.47:0/2202664428 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980af9b0 con 0x7fb598061c60 2020-07-18T03:26:18.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59cd22700 1 --2- 172.21.15.47:0/2202664428 >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 0x7fb598142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:18.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb597fff700 1 --2- 172.21.15.47:0/2202664428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 0x7fb5980620c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb58c002f30 tx=0x7fb58c006f20).ready entity=mon.2 client_cookie=78580f94d96f83c2 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb5977fe700 1 -- 172.21.15.47:0/2202664428 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb58c00cc40 con 0x7fb598061c60 2020-07-18T03:26:18.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb5977fe700 1 -- 172.21.15.47:0/2202664428 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c00cda0 con 0x7fb598061c60 2020-07-18T03:26:18.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb5977fe700 1 -- 172.21.15.47:0/2202664428 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb58c010600 con 0x7fb598061c60 2020-07-18T03:26:18.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59ef86700 1 -- 172.21.15.47:0/2202664428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 msgr2=0x7fb5980620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2202664428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 0x7fb5980620c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb58c002f30 tx=0x7fb58c006f20).stop 2020-07-18T03:26:18.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59ef86700 1 -- 172.21.15.47:0/2202664428 shutdown_connections 2020-07-18T03:26:18.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2202664428 >> v2:172.21.15.18:3300/0 conn(0x7fb598142a20 0x7fb598142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.823+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2202664428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb598061c60 0x7fb5980620c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.824+0000 7fb59ef86700 1 -- 172.21.15.47:0/2202664428 >> 172.21.15.47:0/2202664428 conn(0x7fb59805f5a0 msgr2=0x7fb59805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.824+0000 7fb59ef86700 1 -- 172.21.15.47:0/2202664428 shutdown_connections 2020-07-18T03:26:18.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.824+0000 7fb59ef86700 1 -- 172.21.15.47:0/2202664428 wait complete. 2020-07-18T03:26:18.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.825+0000 7fb59ef86700 1 Processor -- start 2020-07-18T03:26:18.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.825+0000 7fb59ef86700 1 -- start start 2020-07-18T03:26:18.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59ef86700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb5981421e0 0x7fb59814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59ef86700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59814ad40 con 0x7fb598142a20 2020-07-18T03:26:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59ef86700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59814afe0 con 0x7fb598146ac0 2020-07-18T03:26:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59ef86700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59814b280 con 0x7fb5981421e0 2020-07-18T03:26:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb597fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb598146ac0 0x7fb598158b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40442/0 (socket says 172.21.15.47:40442) 2020-07-18T03:26:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb597fff700 1 -- 172.21.15.47:0/2733417348 learned_addr learned my addr 172.21.15.47:0/2733417348 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:18.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59d523700 1 --2- 172.21.15.47:0/2733417348 >> v2:172.21.15.18:3300/0 conn(0x7fb5981421e0 0x7fb59814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875137405 0 0) 0x7fb59814afe0 con 0x7fb598146ac0 2020-07-18T03:26:18.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb570003240 con 0x7fb598146ac0 2020-07-18T03:26:18.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59d523700 1 -- 172.21.15.47:0/2733417348 >> v1:172.21.15.47:6789/0 conn(0x7fb598146ac0 legacy=0x7fb598158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59d523700 1 -- 172.21.15.47:0/2733417348 >> v1:172.21.15.18:6789/0 conn(0x7fb598142a20 legacy=0x7fb598141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863151191 0 0) 0x7fb59814ad40 con 0x7fb598142a20 2020-07-18T03:26:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.826+0000 7fb59d523700 1 -- 172.21.15.47:0/2733417348 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598145bd0 con 0x7fb5981421e0 2020-07-18T03:26:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.827+0000 7fb59d523700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981421e0 0x7fb59814b720 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb5980afd30 tx=0x7fb590007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.827+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5900058b0 con 0x7fb5981421e0 2020-07-18T03:26:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.827+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590005a10 con 0x7fb5981421e0 2020-07-18T03:26:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.827+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb598145e80 con 0x7fb5981421e0 2020-07-18T03:26:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.827+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb590005d70 con 0x7fb5981421e0 2020-07-18T03:26:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.827+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb598159450 con 0x7fb5981421e0 2020-07-18T03:26:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.828+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb598159670 con 0x7fb5981421e0 2020-07-18T03:26:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.828+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb590010030 con 0x7fb5981421e0 2020-07-18T03:26:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.828+0000 7fb594ff9700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb5700502f0 0x7fb5700527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.828+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb590067aa0 con 0x7fb5981421e0 2020-07-18T03:26:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.828+0000 7fb59cd22700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb5700502f0 0x7fb5700527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:18.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.829+0000 7fb59cd22700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb5700502f0 0x7fb5700527b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb588002f30 tx=0x7fb588006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:18.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.832+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb5900401d0 con 0x7fb5981421e0 2020-07-18T03:26:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.988+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb59814b3f0 con 0x7fb5981421e0 2020-07-18T03:26:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.989+0000 7fb594ff9700 1 -- 172.21.15.47:0/2733417348 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb5900437f0 con 0x7fb5981421e0 2020-07-18T03:26:18.990 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb5700502f0 msgr2=0x7fb5700527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb5700502f0 0x7fb5700527b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb588002f30 tx=0x7fb588006dd0).stop 2020-07-18T03:26:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981421e0 msgr2=0x7fb59814b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981421e0 0x7fb59814b720 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb5980afd30 tx=0x7fb590007dd0).stop 2020-07-18T03:26:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 shutdown_connections 2020-07-18T03:26:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981421e0 0x7fb59814b720 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 --2- 172.21.15.47:0/2733417348 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb5700502f0 0x7fb5700527b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 >> 172.21.15.47:0/2733417348 conn(0x7fb59805f5a0 msgr2=0x7fb5980b92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 shutdown_connections 2020-07-18T03:26:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:18.990+0000 7fb59ef86700 1 -- 172.21.15.47:0/2733417348 wait complete. 2020-07-18T03:26:19.001 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:19.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.118+0000 7f45d4ce0700 1 Processor -- start 2020-07-18T03:26:19.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.118+0000 7f45d4ce0700 1 -- start start 2020-07-18T03:26:19.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.119+0000 7f45d4ce0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00b5080 0x7f45d00ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.119+0000 7f45d4ce0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d0144a60 con 0x7f45d00ae6c0 2020-07-18T03:26:19.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.119+0000 7f45d4ce0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00b00a0 con 0x7f45d00afc00 2020-07-18T03:26:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.119+0000 7f45d4ce0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00b6720 con 0x7f45d00b5080 2020-07-18T03:26:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.119+0000 7f45ce59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45d00afc00 0x7f45d00b5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40448/0 (socket says 172.21.15.47:40448) 2020-07-18T03:26:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.119+0000 7f45ce59c700 1 -- 172.21.15.47:0/3807857513 learned_addr learned my addr 172.21.15.47:0/3807857513 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45c7fff700 1 --2- 172.21.15.47:0/3807857513 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00b5080 0x7f45d00ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778399073 0 0) 0x7f45d00b00a0 con 0x7f45d00afc00 2020-07-18T03:26:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b0003240 con 0x7f45d00afc00 2020-07-18T03:26:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668140511 0 0) 0x7f45c8001450 con 0x7f45d00ae6c0 2020-07-18T03:26:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d00b00a0 con 0x7f45d00ae6c0 2020-07-18T03:26:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2458614137 0 0) 0x7f45b0003240 con 0x7f45d00afc00 2020-07-18T03:26:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45c7fff700 1 -- 172.21.15.47:0/3807857513 >> v1:172.21.15.47:6789/0 conn(0x7f45d00afc00 legacy=0x7f45d00b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45c7fff700 1 -- 172.21.15.47:0/3807857513 >> v1:172.21.15.18:6789/0 conn(0x7f45d00ae6c0 legacy=0x7f45d00620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45c7fff700 1 -- 172.21.15.47:0/3807857513 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0061c20 con 0x7f45d00b5080 2020-07-18T03:26:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45c7fff700 1 --2- 172.21.15.47:0/3807857513 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00b5080 0x7f45d00ae1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f45c0001110 tx=0x7f45b8006dd0).ready entity=mon.2 client_cookie=7070861115232f7e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c0001710 con 0x7f45d00b5080 2020-07-18T03:26:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b800d5c0 con 0x7f45d00b5080 2020-07-18T03:26:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45cdd9b700 1 -- 172.21.15.47:0/3807857513 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b800d730 con 0x7f45d00b5080 2020-07-18T03:26:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3807857513 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00b5080 msgr2=0x7f45d00ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3807857513 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00b5080 0x7f45d00ae1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f45c0001110 tx=0x7f45b8006dd0).stop 2020-07-18T03:26:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3807857513 shutdown_connections 2020-07-18T03:26:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3807857513 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00b5080 0x7f45d00ae1d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3807857513 >> 172.21.15.47:0/3807857513 conn(0x7f45d005f5a0 msgr2=0x7f45d005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3807857513 shutdown_connections 2020-07-18T03:26:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.120+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3807857513 wait complete. 2020-07-18T03:26:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 Processor -- start 2020-07-18T03:26:19.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 -- start start 2020-07-18T03:26:19.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00ae6c0 0x7f45d00b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f45d00afc00 0x7f45d0148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00b7da0 con 0x7f45d01396a0 2020-07-18T03:26:19.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00b7860 con 0x7f45d00afc00 2020-07-18T03:26:19.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.122+0000 7f45d4ce0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00b7b00 con 0x7f45d00ae6c0 2020-07-18T03:26:19.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45ced9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45d01396a0 0x7f45d0139b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40456/0 (socket says 172.21.15.47:40456) 2020-07-18T03:26:19.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45ced9d700 1 -- 172.21.15.47:0/3637634728 learned_addr learned my addr 172.21.15.47:0/3637634728 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:19.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c7fff700 1 --2- 172.21.15.47:0/3637634728 >> v2:172.21.15.18:3300/0 conn(0x7f45d00afc00 0x7f45d0148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45ce59c700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00ae6c0 0x7f45d00b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884933225 0 0) 0x7f45d00b7da0 con 0x7f45d01396a0 2020-07-18T03:26:19.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac003240 con 0x7f45d01396a0 2020-07-18T03:26:19.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1100586216 0 0) 0x7f45ac003240 con 0x7f45d01396a0 2020-07-18T03:26:19.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c7fff700 1 -- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00ae6c0 msgr2=0x7f45d00b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c7fff700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00ae6c0 0x7f45d00b9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c7fff700 1 -- 172.21.15.47:0/3637634728 >> v1:172.21.15.47:6789/0 conn(0x7f45d01396a0 legacy=0x7f45d0139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c7fff700 1 -- 172.21.15.47:0/3637634728 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00b9ff0 con 0x7f45d00afc00 2020-07-18T03:26:19.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45ce59c700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00ae6c0 0x7f45d00b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:19.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c7fff700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45d00afc00 0x7f45d0148df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f45c8001f30 tx=0x7f45b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45c8001d90 con 0x7f45d00afc00 2020-07-18T03:26:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b8011de0 con 0x7f45d00afc00 2020-07-18T03:26:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.123+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b801d3c0 con 0x7f45d00afc00 2020-07-18T03:26:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.124+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d01492e0 con 0x7f45d00afc00 2020-07-18T03:26:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.124+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d00b75f0 con 0x7f45d00afc00 2020-07-18T03:26:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.124+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d0147890 con 0x7f45d00afc00 2020-07-18T03:26:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.125+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f45b801a040 con 0x7f45d00afc00 2020-07-18T03:26:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.125+0000 7f45c67fc700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45ac050360 0x7f45ac052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.125+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f45b8068b70 con 0x7f45d00afc00 2020-07-18T03:26:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.125+0000 7f45ce59c700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45ac050360 0x7f45ac052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.125+0000 7f45ce59c700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45ac050360 0x7f45ac052820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f45c000d000 tx=0x7f45c0000b80).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.129+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f45b80411e0 con 0x7f45d00afc00 2020-07-18T03:26:19.225 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:26:19.232 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:26:19.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.289+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45d005b120 con 0x7f45d00afc00 2020-07-18T03:26:19.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.289+0000 7f45c67fc700 1 -- 172.21.15.47:0/3637634728 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f45b80240a0 con 0x7f45d00afc00 2020-07-18T03:26:19.291 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:19.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45ac050360 msgr2=0x7f45ac052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45ac050360 0x7f45ac052820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f45c000d000 tx=0x7f45c0000b80).stop 2020-07-18T03:26:19.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45d00afc00 msgr2=0x7f45d0148df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45d00afc00 0x7f45d0148df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f45c8001f30 tx=0x7f45b8015000).stop 2020-07-18T03:26:19.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 shutdown_connections 2020-07-18T03:26:19.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45ac050360 0x7f45ac052820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45d00ae6c0 0x7f45d00b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 --2- 172.21.15.47:0/3637634728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45d00afc00 0x7f45d0148df0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 >> 172.21.15.47:0/3637634728 conn(0x7f45d005f5a0 msgr2=0x7f45d00b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:19.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 shutdown_connections 2020-07-18T03:26:19.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.291+0000 7f45d4ce0700 1 -- 172.21.15.47:0/3637634728 wait complete. 2020-07-18T03:26:19.311 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:19.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.457+0000 7f0a8b8f2700 1 Processor -- start 2020-07-18T03:26:19.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.458+0000 7f0a8b8f2700 1 -- start start 2020-07-18T03:26:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.458+0000 7f0a8b8f2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0a840ad100 0x7f0a840b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.458+0000 7f0a8b8f2700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a84147330 con 0x7f0a84147a00 2020-07-18T03:26:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.458+0000 7f0a8b8f2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840603c0 con 0x7f0a841443f0 2020-07-18T03:26:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.458+0000 7f0a8b8f2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840b57e0 con 0x7f0a840ad100 2020-07-18T03:26:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a89e8f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a841443f0 0x7f0a8405fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40466/0 (socket says 172.21.15.47:40466) 2020-07-18T03:26:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a89e8f700 1 -- 172.21.15.47:0/3758039820 learned_addr learned my addr 172.21.15.47:0/3758039820 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922248075 0 0) 0x7f0a840603c0 con 0x7f0a841443f0 2020-07-18T03:26:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c003240 con 0x7f0a841443f0 2020-07-18T03:26:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a8968e700 1 --2- 172.21.15.47:0/3758039820 >> v2:172.21.15.18:3300/0 conn(0x7f0a840ad100 0x7f0a840b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3149345435 0 0) 0x7f0a6c003240 con 0x7f0a841443f0 2020-07-18T03:26:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 >> v2:172.21.15.18:3300/0 conn(0x7f0a840ad100 msgr2=0x7f0a840b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 --2- 172.21.15.47:0/3758039820 >> v2:172.21.15.18:3300/0 conn(0x7f0a840ad100 0x7f0a840b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 >> v1:172.21.15.18:6789/0 conn(0x7f0a84147a00 legacy=0x7f0a8405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840b5a80 con 0x7f0a841443f0 2020-07-18T03:26:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f0a7c001a50 con 0x7f0a841443f0 2020-07-18T03:26:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 >> v1:172.21.15.47:6789/0 conn(0x7f0a841443f0 legacy=0x7f0a8405fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a6c003730 0x7f0a6c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0044e0 0x7f0a6c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a6c004e50 0x7f0a6c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840b57e0 con 0x7f0a6c004e50 2020-07-18T03:26:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c004350 con 0x7f0a6c0044e0 2020-07-18T03:26:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c00de60 con 0x7f0a6c003730 2020-07-18T03:26:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a88e8d700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0044e0 0x7f0a6c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a89e8f700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a6c004e50 0x7f0a6c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.459+0000 7f0a8968e700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a6c003730 0x7f0a6c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a88e8d700 1 -- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a6c003730 msgr2=0x7f0a6c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a88e8d700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a6c003730 0x7f0a6c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a88e8d700 1 -- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a6c004e50 msgr2=0x7f0a6c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a88e8d700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a6c004e50 0x7f0a6c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a88e8d700 1 -- 172.21.15.47:0/3758039820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c003240 con 0x7f0a6c0044e0 2020-07-18T03:26:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a89e8f700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a6c004e50 0x7f0a6c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a89e8f700 1 -- 172.21.15.47:0/3758039820 reap_dead start 2020-07-18T03:26:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a88e8d700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0044e0 0x7f0a6c004960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f0a8405ca30 tx=0x7f0a80007b70).ready entity=mon.1 client_cookie=af810d60c5294241 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8000c660 con 0x7f0a6c0044e0 2020-07-18T03:26:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8000c7c0 con 0x7f0a6c0044e0 2020-07-18T03:26:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a77fff700 1 -- 172.21.15.47:0/3758039820 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8000cb20 con 0x7f0a6c0044e0 2020-07-18T03:26:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/3758039820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0044e0 msgr2=0x7f0a6c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0044e0 0x7f0a6c004960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f0a8405ca30 tx=0x7f0a80007b70).stop 2020-07-18T03:26:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/3758039820 shutdown_connections 2020-07-18T03:26:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/3758039820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0044e0 0x7f0a6c004960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/3758039820 >> 172.21.15.47:0/3758039820 conn(0x7f0a840b9340 msgr2=0x7f0a840b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/3758039820 shutdown_connections 2020-07-18T03:26:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.460+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/3758039820 wait complete. 2020-07-18T03:26:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.461+0000 7f0a8b8f2700 1 Processor -- start 2020-07-18T03:26:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8b8f2700 1 -- start start 2020-07-18T03:26:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8b8f2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a84134a50 0x7f0a84155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8b8f2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a84140be0 0x7f0a84065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8b8f2700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a84156130 con 0x7f0a840663d0 2020-07-18T03:26:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8b8f2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840b33a0 con 0x7f0a84140be0 2020-07-18T03:26:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8b8f2700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84155750 con 0x7f0a84134a50 2020-07-18T03:26:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a88e8d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a84140be0 0x7f0a84065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a88e8d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a84140be0 0x7f0a84065ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59388/0 (socket says 172.21.15.47:59388) 2020-07-18T03:26:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a88e8d700 1 -- 172.21.15.47:0/1555181434 learned_addr learned my addr 172.21.15.47:0/1555181434 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.462+0000 7f0a8968e700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a84134a50 0x7f0a84155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a88e8d700 1 -- 172.21.15.47:0/1555181434 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a84134a50 msgr2=0x7f0a84155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a88e8d700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a84134a50 0x7f0a84155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a88e8d700 1 -- 172.21.15.47:0/1555181434 >> v1:172.21.15.18:6789/0 conn(0x7f0a840663d0 legacy=0x7f0a84158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a88e8d700 1 -- 172.21.15.47:0/1555181434 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a841559f0 con 0x7f0a84140be0 2020-07-18T03:26:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a88e8d700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a84140be0 0x7f0a84065ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f0a8405ca30 tx=0x7f0a80016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a80007240 con 0x7f0a84140be0 2020-07-18T03:26:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800073a0 con 0x7f0a84140be0 2020-07-18T03:26:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a84155c90 con 0x7f0a84140be0 2020-07-18T03:26:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a80012bd0 con 0x7f0a84140be0 2020-07-18T03:26:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.463+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841134c0 con 0x7f0a84140be0 2020-07-18T03:26:19.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.464+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8405d8f0 con 0x7f0a84140be0 2020-07-18T03:26:19.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.464+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0a80025030 con 0x7f0a84140be0 2020-07-18T03:26:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.464+0000 7f0a757fa700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a5c04ff80 0x7f0a5c052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.465+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0a80068e40 con 0x7f0a84140be0 2020-07-18T03:26:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.465+0000 7f0a8968e700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a5c04ff80 0x7f0a5c052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.465+0000 7f0a8968e700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a5c04ff80 0x7f0a5c052440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0a78008000 tx=0x7f0a78008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.468+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0a800415d0 con 0x7f0a84140be0 2020-07-18T03:26:19.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.627+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a84155e20 con 0x7f0a84140be0 2020-07-18T03:26:19.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.627+0000 7f0a757fa700 1 -- 172.21.15.47:0/1555181434 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f0a80023070 con 0x7f0a84140be0 2020-07-18T03:26:19.629 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:19.629 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a5c04ff80 msgr2=0x7f0a5c052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a5c04ff80 0x7f0a5c052440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0a78008000 tx=0x7f0a78008040).stop 2020-07-18T03:26:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a84140be0 msgr2=0x7f0a84065ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a84140be0 0x7f0a84065ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f0a8405ca30 tx=0x7f0a80016000).stop 2020-07-18T03:26:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 shutdown_connections 2020-07-18T03:26:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a5c04ff80 0x7f0a5c052440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0a84134a50 0x7f0a84155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 --2- 172.21.15.47:0/1555181434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a84140be0 0x7f0a84065ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 >> 172.21.15.47:0/1555181434 conn(0x7f0a840b9340 msgr2=0x7f0a840b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:19.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 shutdown_connections 2020-07-18T03:26:19.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.629+0000 7f0a8b8f2700 1 -- 172.21.15.47:0/1555181434 wait complete. 2020-07-18T03:26:19.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:19.645 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.761+0000 7fb57d48d700 1 Processor -- start 2020-07-18T03:26:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.761+0000 7fb57d48d700 1 -- start start 2020-07-18T03:26:19.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb57d48d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb5780ad330 0x7fb578143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb57d48d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb5780b59b0 0x7fb5780b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb57d48d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb578147de0 con 0x7fb5780b6300 2020-07-18T03:26:19.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb57d48d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5780ae000 con 0x7fb5780ad330 2020-07-18T03:26:19.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb57d48d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb578148010 con 0x7fb5780b59b0 2020-07-18T03:26:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb576ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb5780ad330 0x7fb578143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb5767fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb5780b59b0 0x7fb5780b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb5777fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fb5780b6300 0x7fb5780b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59252/0 (socket says 172.21.15.47:59252) 2020-07-18T03:26:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.762+0000 7fb5777fe700 1 -- 172.21.15.47:0/3206414364 learned_addr learned my addr 172.21.15.47:0/3206414364 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb5767fc700 1 -- 172.21.15.47:0/3206414364 >> v2:172.21.15.18:3300/0 conn(0x7fb5780ad330 msgr2=0x7fb578143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb5767fc700 1 --2- 172.21.15.47:0/3206414364 >> v2:172.21.15.18:3300/0 conn(0x7fb5780ad330 0x7fb578143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb5767fc700 1 -- 172.21.15.47:0/3206414364 >> v1:172.21.15.18:6789/0 conn(0x7fb5780b6300 legacy=0x7fb5780b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb575ffb700 1 -- 172.21.15.47:0/3206414364 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818395644 0 0) 0x7fb578147de0 con 0x7fb5780b6300 2020-07-18T03:26:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb5767fc700 1 -- 172.21.15.47:0/3206414364 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5780ad710 con 0x7fb5780b59b0 2020-07-18T03:26:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb576ffd700 1 --2- 172.21.15.47:0/3206414364 >> v2:172.21.15.18:3300/0 conn(0x7fb5780ad330 0x7fb578143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb5767fc700 1 --2- 172.21.15.47:0/3206414364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb5780b59b0 0x7fb5780b5e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb568002f30 tx=0x7fb568006dd0).ready entity=mon.2 client_cookie=5a3e86b51502f27c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb575ffb700 1 -- 172.21.15.47:0/3206414364 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb568009760 con 0x7fb5780b59b0 2020-07-18T03:26:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb575ffb700 1 -- 172.21.15.47:0/3206414364 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5680098c0 con 0x7fb5780b59b0 2020-07-18T03:26:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb575ffb700 1 -- 172.21.15.47:0/3206414364 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb568009c20 con 0x7fb5780b59b0 2020-07-18T03:26:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 -- 172.21.15.47:0/3206414364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb5780b59b0 msgr2=0x7fb5780b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 --2- 172.21.15.47:0/3206414364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb5780b59b0 0x7fb5780b5e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb568002f30 tx=0x7fb568006dd0).stop 2020-07-18T03:26:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 -- 172.21.15.47:0/3206414364 shutdown_connections 2020-07-18T03:26:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 --2- 172.21.15.47:0/3206414364 >> v2:172.21.15.18:3300/0 conn(0x7fb5780ad330 0x7fb578143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 --2- 172.21.15.47:0/3206414364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb5780b59b0 0x7fb5780b5e10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 -- 172.21.15.47:0/3206414364 >> 172.21.15.47:0/3206414364 conn(0x7fb57805f5a0 msgr2=0x7fb57805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 -- 172.21.15.47:0/3206414364 shutdown_connections 2020-07-18T03:26:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.763+0000 7fb57d48d700 1 -- 172.21.15.47:0/3206414364 wait complete. 2020-07-18T03:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 Processor -- start 2020-07-18T03:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 -- start start 2020-07-18T03:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5780ad330 0x7fb578063060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb5780b6300 0x7fb5780b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb57814e660 0x7fb57814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5781393a0 con 0x7fb5780b6300 2020-07-18T03:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb578139580 con 0x7fb5780ad330 2020-07-18T03:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.765+0000 7fb57d48d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb578139820 con 0x7fb57814e660 2020-07-18T03:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5780ad330 0x7fb578063060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5780ad330 0x7fb578063060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59400/0 (socket says 172.21.15.47:59400) 2020-07-18T03:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 -- 172.21.15.47:0/621361533 learned_addr learned my addr 172.21.15.47:0/621361533 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb5767fc700 1 --2- 172.21.15.47:0/621361533 >> v2:172.21.15.18:3300/0 conn(0x7fb5780b6300 0x7fb5780b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb5777fe700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb57814e660 0x7fb57814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 -- 172.21.15.47:0/621361533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb57814e660 msgr2=0x7fb57814eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb57814e660 0x7fb57814eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 -- 172.21.15.47:0/621361533 >> v2:172.21.15.18:3300/0 conn(0x7fb5780b6300 msgr2=0x7fb5780b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 --2- 172.21.15.47:0/621361533 >> v2:172.21.15.18:3300/0 conn(0x7fb5780b6300 0x7fb5780b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 -- 172.21.15.47:0/621361533 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578139c80 con 0x7fb5780ad330 2020-07-18T03:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb576ffd700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5780ad330 0x7fb578063060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fb56c00c000 tx=0x7fb56c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb56c00c950 con 0x7fb5780ad330 2020-07-18T03:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c00cab0 con 0x7fb5780ad330 2020-07-18T03:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5780b2530 con 0x7fb5780ad330 2020-07-18T03:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb56c00ce10 con 0x7fb5780ad330 2020-07-18T03:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.766+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb578065d00 con 0x7fb5780ad330 2020-07-18T03:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.767+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5780b1eb0 con 0x7fb5780ad330 2020-07-18T03:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.768+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb56c01b400 con 0x7fb5780ad330 2020-07-18T03:26:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.768+0000 7fb55effd700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb55404ffb0 0x7fb554052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.768+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb56c067d60 con 0x7fb5780ad330 2020-07-18T03:26:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.768+0000 7fb5767fc700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb55404ffb0 0x7fb554052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.768+0000 7fb5767fc700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb55404ffb0 0x7fb554052470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fb56801b000 tx=0x7fb568015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.771+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb56c040490 con 0x7fb5780ad330 2020-07-18T03:26:19.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.930+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb578139990 con 0x7fb5780ad330 2020-07-18T03:26:19.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.931+0000 7fb55effd700 1 -- 172.21.15.47:0/621361533 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb578139990 con 0x7fb5780ad330 2020-07-18T03:26:19.932 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:19.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb55404ffb0 msgr2=0x7fb554052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb55404ffb0 0x7fb554052470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fb56801b000 tx=0x7fb568015000).stop 2020-07-18T03:26:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5780ad330 msgr2=0x7fb578063060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5780ad330 0x7fb578063060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fb56c00c000 tx=0x7fb56c008040).stop 2020-07-18T03:26:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 shutdown_connections 2020-07-18T03:26:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb55404ffb0 0x7fb554052470 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb57814e660 0x7fb57814eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 --2- 172.21.15.47:0/621361533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5780ad330 0x7fb578063060 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 --2- 172.21.15.47:0/621361533 >> v2:172.21.15.18:3300/0 conn(0x7fb5780b6300 0x7fb5780b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 >> 172.21.15.47:0/621361533 conn(0x7fb57805f5a0 msgr2=0x7fb578062090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 shutdown_connections 2020-07-18T03:26:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:19.932+0000 7fb57d48d700 1 -- 172.21.15.47:0/621361533 wait complete. 2020-07-18T03:26:19.943 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.060+0000 7f34edc44700 1 Processor -- start 2020-07-18T03:26:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.060+0000 7f34edc44700 1 -- start start 2020-07-18T03:26:20.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34edc44700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34edc44700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34e80b3a10 0x7f34e80b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34edc44700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e8147de0 con 0x7f34e80b59b0 2020-07-18T03:26:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34edc44700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e80ae000 con 0x7f34e80ad330 2020-07-18T03:26:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34edc44700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e8148010 con 0x7f34e80b3a10 2020-07-18T03:26:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34e80b3a10 0x7f34e80b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34e80b3a10 0x7f34e80b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59412/0 (socket says 172.21.15.47:59412) 2020-07-18T03:26:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 -- 172.21.15.47:0/773032347 learned_addr learned my addr 172.21.15.47:0/773032347 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 -- 172.21.15.47:0/773032347 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 msgr2=0x7f34e8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e77fe700 1 --2- 172.21.15.47:0/773032347 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 --2- 172.21.15.47:0/773032347 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 -- 172.21.15.47:0/773032347 >> v1:172.21.15.18:6789/0 conn(0x7f34e80b59b0 legacy=0x7f34e80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 -- 172.21.15.47:0/773032347 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e80ad710 con 0x7f34e80b3a10 2020-07-18T03:26:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.061+0000 7f34e7fff700 1 --2- 172.21.15.47:0/773032347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e80b3a10 0x7f34e80b3e80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f34e80ae4e0 tx=0x7f34e0009270).ready entity=mon.1 client_cookie=85ae8cb3f6771cb3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34e67fc700 1 -- 172.21.15.47:0/773032347 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e000bbc0 con 0x7f34e80b3a10 2020-07-18T03:26:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34e67fc700 1 -- 172.21.15.47:0/773032347 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e000bd20 con 0x7f34e80b3a10 2020-07-18T03:26:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34e67fc700 1 -- 172.21.15.47:0/773032347 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e000be90 con 0x7f34e80b3a10 2020-07-18T03:26:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 -- 172.21.15.47:0/773032347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e80b3a10 msgr2=0x7f34e80b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 --2- 172.21.15.47:0/773032347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e80b3a10 0x7f34e80b3e80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f34e80ae4e0 tx=0x7f34e0009270).stop 2020-07-18T03:26:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 -- 172.21.15.47:0/773032347 shutdown_connections 2020-07-18T03:26:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 --2- 172.21.15.47:0/773032347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e80b3a10 0x7f34e80b3e80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 --2- 172.21.15.47:0/773032347 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 -- 172.21.15.47:0/773032347 >> 172.21.15.47:0/773032347 conn(0x7f34e805f5a0 msgr2=0x7f34e805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 -- 172.21.15.47:0/773032347 shutdown_connections 2020-07-18T03:26:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.062+0000 7f34edc44700 1 -- 172.21.15.47:0/773032347 wait complete. 2020-07-18T03:26:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 Processor -- start 2020-07-18T03:26:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 -- start start 2020-07-18T03:26:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8066110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34e80b59b0 0x7f34e8063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34e8144b90 0x7f34e8145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e8065730 con 0x7f34e80ad330 2020-07-18T03:26:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e80659d0 con 0x7f34e8144b90 2020-07-18T03:26:20.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34edc44700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8065c70 con 0x7f34e80b59b0 2020-07-18T03:26:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34e7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34e8144b90 0x7f34e8145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34e7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f34e8144b90 0x7f34e8145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59418/0 (socket says 172.21.15.47:59418) 2020-07-18T03:26:20.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.064+0000 7f34e7fff700 1 -- 172.21.15.47:0/2241206756 learned_addr learned my addr 172.21.15.47:0/2241206756 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e77fe700 1 --2- 172.21.15.47:0/2241206756 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8066110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e7fff700 1 -- 172.21.15.47:0/2241206756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34e80b59b0 msgr2=0x7f34e8063ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e6ffd700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34e80b59b0 0x7f34e8063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e7fff700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34e80b59b0 0x7f34e8063ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e7fff700 1 -- 172.21.15.47:0/2241206756 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 msgr2=0x7f34e8066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e7fff700 1 --2- 172.21.15.47:0/2241206756 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8066110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e7fff700 1 -- 172.21.15.47:0/2241206756 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e80653b0 con 0x7f34e8144b90 2020-07-18T03:26:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e77fe700 1 --2- 172.21.15.47:0/2241206756 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8066110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34e7fff700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e8144b90 0x7f34e8145c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f34e80ae4e0 tx=0x7f34e0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e00108f0 con 0x7f34e8144b90 2020-07-18T03:26:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e0010a50 con 0x7f34e8144b90 2020-07-18T03:26:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e8063000 con 0x7f34e8144b90 2020-07-18T03:26:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e0010db0 con 0x7f34e8144b90 2020-07-18T03:26:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.065+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e8146110 con 0x7f34e8144b90 2020-07-18T03:26:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.066+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e805b120 con 0x7f34e8144b90 2020-07-18T03:26:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.066+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f34e0027020 con 0x7f34e8144b90 2020-07-18T03:26:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.066+0000 7f34c77fe700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f34c004ff70 0x7f34c0052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.066+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f34e00694b0 con 0x7f34e8144b90 2020-07-18T03:26:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.067+0000 7f34e77fe700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f34c004ff70 0x7f34c0052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.067+0000 7f34e77fe700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f34c004ff70 0x7f34c0052430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f34d8008000 tx=0x7f34d8008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.070+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f34e0041c40 con 0x7f34e8144b90 2020-07-18T03:26:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.226+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34e80690d0 con 0x7f34e8144b90 2020-07-18T03:26:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.226+0000 7f34c77fe700 1 -- 172.21.15.47:0/2241206756 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34e0015020 con 0x7f34e8144b90 2020-07-18T03:26:20.227 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.227+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f34c004ff70 msgr2=0x7f34c0052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.227+0000 7f34edc44700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f34c004ff70 0x7f34c0052430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f34d8008000 tx=0x7f34d8008040).stop 2020-07-18T03:26:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.227+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e8144b90 msgr2=0x7f34e8145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.227+0000 7f34edc44700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e8144b90 0x7f34e8145c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f34e80ae4e0 tx=0x7f34e0016000).stop 2020-07-18T03:26:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 shutdown_connections 2020-07-18T03:26:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f34c004ff70 0x7f34c0052430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34e8144b90 0x7f34e8145c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 --2- 172.21.15.47:0/2241206756 >> v2:172.21.15.18:3300/0 conn(0x7f34e80ad330 0x7f34e8066110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 --2- 172.21.15.47:0/2241206756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34e80b59b0 0x7f34e8063ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 >> 172.21.15.47:0/2241206756 conn(0x7f34e805f5a0 msgr2=0x7f34e80b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 shutdown_connections 2020-07-18T03:26:20.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.228+0000 7f34edc44700 1 -- 172.21.15.47:0/2241206756 wait complete. 2020-07-18T03:26:20.239 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.356+0000 7f5d6cb1b700 1 Processor -- start 2020-07-18T03:26:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.356+0000 7f5d6cb1b700 1 -- start start 2020-07-18T03:26:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.356+0000 7f5d6cb1b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5d680b3a10 0x7f5d680b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.356+0000 7f5d6cb1b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d68147de0 con 0x7f5d680ad330 2020-07-18T03:26:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.356+0000 7f5d6cb1b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d680ae000 con 0x7f5d680b59b0 2020-07-18T03:26:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.356+0000 7f5d6cb1b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d68148010 con 0x7f5d680b3a10 2020-07-18T03:26:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d65d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5d680b59b0 0x7f5d680b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40514/0 (socket says 172.21.15.47:40514) 2020-07-18T03:26:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d65d9b700 1 -- 172.21.15.47:0/2828943545 learned_addr learned my addr 172.21.15.47:0/2828943545 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876404068 0 0) 0x7f5d680ae000 con 0x7f5d680b59b0 2020-07-18T03:26:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d66d9d700 1 --2- 172.21.15.47:0/2828943545 >> v2:172.21.15.18:3300/0 conn(0x7f5d680b3a10 0x7f5d680b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d48003240 con 0x7f5d680b59b0 2020-07-18T03:26:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2212414572 0 0) 0x7f5d48003240 con 0x7f5d680b59b0 2020-07-18T03:26:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 >> v2:172.21.15.18:3300/0 conn(0x7f5d680b3a10 msgr2=0x7f5d680b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 --2- 172.21.15.47:0/2828943545 >> v2:172.21.15.18:3300/0 conn(0x7f5d680b3a10 0x7f5d680b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 >> v1:172.21.15.18:6789/0 conn(0x7f5d680ad330 legacy=0x7f5d68143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680ad710 con 0x7f5d680b59b0 2020-07-18T03:26:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5d580016f0 con 0x7f5d680b59b0 2020-07-18T03:26:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 >> v1:172.21.15.47:6789/0 conn(0x7f5d680b59b0 legacy=0x7f5d680b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d48003780 0x7f5d48003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d480044e0 0x7f5d48004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d48004e50 0x7f5d4800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68148010 con 0x7f5d48004e50 2020-07-18T03:26:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4800bc20 con 0x7f5d48003780 2020-07-18T03:26:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4800bd60 con 0x7f5d480044e0 2020-07-18T03:26:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.357+0000 7f5d6659c700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d48003780 0x7f5d48003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d65d9b700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d480044e0 0x7f5d48004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d66d9d700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d48004e50 0x7f5d4800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6659c700 1 -- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d480044e0 msgr2=0x7f5d48004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6659c700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d480044e0 0x7f5d48004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6659c700 1 -- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d48004e50 msgr2=0x7f5d4800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6659c700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d48004e50 0x7f5d4800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6659c700 1 -- 172.21.15.47:0/2828943545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48003240 con 0x7f5d48003780 2020-07-18T03:26:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d66d9d700 1 -- 172.21.15.47:0/2828943545 reap_dead start 2020-07-18T03:26:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6659c700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d48003780 0x7f5d48003ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5d680ae4e0 tx=0x7f5d50007b70).ready entity=mon.1 client_cookie=a4a37dd2c0a7cc6a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5000c810 con 0x7f5d48003780 2020-07-18T03:26:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5000c970 con 0x7f5d48003780 2020-07-18T03:26:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6559a700 1 -- 172.21.15.47:0/2828943545 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5000ccd0 con 0x7f5d48003780 2020-07-18T03:26:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/2828943545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d48003780 msgr2=0x7f5d48003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d48003780 0x7f5d48003ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5d680ae4e0 tx=0x7f5d50007b70).stop 2020-07-18T03:26:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/2828943545 shutdown_connections 2020-07-18T03:26:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/2828943545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d48003780 0x7f5d48003ba0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/2828943545 >> 172.21.15.47:0/2828943545 conn(0x7f5d6805f5a0 msgr2=0x7f5d6805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/2828943545 shutdown_connections 2020-07-18T03:26:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.358+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/2828943545 wait complete. 2020-07-18T03:26:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 Processor -- start 2020-07-18T03:26:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 -- start start 2020-07-18T03:26:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5d68142650 0x7f5d6814c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d68146aa0 0x7f5d68143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d68064440 0x7f5d681442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d681465b0 con 0x7f5d68142650 2020-07-18T03:26:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6814cb10 con 0x7f5d68146aa0 2020-07-18T03:26:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6cb1b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68146310 con 0x7f5d68064440 2020-07-18T03:26:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d65d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d68146aa0 0x7f5d68143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d65d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d68146aa0 0x7f5d68143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59436/0 (socket says 172.21.15.47:59436) 2020-07-18T03:26:20.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d65d9b700 1 -- 172.21.15.47:0/3750455438 learned_addr learned my addr 172.21.15.47:0/3750455438 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:20.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d6659c700 1 --2- 172.21.15.47:0/3750455438 >> v2:172.21.15.18:3300/0 conn(0x7f5d68142650 0x7f5d6814c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.360+0000 7f5d66d9d700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d68064440 0x7f5d681442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d65d9b700 1 -- 172.21.15.47:0/3750455438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d68064440 msgr2=0x7f5d681442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d65d9b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d68064440 0x7f5d681442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d65d9b700 1 -- 172.21.15.47:0/3750455438 >> v2:172.21.15.18:3300/0 conn(0x7f5d68142650 msgr2=0x7f5d6814c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d65d9b700 1 --2- 172.21.15.47:0/3750455438 >> v2:172.21.15.18:3300/0 conn(0x7f5d68142650 0x7f5d6814c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d65d9b700 1 -- 172.21.15.47:0/3750455438 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68142a30 con 0x7f5d68146aa0 2020-07-18T03:26:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d66d9d700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d68064440 0x7f5d681442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d65d9b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d68146aa0 0x7f5d68143e00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5d5800d000 tx=0x7f5d580076a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5800f860 con 0x7f5d68146aa0 2020-07-18T03:26:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5800f9c0 con 0x7f5d68146aa0 2020-07-18T03:26:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d68063340 con 0x7f5d68146aa0 2020-07-18T03:26:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5800fd20 con 0x7f5d68146aa0 2020-07-18T03:26:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.361+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d681447e0 con 0x7f5d68146aa0 2020-07-18T03:26:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.362+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d68146790 con 0x7f5d68146aa0 2020-07-18T03:26:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.362+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5d5801b030 con 0x7f5d68146aa0 2020-07-18T03:26:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.362+0000 7f5d5e7fc700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4404ff70 0x7f5d44052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.362+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5d58067690 con 0x7f5d68146aa0 2020-07-18T03:26:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.362+0000 7f5d6659c700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4404ff70 0x7f5d44052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.363+0000 7f5d6659c700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4404ff70 0x7f5d44052430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f5d5001b000 tx=0x7f5d50016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.366+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5d5803fe20 con 0x7f5d68146aa0 2020-07-18T03:26:20.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.523+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d68063890 con 0x7f5d68146aa0 2020-07-18T03:26:20.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.524+0000 7f5d5e7fc700 1 -- 172.21.15.47:0/3750455438 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5d580190a0 con 0x7f5d68146aa0 2020-07-18T03:26:20.525 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:20.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4404ff70 msgr2=0x7f5d44052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4404ff70 0x7f5d44052430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f5d5001b000 tx=0x7f5d50016000).stop 2020-07-18T03:26:20.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d68146aa0 msgr2=0x7f5d68143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d68146aa0 0x7f5d68143e00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5d5800d000 tx=0x7f5d580076a0).stop 2020-07-18T03:26:20.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 shutdown_connections 2020-07-18T03:26:20.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4404ff70 0x7f5d44052430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d68064440 0x7f5d681442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/3750455438 >> v2:172.21.15.18:3300/0 conn(0x7f5d68142650 0x7f5d6814c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 --2- 172.21.15.47:0/3750455438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d68146aa0 0x7f5d68143e00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 >> 172.21.15.47:0/3750455438 conn(0x7f5d6805f5a0 msgr2=0x7f5d680aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 shutdown_connections 2020-07-18T03:26:20.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.525+0000 7f5d6cb1b700 1 -- 172.21.15.47:0/3750455438 wait complete. 2020-07-18T03:26:20.537 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 Processor -- start 2020-07-18T03:26:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 -- start start 2020-07-18T03:26:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280ad020 0x7f01280b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0128147d50 0x7f01280616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012814b520 con 0x7f01281478f0 2020-07-18T03:26:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01281471b0 con 0x7f0128147d50 2020-07-18T03:26:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.653+0000 7f012fe1a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0128147450 con 0x7f01280ad020 2020-07-18T03:26:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012e3b7700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0128147d50 0x7f01280616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012dbb6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280ad020 0x7f01280b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012d3b5700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f01281478f0 0x7f0128144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59296/0 (socket says 172.21.15.47:59296) 2020-07-18T03:26:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012d3b5700 1 -- 172.21.15.47:0/1719573424 learned_addr learned my addr 172.21.15.47:0/1719573424 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012cbb4700 1 -- 172.21.15.47:0/1719573424 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680020230 0 0) 0x7f012814b520 con 0x7f01281478f0 2020-07-18T03:26:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012cbb4700 1 -- 172.21.15.47:0/1719573424 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0110003240 con 0x7f01281478f0 2020-07-18T03:26:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012e3b7700 1 -- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280ad020 msgr2=0x7f01280b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012e3b7700 1 --2- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280ad020 0x7f01280b9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012e3b7700 1 -- 172.21.15.47:0/1719573424 >> v1:172.21.15.18:6789/0 conn(0x7f01281478f0 legacy=0x7f0128144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012e3b7700 1 -- 172.21.15.47:0/1719573424 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280ba390 con 0x7f0128147d50 2020-07-18T03:26:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.654+0000 7f012e3b7700 1 --2- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128147d50 0x7f01280616b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f01200032b0 tx=0x7f01200091d0).ready entity=mon.0 client_cookie=cd547aac95351e78 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012cbb4700 1 -- 172.21.15.47:0/1719573424 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012000b940 con 0x7f0128147d50 2020-07-18T03:26:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012cbb4700 1 -- 172.21.15.47:0/1719573424 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012000baa0 con 0x7f0128147d50 2020-07-18T03:26:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012cbb4700 1 -- 172.21.15.47:0/1719573424 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012000bc10 con 0x7f0128147d50 2020-07-18T03:26:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 -- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128147d50 msgr2=0x7f01280616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128147d50 0x7f01280616b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f01200032b0 tx=0x7f01200091d0).stop 2020-07-18T03:26:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 -- 172.21.15.47:0/1719573424 shutdown_connections 2020-07-18T03:26:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128147d50 0x7f01280616b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1719573424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280ad020 0x7f01280b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 -- 172.21.15.47:0/1719573424 >> 172.21.15.47:0/1719573424 conn(0x7f012805f5a0 msgr2=0x7f012805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 -- 172.21.15.47:0/1719573424 shutdown_connections 2020-07-18T03:26:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.655+0000 7f012fe1a700 1 -- 172.21.15.47:0/1719573424 wait complete. 2020-07-18T03:26:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.656+0000 7f012fe1a700 1 Processor -- start 2020-07-18T03:26:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.656+0000 7f012fe1a700 1 -- start start 2020-07-18T03:26:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012fe1a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f01280ad020 0x7f01280bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012fe1a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280bbaf0 0x7f0128138170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012fe1a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01280bd350 con 0x7f01281478f0 2020-07-18T03:26:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012fe1a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01280bceb0 con 0x7f01280ad020 2020-07-18T03:26:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012fe1a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01280bd0b0 con 0x7f01280bbaf0 2020-07-18T03:26:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012d3b5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01281478f0 0x7f012813c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40538/0 (socket says 172.21.15.47:40538) 2020-07-18T03:26:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012d3b5700 1 -- 172.21.15.47:0/1600855977 learned_addr learned my addr 172.21.15.47:0/1600855977 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012dbb6700 1 --2- 172.21.15.47:0/1600855977 >> v2:172.21.15.18:3300/0 conn(0x7f01280ad020 0x7f01280bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012e3b7700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280bbaf0 0x7f0128138170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068492988 0 0) 0x7f01280bd350 con 0x7f01281478f0 2020-07-18T03:26:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0100003240 con 0x7f01281478f0 2020-07-18T03:26:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012dbb6700 1 -- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280bbaf0 msgr2=0x7f0128138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012dbb6700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280bbaf0 0x7f0128138170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.657+0000 7f012dbb6700 1 -- 172.21.15.47:0/1600855977 >> v1:172.21.15.47:6789/0 conn(0x7f01281478f0 legacy=0x7f012813c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f012dbb6700 1 -- 172.21.15.47:0/1600855977 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01281453f0 con 0x7f01280ad020 2020-07-18T03:26:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f012dbb6700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01280ad020 0x7f01280bc9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f011800c000 tx=0x7f0118006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011800ca10 con 0x7f01280ad020 2020-07-18T03:26:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0128138660 con 0x7f01280ad020 2020-07-18T03:26:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011800cb70 con 0x7f01280ad020 2020-07-18T03:26:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f011800be70 con 0x7f01280ad020 2020-07-18T03:26:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.658+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0128145ad0 con 0x7f01280ad020 2020-07-18T03:26:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.659+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128138940 con 0x7f01280ad020 2020-07-18T03:26:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.659+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f01180123f0 con 0x7f01280ad020 2020-07-18T03:26:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.659+0000 7f0115ffb700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f01000502e0 0x7f01000527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.660+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f01180680f0 con 0x7f01280ad020 2020-07-18T03:26:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.660+0000 7f012d3b5700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f01000502e0 0x7f01000527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.660+0000 7f012d3b5700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f01000502e0 0x7f01000527a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f0124001010 tx=0x7f0124007880).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.664+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f01180407a0 con 0x7f01280ad020 2020-07-18T03:26:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.819+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01280ad7c0 con 0x7f01280ad020 2020-07-18T03:26:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.819+0000 7f0115ffb700 1 -- 172.21.15.47:0/1600855977 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0118043dc0 con 0x7f01280ad020 2020-07-18T03:26:20.821 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:20.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f01000502e0 msgr2=0x7f01000527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f01000502e0 0x7f01000527a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f0124001010 tx=0x7f0124007880).stop 2020-07-18T03:26:20.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01280ad020 msgr2=0x7f01280bc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01280ad020 0x7f01280bc9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f011800c000 tx=0x7f0118006dd0).stop 2020-07-18T03:26:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 shutdown_connections 2020-07-18T03:26:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f01280bbaf0 0x7f0128138170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01280ad020 0x7f01280bc9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 --2- 172.21.15.47:0/1600855977 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f01000502e0 0x7f01000527a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 >> 172.21.15.47:0/1600855977 conn(0x7f012805f5a0 msgr2=0x7f01280616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 shutdown_connections 2020-07-18T03:26:20.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.821+0000 7f012fe1a700 1 -- 172.21.15.47:0/1600855977 wait complete. 2020-07-18T03:26:20.832 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.950+0000 7f5d712e0700 1 Processor -- start 2020-07-18T03:26:20.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.950+0000 7f5d712e0700 1 -- start start 2020-07-18T03:26:20.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d712e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d6c0b2e10 0x7f5d6c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d712e0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.962 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d712e0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d6c0b2130 0x7f5d6c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d712e0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c144e30 con 0x7f5d6c0b1560 2020-07-18T03:26:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d712e0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c0afbd0 con 0x7f5d6c0b2e10 2020-07-18T03:26:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d712e0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c1491c0 con 0x7f5d6c0b2130 2020-07-18T03:26:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d6affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d6c0b2e10 0x7f5d6c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d6affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d6c0b2e10 0x7f5d6c0b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59456/0 (socket says 172.21.15.47:59456) 2020-07-18T03:26:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.951+0000 7f5d6affd700 1 -- 172.21.15.47:0/757143893 learned_addr learned my addr 172.21.15.47:0/757143893 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:20.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6a7fc700 1 --2- 172.21.15.47:0/757143893 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6b7fe700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d6c0b2130 0x7f5d6c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6affd700 1 -- 172.21.15.47:0/757143893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d6c0b2130 msgr2=0x7f5d6c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6affd700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d6c0b2130 0x7f5d6c0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6affd700 1 -- 172.21.15.47:0/757143893 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b1560 msgr2=0x7f5d6c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6affd700 1 --2- 172.21.15.47:0/757143893 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6affd700 1 -- 172.21.15.47:0/757143893 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c145050 con 0x7f5d6c0b2e10 2020-07-18T03:26:20.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6b7fe700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d6c0b2130 0x7f5d6c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:20.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d6affd700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b2e10 0x7f5d6c0b31f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f5d6c0ba340 tx=0x7f5d54006e90).ready entity=mon.1 client_cookie=f53e1c84ac901b24 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d69ffb700 1 -- 172.21.15.47:0/757143893 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d540098e0 con 0x7f5d6c0b2e10 2020-07-18T03:26:20.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d69ffb700 1 -- 172.21.15.47:0/757143893 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54009a40 con 0x7f5d6c0b2e10 2020-07-18T03:26:20.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d69ffb700 1 -- 172.21.15.47:0/757143893 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d54009da0 con 0x7f5d6c0b2e10 2020-07-18T03:26:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 -- 172.21.15.47:0/757143893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b2e10 msgr2=0x7f5d6c0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b2e10 0x7f5d6c0b31f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f5d6c0ba340 tx=0x7f5d54006e90).stop 2020-07-18T03:26:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 -- 172.21.15.47:0/757143893 shutdown_connections 2020-07-18T03:26:20.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d6c0b2130 0x7f5d6c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 --2- 172.21.15.47:0/757143893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b2e10 0x7f5d6c0b31f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 --2- 172.21.15.47:0/757143893 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 -- 172.21.15.47:0/757143893 >> 172.21.15.47:0/757143893 conn(0x7f5d6c0b9340 msgr2=0x7f5d6c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:20.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 -- 172.21.15.47:0/757143893 shutdown_connections 2020-07-18T03:26:20.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.952+0000 7f5d712e0700 1 -- 172.21.15.47:0/757143893 wait complete. 2020-07-18T03:26:20.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.954+0000 7f5d712e0700 1 Processor -- start 2020-07-18T03:26:20.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.954+0000 7f5d712e0700 1 -- start start 2020-07-18T03:26:20.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b2130 0x7f5d6c13a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c158c50 con 0x7f5d6c0ae700 2020-07-18T03:26:20.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c1425a0 con 0x7f5d6c0b2130 2020-07-18T03:26:20.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c158ef0 con 0x7f5d6c0b1560 2020-07-18T03:26:20.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d6c0b1560 0x7f5d6c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59460/0 (socket says 172.21.15.47:59460) 2020-07-18T03:26:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 -- 172.21.15.47:0/4142559330 learned_addr learned my addr 172.21.15.47:0/4142559330 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5d6c0ae700 0x7f5d6c13c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40552/0 (socket says 172.21.15.47:40552) 2020-07-18T03:26:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6a7fc700 1 --2- 172.21.15.47:0/4142559330 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b2130 0x7f5d6c13a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 -- 172.21.15.47:0/4142559330 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b2130 msgr2=0x7f5d6c13a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 --2- 172.21.15.47:0/4142559330 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b2130 0x7f5d6c13a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 -- 172.21.15.47:0/4142559330 >> v1:172.21.15.47:6789/0 conn(0x7f5d6c0ae700 legacy=0x7f5d6c13c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 -- 172.21.15.47:0/4142559330 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c159190 con 0x7f5d6c0b1560 2020-07-18T03:26:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441931312 0 0) 0x7f5d6c158c50 con 0x7f5d6c0ae700 2020-07-18T03:26:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d6affd700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b1560 0x7f5d6c159630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f5d6c0ba340 tx=0x7f5d54015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d540098e0 con 0x7f5d6c0b1560 2020-07-18T03:26:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54011e80 con 0x7f5d6c0b1560 2020-07-18T03:26:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5401d7c0 con 0x7f5d6c0b1560 2020-07-18T03:26:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c065d20 con 0x7f5d6c0b1560 2020-07-18T03:26:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.955+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c066190 con 0x7f5d6c0b1560 2020-07-18T03:26:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.957+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5d54025030 con 0x7f5d6c0b1560 2020-07-18T03:26:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.957+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c057450 con 0x7f5d6c0b1560 2020-07-18T03:26:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.957+0000 7f5d62ffd700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4804fe80 0x7f5d48052340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.957+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5d54069300 con 0x7f5d6c0b1560 2020-07-18T03:26:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.957+0000 7f5d6a7fc700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4804fe80 0x7f5d48052340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.957+0000 7f5d6a7fc700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4804fe80 0x7f5d48052340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5d5c00c000 tx=0x7f5d5c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:20.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:20.961+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5d54041a90 con 0x7f5d6c0b1560 2020-07-18T03:26:21.119 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.119+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d6c066b60 con 0x7f5d6c0b1560 2020-07-18T03:26:21.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.119+0000 7f5d62ffd700 1 -- 172.21.15.47:0/4142559330 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f5d540450b0 con 0x7f5d6c0b1560 2020-07-18T03:26:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:21.122 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4804fe80 msgr2=0x7f5d48052340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4804fe80 0x7f5d48052340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5d5c00c000 tx=0x7f5d5c008040).stop 2020-07-18T03:26:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b1560 msgr2=0x7f5d6c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b1560 0x7f5d6c159630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f5d6c0ba340 tx=0x7f5d54015000).stop 2020-07-18T03:26:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 shutdown_connections 2020-07-18T03:26:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d4804fe80 0x7f5d48052340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 --2- 172.21.15.47:0/4142559330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d6c0b1560 0x7f5d6c159630 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 --2- 172.21.15.47:0/4142559330 >> v2:172.21.15.18:3300/0 conn(0x7f5d6c0b2130 0x7f5d6c13a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 >> 172.21.15.47:0/4142559330 conn(0x7f5d6c0b9340 msgr2=0x7f5d6c0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 shutdown_connections 2020-07-18T03:26:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.121+0000 7f5d712e0700 1 -- 172.21.15.47:0/4142559330 wait complete. 2020-07-18T03:26:21.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:21.138 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:21.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.262+0000 7f500fb10700 1 Processor -- start 2020-07-18T03:26:21.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.262+0000 7f500fb10700 1 -- start start 2020-07-18T03:26:21.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500fb10700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 0x7f50080b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500fb10700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5008147de0 con 0x7f50080b6300 2020-07-18T03:26:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500fb10700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50080ae000 con 0x7f50080ad330 2020-07-18T03:26:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500fb10700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5008148010 con 0x7f50080b59b0 2020-07-18T03:26:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500d8ac700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f50080ad330 0x7f5008143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40556/0 (socket says 172.21.15.47:40556) 2020-07-18T03:26:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500d8ac700 1 -- 172.21.15.47:0/3012919912 learned_addr learned my addr 172.21.15.47:0/3012919912 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500d0ab700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 0x7f50080b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884177473 0 0) 0x7f50080ae000 con 0x7f50080ad330 2020-07-18T03:26:21.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.263+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ff0003240 con 0x7f50080ad330 2020-07-18T03:26:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1089734899 0 0) 0x7f4ff0003240 con 0x7f50080ad330 2020-07-18T03:26:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 msgr2=0x7f50080b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 0x7f50080b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 >> v1:172.21.15.18:6789/0 conn(0x7f50080b6300 legacy=0x7f50080b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50080ad710 con 0x7f50080ad330 2020-07-18T03:26:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4ffc0016d0 con 0x7f50080ad330 2020-07-18T03:26:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 >> v1:172.21.15.47:6789/0 conn(0x7f50080ad330 legacy=0x7f5008143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ff0003730 0x7f4ff0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 0x7f4ff00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff00049d0 0x7f4ff0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008148010 con 0x7f4ff0003730 2020-07-18T03:26:21.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00041d0 con 0x7f4ff00049d0 2020-07-18T03:26:21.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff000b730 con 0x7f50080b59b0 2020-07-18T03:26:21.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff00049d0 0x7f4ff0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500d8ac700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ff0003730 0x7f4ff0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500d0ab700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 0x7f4ff00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 -- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 msgr2=0x7f4ff00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f50080b59b0 0x7f4ff00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.282 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 -- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ff0003730 msgr2=0x7f4ff0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ff0003730 0x7f4ff0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 -- 172.21.15.47:0/3012919912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0003240 con 0x7f4ff00049d0 2020-07-18T03:26:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500e0ad700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff00049d0 0x7f4ff0004e50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f50080ae4e0 tx=0x7f5000007e60).ready entity=mon.1 client_cookie=f0300f2705a91ec8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.264+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f500000c960 con 0x7f4ff00049d0 2020-07-18T03:26:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000cac0 con 0x7f4ff00049d0 2020-07-18T03:26:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500c8aa700 1 -- 172.21.15.47:0/3012919912 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f500000ce20 con 0x7f4ff00049d0 2020-07-18T03:26:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500fb10700 1 -- 172.21.15.47:0/3012919912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff00049d0 msgr2=0x7f4ff0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500fb10700 1 --2- 172.21.15.47:0/3012919912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff00049d0 0x7f4ff0004e50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f50080ae4e0 tx=0x7f5000007e60).stop 2020-07-18T03:26:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500e0ad700 1 -- 172.21.15.47:0/3012919912 reap_dead start 2020-07-18T03:26:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500fb10700 1 -- 172.21.15.47:0/3012919912 shutdown_connections 2020-07-18T03:26:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500fb10700 1 -- 172.21.15.47:0/3012919912 >> 172.21.15.47:0/3012919912 conn(0x7f500805f5a0 msgr2=0x7f500805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:21.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500fb10700 1 -- 172.21.15.47:0/3012919912 shutdown_connections 2020-07-18T03:26:21.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.265+0000 7f500fb10700 1 -- 172.21.15.47:0/3012919912 wait complete. 2020-07-18T03:26:21.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.266+0000 7f500fb10700 1 Processor -- start 2020-07-18T03:26:21.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.266+0000 7f500fb10700 1 -- start start 2020-07-18T03:26:21.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500fb10700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f50080b1c10 0x7f5008156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500fb10700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f500813fef0 0x7f500812fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500fb10700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50081302e0 con 0x7f500812fa10 2020-07-18T03:26:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500fb10700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5008156420 con 0x7f50080b1c10 2020-07-18T03:26:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500fb10700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50081566c0 con 0x7f500813fef0 2020-07-18T03:26:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500d8ac700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f500812fa10 0x7f5008061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40566/0 (socket says 172.21.15.47:40566) 2020-07-18T03:26:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500e0ad700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f500813fef0 0x7f500812fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500d8ac700 1 -- 172.21.15.47:0/1151223460 learned_addr learned my addr 172.21.15.47:0/1151223460 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500d0ab700 1 --2- 172.21.15.47:0/1151223460 >> v2:172.21.15.18:3300/0 conn(0x7f50080b1c10 0x7f5008156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142960922 0 0) 0x7f50081302e0 con 0x7f500812fa10 2020-07-18T03:26:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe0003240 con 0x7f500812fa10 2020-07-18T03:26:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500e0ad700 1 -- 172.21.15.47:0/1151223460 >> v2:172.21.15.18:3300/0 conn(0x7f50080b1c10 msgr2=0x7f5008156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500e0ad700 1 --2- 172.21.15.47:0/1151223460 >> v2:172.21.15.18:3300/0 conn(0x7f50080b1c10 0x7f5008156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500e0ad700 1 -- 172.21.15.47:0/1151223460 >> v1:172.21.15.47:6789/0 conn(0x7f500812fa10 legacy=0x7f5008061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500e0ad700 1 -- 172.21.15.47:0/1151223460 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008156960 con 0x7f500813fef0 2020-07-18T03:26:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.267+0000 7f500e0ad700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f500813fef0 0x7f500812fdf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f5008141590 tx=0x7f5000017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.268+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f500000c960 con 0x7f500813fef0 2020-07-18T03:26:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.268+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000b030 con 0x7f500813fef0 2020-07-18T03:26:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.268+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5008062290 con 0x7f500813fef0 2020-07-18T03:26:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.268+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f500001e810 con 0x7f500813fef0 2020-07-18T03:26:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.268+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50081418c0 con 0x7f500813fef0 2020-07-18T03:26:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.269+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500805b120 con 0x7f500813fef0 2020-07-18T03:26:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.269+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5000010070 con 0x7f500813fef0 2020-07-18T03:26:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.269+0000 7f4ff9ffb700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4fe00503f0 0x7f4fe00528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.269+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5000068b00 con 0x7f500813fef0 2020-07-18T03:26:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.269+0000 7f500d8ac700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4fe00503f0 0x7f4fe00528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.270+0000 7f500d8ac700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4fe00503f0 0x7f4fe00528b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f4ffc00d000 tx=0x7f4ffc007750).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.273+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5000041300 con 0x7f500813fef0 2020-07-18T03:26:21.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.430+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f50080690d0 con 0x7f500813fef0 2020-07-18T03:26:21.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.430+0000 7f4ff9ffb700 1 -- 172.21.15.47:0/1151223460 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5000023070 con 0x7f500813fef0 2020-07-18T03:26:21.431 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:21.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4fe00503f0 msgr2=0x7f4fe00528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4fe00503f0 0x7f4fe00528b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f4ffc00d000 tx=0x7f4ffc007750).stop 2020-07-18T03:26:21.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f500813fef0 msgr2=0x7f500812fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f500813fef0 0x7f500812fdf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f5008141590 tx=0x7f5000017000).stop 2020-07-18T03:26:21.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 shutdown_connections 2020-07-18T03:26:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f500813fef0 0x7f500812fdf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 --2- 172.21.15.47:0/1151223460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4fe00503f0 0x7f4fe00528b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 --2- 172.21.15.47:0/1151223460 >> v2:172.21.15.18:3300/0 conn(0x7f50080b1c10 0x7f5008156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.431+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 >> 172.21.15.47:0/1151223460 conn(0x7f500805f5a0 msgr2=0x7f50080b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:21.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.432+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 shutdown_connections 2020-07-18T03:26:21.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.432+0000 7f500fb10700 1 -- 172.21.15.47:0/1151223460 wait complete. 2020-07-18T03:26:21.443 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.559+0000 7f9d85f2a700 1 Processor -- start 2020-07-18T03:26:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.559+0000 7f9d85f2a700 1 -- start start 2020-07-18T03:26:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d85f2a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9d800ad330 0x7f9d80143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d85f2a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d80147de0 con 0x7f9d800b59b0 2020-07-18T03:26:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d85f2a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800ae000 con 0x7f9d800b3a10 2020-07-18T03:26:21.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d85f2a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d80148010 con 0x7f9d800ad330 2020-07-18T03:26:21.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d7ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d800b3a10 0x7f9d800b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40574/0 (socket says 172.21.15.47:40574) 2020-07-18T03:26:21.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d7ffff700 1 -- 172.21.15.47:0/2226903103 learned_addr learned my addr 172.21.15.47:0/2226903103 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:21.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2226903103 >> v2:172.21.15.18:3300/0 conn(0x7f9d800ad330 0x7f9d80143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684389408 0 0) 0x7f9d800ae000 con 0x7f9d800b3a10 2020-07-18T03:26:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.560+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d6c003240 con 0x7f9d800b3a10 2020-07-18T03:26:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799195091 0 0) 0x7f9d80147de0 con 0x7f9d800b59b0 2020-07-18T03:26:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d800ae000 con 0x7f9d800b59b0 2020-07-18T03:26:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1648302942 0 0) 0x7f9d6c003240 con 0x7f9d800b3a10 2020-07-18T03:26:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 >> v2:172.21.15.18:3300/0 conn(0x7f9d800ad330 msgr2=0x7f9d80143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 --2- 172.21.15.47:0/2226903103 >> v2:172.21.15.18:3300/0 conn(0x7f9d800ad330 0x7f9d80143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 >> v1:172.21.15.18:6789/0 conn(0x7f9d800b59b0 legacy=0x7f9d800b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800ad710 con 0x7f9d800b3a10 2020-07-18T03:26:21.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f9d78001a50 con 0x7f9d800b3a10 2020-07-18T03:26:21.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 >> v1:172.21.15.47:6789/0 conn(0x7f9d800b3a10 legacy=0x7f9d800b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c0038c0 0x7f9d6c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d6c0044e0 0x7f9d6c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d6c004e50 0x7f9d6c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80148010 con 0x7f9d6c0038c0 2020-07-18T03:26:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c00bc20 con 0x7f9d6c004e50 2020-07-18T03:26:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c00bde0 con 0x7f9d6c0044e0 2020-07-18T03:26:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2226903103 >> v2:172.21.15.18:3300/0 conn(0x7f9d800ad330 0x7f9d80143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d6c004e50 0x7f9d6c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7effd700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d6c0044e0 0x7f9d6c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c0038c0 0x7f9d6c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 -- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d6c0044e0 msgr2=0x7f9d6c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d6c0044e0 0x7f9d6c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 -- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c0038c0 msgr2=0x7f9d6c003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c0038c0 0x7f9d6c003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 -- 172.21.15.47:0/2226903103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c003240 con 0x7f9d6c004e50 2020-07-18T03:26:21.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c0038c0 0x7f9d6c003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:21.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 -- 172.21.15.47:0/2226903103 reap_dead start 2020-07-18T03:26:21.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7ffff700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d6c004e50 0x7f9d6c00b730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f9d6c0054b0 tx=0x7f9d78007a20).ready entity=mon.1 client_cookie=a95b0e868cc3c149 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d78005b80 con 0x7f9d6c004e50 2020-07-18T03:26:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78005ce0 con 0x7f9d6c004e50 2020-07-18T03:26:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d7e7fc700 1 -- 172.21.15.47:0/2226903103 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d780072c0 con 0x7f9d6c004e50 2020-07-18T03:26:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2226903103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d6c004e50 msgr2=0x7f9d6c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.561+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d6c004e50 0x7f9d6c00b730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f9d6c0054b0 tx=0x7f9d78007a20).stop 2020-07-18T03:26:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.562+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2226903103 shutdown_connections 2020-07-18T03:26:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.562+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2226903103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d6c004e50 0x7f9d6c00b730 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.562+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2226903103 >> 172.21.15.47:0/2226903103 conn(0x7f9d8005f5a0 msgr2=0x7f9d8005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.562+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2226903103 shutdown_connections 2020-07-18T03:26:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.562+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2226903103 wait complete. 2020-07-18T03:26:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.563+0000 7f9d85f2a700 1 Processor -- start 2020-07-18T03:26:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.563+0000 7f9d85f2a700 1 -- start start 2020-07-18T03:26:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d85f2a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9d80143800 0x7f9d80066750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d85f2a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d80143be0 0x7f9d8013c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d85f2a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d800651f0 0x7f9d8013cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d85f2a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d80154c50 con 0x7f9d80143800 2020-07-18T03:26:21.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d85f2a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d80154ef0 con 0x7f9d80143be0 2020-07-18T03:26:21.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d85f2a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8013d950 con 0x7f9d800651f0 2020-07-18T03:26:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d80143be0 0x7f9d8013c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d80143be0 0x7f9d8013c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59500/0 (socket says 172.21.15.47:59500) 2020-07-18T03:26:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 -- 172.21.15.47:0/2374692438 learned_addr learned my addr 172.21.15.47:0/2374692438 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2374692438 >> v2:172.21.15.18:3300/0 conn(0x7f9d80143800 0x7f9d80066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7ffff700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d800651f0 0x7f9d8013cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 -- 172.21.15.47:0/2374692438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d800651f0 msgr2=0x7f9d8013cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d800651f0 0x7f9d8013cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 -- 172.21.15.47:0/2374692438 >> v2:172.21.15.18:3300/0 conn(0x7f9d80143800 msgr2=0x7f9d80066750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 --2- 172.21.15.47:0/2374692438 >> v2:172.21.15.18:3300/0 conn(0x7f9d80143800 0x7f9d80066750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 -- 172.21.15.47:0/2374692438 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8013dac0 con 0x7f9d80143be0 2020-07-18T03:26:21.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7ffff700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d800651f0 0x7f9d8013cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:21.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.564+0000 7f9d7effd700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d80143be0 0x7f9d8013c760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f9d7400d000 tx=0x7f9d74007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.565+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7400fa90 con 0x7f9d80143be0 2020-07-18T03:26:21.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.565+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7400fbf0 con 0x7f9d80143be0 2020-07-18T03:26:21.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.565+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d80064d60 con 0x7f9d80143be0 2020-07-18T03:26:21.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.565+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d74014450 con 0x7f9d80143be0 2020-07-18T03:26:21.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.565+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d800633c0 con 0x7f9d80143be0 2020-07-18T03:26:21.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.565+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d80142f40 con 0x7f9d80143be0 2020-07-18T03:26:21.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.566+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9d7401b030 con 0x7f9d80143be0 2020-07-18T03:26:21.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.566+0000 7f9d5f7fe700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d5804ff10 0x7f9d580523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.566+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9d74067cd0 con 0x7f9d80143be0 2020-07-18T03:26:21.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.566+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d5804ff10 0x7f9d580523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.566+0000 7f9d7f7fe700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d5804ff10 0x7f9d580523d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9d70008000 tx=0x7f9d70008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.569+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9d74040460 con 0x7f9d80143be0 2020-07-18T03:26:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.725+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9d80064a50 con 0x7f9d80143be0 2020-07-18T03:26:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.726+0000 7f9d5f7fe700 1 -- 172.21.15.47:0/2374692438 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d74043a80 con 0x7f9d80143be0 2020-07-18T03:26:21.728 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d5804ff10 msgr2=0x7f9d580523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d5804ff10 0x7f9d580523d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9d70008000 tx=0x7f9d70008040).stop 2020-07-18T03:26:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d80143be0 msgr2=0x7f9d8013c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d80143be0 0x7f9d8013c760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f9d7400d000 tx=0x7f9d74007b70).stop 2020-07-18T03:26:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 shutdown_connections 2020-07-18T03:26:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d5804ff10 0x7f9d580523d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d800651f0 0x7f9d8013cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2374692438 >> v2:172.21.15.18:3300/0 conn(0x7f9d80143800 0x7f9d80066750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 --2- 172.21.15.47:0/2374692438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d80143be0 0x7f9d8013c760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 >> 172.21.15.47:0/2374692438 conn(0x7f9d8005f5a0 msgr2=0x7f9d800af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 shutdown_connections 2020-07-18T03:26:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.727+0000 7f9d85f2a700 1 -- 172.21.15.47:0/2374692438 wait complete. 2020-07-18T03:26:21.739 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:21.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.856+0000 7f0ac312b700 1 Processor -- start 2020-07-18T03:26:21.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.856+0000 7f0ac312b700 1 -- start start 2020-07-18T03:26:21.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0ac312b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0ac312b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0abc142430 0x7f0abc148b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0ac312b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc061db0 con 0x7f0abc141fb0 2020-07-18T03:26:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0ac312b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc0b14c0 con 0x7f0abc144e70 2020-07-18T03:26:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0ac312b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc1428c0 con 0x7f0abc142430 2020-07-18T03:26:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0ac16c8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0abc142430 0x7f0abc148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0abbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0abc141fb0 0x7f0abc148440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40596/0 (socket says 172.21.15.47:40596) 2020-07-18T03:26:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.857+0000 7f0abbfff700 1 -- 172.21.15.47:0/298156150 learned_addr learned my addr 172.21.15.47:0/298156150 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac0ec7700 1 --2- 172.21.15.47:0/298156150 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac16c8700 1 -- 172.21.15.47:0/298156150 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 msgr2=0x7f0abc0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac16c8700 1 --2- 172.21.15.47:0/298156150 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc0b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac16c8700 1 -- 172.21.15.47:0/298156150 >> v1:172.21.15.47:6789/0 conn(0x7f0abc141fb0 legacy=0x7f0abc148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0abb7fe700 1 -- 172.21.15.47:0/298156150 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390416748 0 0) 0x7f0abc061db0 con 0x7f0abc141fb0 2020-07-18T03:26:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac16c8700 1 -- 172.21.15.47:0/298156150 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc062050 con 0x7f0abc142430 2020-07-18T03:26:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac16c8700 1 --2- 172.21.15.47:0/298156150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0abc142430 0x7f0abc148b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f0ab40032b0 tx=0x7f0ab40091d0).ready entity=mon.1 client_cookie=81fddcedfb71f7ae server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0abb7fe700 1 -- 172.21.15.47:0/298156150 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab400ba50 con 0x7f0abc142430 2020-07-18T03:26:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0abb7fe700 1 -- 172.21.15.47:0/298156150 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab400bbb0 con 0x7f0abc142430 2020-07-18T03:26:21.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0abb7fe700 1 -- 172.21.15.47:0/298156150 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab400bd20 con 0x7f0abc142430 2020-07-18T03:26:21.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 -- 172.21.15.47:0/298156150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0abc142430 msgr2=0x7f0abc148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 --2- 172.21.15.47:0/298156150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0abc142430 0x7f0abc148b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f0ab40032b0 tx=0x7f0ab40091d0).stop 2020-07-18T03:26:21.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 -- 172.21.15.47:0/298156150 shutdown_connections 2020-07-18T03:26:21.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 --2- 172.21.15.47:0/298156150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0abc142430 0x7f0abc148b50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 --2- 172.21.15.47:0/298156150 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 -- 172.21.15.47:0/298156150 >> 172.21.15.47:0/298156150 conn(0x7f0abc05f5a0 msgr2=0x7f0abc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:21.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 -- 172.21.15.47:0/298156150 shutdown_connections 2020-07-18T03:26:21.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.858+0000 7f0ac312b700 1 -- 172.21.15.47:0/298156150 wait complete. 2020-07-18T03:26:21.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 Processor -- start 2020-07-18T03:26:21.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 -- start start 2020-07-18T03:26:21.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc062e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0abc146a50 0x7f0abc1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc146ef0 con 0x7f0abc141fb0 2020-07-18T03:26:21.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc147190 con 0x7f0abc144e70 2020-07-18T03:26:21.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.860+0000 7f0ac312b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc147430 con 0x7f0abc146a50 2020-07-18T03:26:21.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac0ec7700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f0abc141fb0 0x7f0abc156610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59366/0 (socket says 172.21.15.47:59366) 2020-07-18T03:26:21.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0abbfff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc062e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac16c8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0abc146a50 0x7f0abc1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac0ec7700 1 -- 172.21.15.47:0/2483231806 learned_addr learned my addr 172.21.15.47:0/2483231806 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:21.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac16c8700 1 -- 172.21.15.47:0/2483231806 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 msgr2=0x7f0abc062e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac16c8700 1 --2- 172.21.15.47:0/2483231806 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc062e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac16c8700 1 -- 172.21.15.47:0/2483231806 >> v1:172.21.15.18:6789/0 conn(0x7f0abc141fb0 legacy=0x7f0abc156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac16c8700 1 -- 172.21.15.47:0/2483231806 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc156fd0 con 0x7f0abc146a50 2020-07-18T03:26:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac16c8700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0abc146a50 0x7f0abc1478d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0ab40032b0 tx=0x7f0ab4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab4010870 con 0x7f0abc146a50 2020-07-18T03:26:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40109d0 con 0x7f0abc146a50 2020-07-18T03:26:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab4010d30 con 0x7f0abc146a50 2020-07-18T03:26:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc063370 con 0x7f0abc146a50 2020-07-18T03:26:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.861+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc148000 con 0x7f0abc146a50 2020-07-18T03:26:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.862+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc1475a0 con 0x7f0abc146a50 2020-07-18T03:26:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.863+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0ab401e070 con 0x7f0abc146a50 2020-07-18T03:26:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.863+0000 7f0ab8ff9700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a9404fea0 0x7f0a94052360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.863+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0ab4068f40 con 0x7f0abc146a50 2020-07-18T03:26:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.863+0000 7f0ac0ec7700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a9404fea0 0x7f0a94052360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.863+0000 7f0ac0ec7700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a9404fea0 0x7f0a94052360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0abc143e50 tx=0x7f0aac008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:21.867+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0ab4041630 con 0x7f0abc146a50 2020-07-18T03:26:22.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.024+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0abc05b120 con 0x7f0abc146a50 2020-07-18T03:26:22.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.025+0000 7f0ab8ff9700 1 -- 172.21.15.47:0/2483231806 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0ab40150a0 con 0x7f0abc146a50 2020-07-18T03:26:22.026 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:22.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a9404fea0 msgr2=0x7f0a94052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a9404fea0 0x7f0a94052360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0abc143e50 tx=0x7f0aac008040).stop 2020-07-18T03:26:22.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0abc146a50 msgr2=0x7f0abc1478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0abc146a50 0x7f0abc1478d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0ab40032b0 tx=0x7f0ab4016000).stop 2020-07-18T03:26:22.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 shutdown_connections 2020-07-18T03:26:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0a9404fea0 0x7f0a94052360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 --2- 172.21.15.47:0/2483231806 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0abc146a50 0x7f0abc1478d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 --2- 172.21.15.47:0/2483231806 >> v2:172.21.15.18:3300/0 conn(0x7f0abc144e70 0x7f0abc062e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 >> 172.21.15.47:0/2483231806 conn(0x7f0abc05f5a0 msgr2=0x7f0abc0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 shutdown_connections 2020-07-18T03:26:22.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.026+0000 7f0ac312b700 1 -- 172.21.15.47:0/2483231806 wait complete. 2020-07-18T03:26:22.038 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.154+0000 7feaeb59e700 1 Processor -- start 2020-07-18T03:26:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.154+0000 7feaeb59e700 1 -- start start 2020-07-18T03:26:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feaeb59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feaec0b7670 0x7feaec1440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feaeb59e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec066d00 con 0x7feaec065fd0 2020-07-18T03:26:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feaeb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec0ad640 con 0x7feaec1445a0 2020-07-18T03:26:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feaeb59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaec0b7b00 con 0x7feaec0b7670 2020-07-18T03:26:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feae9d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feaec0b7670 0x7feaec1440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feaead9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feaec1445a0 0x7feaec0af380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40612/0 (socket says 172.21.15.47:40612) 2020-07-18T03:26:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feaead9d700 1 -- 172.21.15.47:0/3374356977 learned_addr learned my addr 172.21.15.47:0/3374356977 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feae9d9b700 1 -- 172.21.15.47:0/3374356977 >> v1:172.21.15.47:6789/0 conn(0x7feaec1445a0 legacy=0x7feaec0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feae9d9b700 1 -- 172.21.15.47:0/3374356977 >> v1:172.21.15.18:6789/0 conn(0x7feaec065fd0 legacy=0x7feaec05f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feae9d9b700 1 -- 172.21.15.47:0/3374356977 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec066980 con 0x7feaec0b7670 2020-07-18T03:26:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.155+0000 7feae959a700 1 -- 172.21.15.47:0/3374356977 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882732539 0 0) 0x7feaec0ad640 con 0x7feaec1445a0 2020-07-18T03:26:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feae9d9b700 1 --2- 172.21.15.47:0/3374356977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaec0b7670 0x7feaec1440b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7feadc002f30 tx=0x7feadc006dd0).ready entity=mon.1 client_cookie=d601a5cfb456ca22 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feae959a700 1 -- 172.21.15.47:0/3374356977 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feadc0097f0 con 0x7feaec0b7670 2020-07-18T03:26:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feae959a700 1 -- 172.21.15.47:0/3374356977 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc009950 con 0x7feaec0b7670 2020-07-18T03:26:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feae959a700 1 -- 172.21.15.47:0/3374356977 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feadc009cb0 con 0x7feaec0b7670 2020-07-18T03:26:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 -- 172.21.15.47:0/3374356977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaec0b7670 msgr2=0x7feaec1440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 --2- 172.21.15.47:0/3374356977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaec0b7670 0x7feaec1440b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7feadc002f30 tx=0x7feadc006dd0).stop 2020-07-18T03:26:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 -- 172.21.15.47:0/3374356977 shutdown_connections 2020-07-18T03:26:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 --2- 172.21.15.47:0/3374356977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaec0b7670 0x7feaec1440b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 -- 172.21.15.47:0/3374356977 >> 172.21.15.47:0/3374356977 conn(0x7feaec06b070 msgr2=0x7feaec05b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 -- 172.21.15.47:0/3374356977 shutdown_connections 2020-07-18T03:26:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.156+0000 7feaeb59e700 1 -- 172.21.15.47:0/3374356977 wait complete. 2020-07-18T03:26:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.157+0000 7feaeb59e700 1 Processor -- start 2020-07-18T03:26:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.157+0000 7feaeb59e700 1 -- start start 2020-07-18T03:26:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaeb59e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feaec156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaeb59e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec1572d0 con 0x7feaec1445a0 2020-07-18T03:26:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaeb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec148e60 con 0x7feaec065fd0 2020-07-18T03:26:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaeb59e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec1570b0 con 0x7feaec14a350 2020-07-18T03:26:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaea59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feaec065fd0 0x7feaec1564b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40614/0 (socket says 172.21.15.47:40614) 2020-07-18T03:26:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaea59c700 1 -- 172.21.15.47:0/3784512039 learned_addr learned my addr 172.21.15.47:0/3784512039 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602571899 0 0) 0x7feaec148e60 con 0x7feaec065fd0 2020-07-18T03:26:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac0003240 con 0x7feaec065fd0 2020-07-18T03:26:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.158+0000 7feaead9d700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feaec156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2215168777 0 0) 0x7feac0003240 con 0x7feaec065fd0 2020-07-18T03:26:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feaec148e60 con 0x7feaec065fd0 2020-07-18T03:26:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7feae0001960 con 0x7feaec065fd0 2020-07-18T03:26:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 msgr2=0x7feaec156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feaec156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 >> v1:172.21.15.47:6789/0 conn(0x7feaec065fd0 legacy=0x7feaec1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 >> v1:172.21.15.18:6789/0 conn(0x7feaec1445a0 legacy=0x7feaec1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0003800 0x7feac0003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feac00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac00049d0 0x7feac0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec1570b0 con 0x7feac00049d0 2020-07-18T03:26:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00041f0 con 0x7feac0003800 2020-07-18T03:26:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac000fb40 con 0x7feaec14a350 2020-07-18T03:26:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0003800 0x7feac0003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feae9d9b700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feac00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaead9d700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac00049d0 0x7feac0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 -- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 msgr2=0x7feac00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feac00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 -- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac00049d0 msgr2=0x7feac0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac00049d0 0x7feac0004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec149100 con 0x7feac0003800 2020-07-18T03:26:22.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feae9d9b700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feaec14a350 0x7feac00044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:22.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec147e40 con 0x7feac0003800 2020-07-18T03:26:22.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec156e00 con 0x7feac0003800 2020-07-18T03:26:22.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feaea59c700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0003800 0x7feac0003be0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7feae0001670 tx=0x7feae0007890).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feae0007320 con 0x7feac0003800 2020-07-18T03:26:22.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae000abb0 con 0x7feac0003800 2020-07-18T03:26:22.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.159+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feae00054e0 con 0x7feac0003800 2020-07-18T03:26:22.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.160+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec149300 con 0x7feac0003800 2020-07-18T03:26:22.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.160+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7feae0010030 con 0x7feac0003800 2020-07-18T03:26:22.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.161+0000 7feada7fc700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7feac005c760 0x7feac005ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.161+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feae0068220 con 0x7feac0003800 2020-07-18T03:26:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.161+0000 7feae9d9b700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7feac005c760 0x7feac005ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.161+0000 7feae9d9b700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7feac005c760 0x7feac005ec20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7feadc01a000 tx=0x7feadc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.164+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7feaec149300 con 0x7feac0003800 2020-07-18T03:26:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.321+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feaec149300 con 0x7feac0003800 2020-07-18T03:26:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.322+0000 7feada7fc700 1 -- 172.21.15.47:0/3784512039 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7feaec149300 con 0x7feac0003800 2020-07-18T03:26:22.323 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7feac005c760 msgr2=0x7feac005ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7feac005c760 0x7feac005ec20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7feadc01a000 tx=0x7feadc015000).stop 2020-07-18T03:26:22.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0003800 msgr2=0x7feac0003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 --2- 172.21.15.47:0/3784512039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac0003800 0x7feac0003be0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7feae0001670 tx=0x7feae0007890).stop 2020-07-18T03:26:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaead9d700 1 -- 172.21.15.47:0/3784512039 reap_dead start 2020-07-18T03:26:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 shutdown_connections 2020-07-18T03:26:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 >> 172.21.15.47:0/3784512039 conn(0x7feaec06b070 msgr2=0x7feaec143e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 shutdown_connections 2020-07-18T03:26:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.323+0000 7feaeb59e700 1 -- 172.21.15.47:0/3784512039 wait complete. 2020-07-18T03:26:22.335 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:22.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.448+0000 7f134448a700 1 Processor -- start 2020-07-18T03:26:22.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.448+0000 7f134448a700 1 -- start start 2020-07-18T03:26:22.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f134448a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f133c147a00 0x7f133c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f134448a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f133c1433e0 0x7f133c143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f134448a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c147330 con 0x7f133c0ad100 2020-07-18T03:26:22.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f134448a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c0ae530 con 0x7f133c147a00 2020-07-18T03:26:22.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f134448a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c0b57e0 con 0x7f133c1433e0 2020-07-18T03:26:22.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f133c1433e0 0x7f133c143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f133c1433e0 0x7f133c143820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59542/0 (socket says 172.21.15.47:59542) 2020-07-18T03:26:22.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342226700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f133c0ad100 0x7f133c0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40628/0 (socket says 172.21.15.47:40628) 2020-07-18T03:26:22.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 -- 172.21.15.47:0/2025227976 learned_addr learned my addr 172.21.15.47:0/2025227976 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:22.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1341a25700 1 --2- 172.21.15.47:0/2025227976 >> v2:172.21.15.18:3300/0 conn(0x7f133c147a00 0x7f133c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 -- 172.21.15.47:0/2025227976 >> v2:172.21.15.18:3300/0 conn(0x7f133c147a00 msgr2=0x7f133c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:22.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 --2- 172.21.15.47:0/2025227976 >> v2:172.21.15.18:3300/0 conn(0x7f133c147a00 0x7f133c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 -- 172.21.15.47:0/2025227976 >> v1:172.21.15.47:6789/0 conn(0x7f133c0ad100 legacy=0x7f133c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1341224700 1 -- 172.21.15.47:0/2025227976 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767393150 0 0) 0x7f133c147330 con 0x7f133c0ad100 2020-07-18T03:26:22.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 -- 172.21.15.47:0/2025227976 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0b5a80 con 0x7f133c1433e0 2020-07-18T03:26:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.449+0000 7f1342a27700 1 --2- 172.21.15.47:0/2025227976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c1433e0 0x7f133c143820 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f132c002f30 tx=0x7f132c006e90).ready entity=mon.1 client_cookie=e951edc35e5d867a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f1341224700 1 -- 172.21.15.47:0/2025227976 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c009850 con 0x7f133c1433e0 2020-07-18T03:26:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f1341224700 1 -- 172.21.15.47:0/2025227976 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c0099b0 con 0x7f133c1433e0 2020-07-18T03:26:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f1341224700 1 -- 172.21.15.47:0/2025227976 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c009d10 con 0x7f133c1433e0 2020-07-18T03:26:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 -- 172.21.15.47:0/2025227976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c1433e0 msgr2=0x7f133c143820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 --2- 172.21.15.47:0/2025227976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c1433e0 0x7f133c143820 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f132c002f30 tx=0x7f132c006e90).stop 2020-07-18T03:26:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 -- 172.21.15.47:0/2025227976 shutdown_connections 2020-07-18T03:26:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 --2- 172.21.15.47:0/2025227976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c1433e0 0x7f133c143820 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 --2- 172.21.15.47:0/2025227976 >> v2:172.21.15.18:3300/0 conn(0x7f133c147a00 0x7f133c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 -- 172.21.15.47:0/2025227976 >> 172.21.15.47:0/2025227976 conn(0x7f133c0b9340 msgr2=0x7f133c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 -- 172.21.15.47:0/2025227976 shutdown_connections 2020-07-18T03:26:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.450+0000 7f134448a700 1 -- 172.21.15.47:0/2025227976 wait complete. 2020-07-18T03:26:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.451+0000 7f134448a700 1 Processor -- start 2020-07-18T03:26:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.451+0000 7f134448a700 1 -- start start 2020-07-18T03:26:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f134448a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f133c0ad100 0x7f133c0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f134448a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f133c147a00 0x7f133c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f134448a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c14c840 con 0x7f133c0b30f0 2020-07-18T03:26:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f134448a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c14cae0 con 0x7f133c0ad100 2020-07-18T03:26:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f134448a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c14cd80 con 0x7f133c147a00 2020-07-18T03:26:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f133c0ad100 0x7f133c0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f133c0ad100 0x7f133c0b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59546/0 (socket says 172.21.15.47:59546) 2020-07-18T03:26:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 -- 172.21.15.47:0/2173718331 learned_addr learned my addr 172.21.15.47:0/2173718331 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1341a25700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f133c147a00 0x7f133c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 -- 172.21.15.47:0/2173718331 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f133c147a00 msgr2=0x7f133c14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f133c147a00 0x7f133c14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 -- 172.21.15.47:0/2173718331 >> v1:172.21.15.18:6789/0 conn(0x7f133c0b30f0 legacy=0x7f133c0660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 -- 172.21.15.47:0/2173718331 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c14d020 con 0x7f133c0ad100 2020-07-18T03:26:22.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f1342226700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c0ad100 0x7f133c0b2b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f1334000f00 tx=0x7f1334007d40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.452+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1334005ac0 con 0x7f133c0ad100 2020-07-18T03:26:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.453+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1334005c20 con 0x7f133c0ad100 2020-07-18T03:26:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.453+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c1421d0 con 0x7f133c0ad100 2020-07-18T03:26:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.453+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1334007110 con 0x7f133c0ad100 2020-07-18T03:26:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.453+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c1426f0 con 0x7f133c0ad100 2020-07-18T03:26:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.453+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c14d1b0 con 0x7f133c0ad100 2020-07-18T03:26:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.454+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1334010030 con 0x7f133c0ad100 2020-07-18T03:26:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.454+0000 7f133a7fc700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1314050010 0x7f13140524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.454+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1334067d90 con 0x7f133c0ad100 2020-07-18T03:26:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.454+0000 7f1341a25700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1314050010 0x7f13140524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.454+0000 7f1341a25700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1314050010 0x7f13140524d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f1330008000 tx=0x7f1330008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.458+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1334040520 con 0x7f133c0ad100 2020-07-18T03:26:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.615+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f133c057450 con 0x7f133c0ad100 2020-07-18T03:26:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.616+0000 7f133a7fc700 1 -- 172.21.15.47:0/2173718331 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f1334043b40 con 0x7f133c0ad100 2020-07-18T03:26:22.617 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:22.618 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:25:45.899129+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:25:41.055507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-07-18T03:25:43.561170+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1314050010 msgr2=0x7f13140524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1314050010 0x7f13140524d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f1330008000 tx=0x7f1330008040).stop 2020-07-18T03:26:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c0ad100 msgr2=0x7f133c0b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c0ad100 0x7f133c0b2b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f1334000f00 tx=0x7f1334007d40).stop 2020-07-18T03:26:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 shutdown_connections 2020-07-18T03:26:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1314050010 0x7f13140524d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f133c0ad100 0x7f133c0b2b00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 --2- 172.21.15.47:0/2173718331 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f133c147a00 0x7f133c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 >> 172.21.15.47:0/2173718331 conn(0x7f133c0b9340 msgr2=0x7f133c1469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 shutdown_connections 2020-07-18T03:26:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.618+0000 7f134448a700 1 -- 172.21.15.47:0/2173718331 wait complete. 2020-07-18T03:26:22.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:25:41.055507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:25:43.561170+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:22.632 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:22.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.749+0000 7f7d7e27e700 1 Processor -- start 2020-07-18T03:26:22.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d7e27e700 1 -- start start 2020-07-18T03:26:22.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d7e27e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d780b59b0 0x7f7d780b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d7e27e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d78147de0 con 0x7f7d780b6300 2020-07-18T03:26:22.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d7e27e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d780ae000 con 0x7f7d780ad330 2020-07-18T03:26:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d7e27e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d78148010 con 0x7f7d780b59b0 2020-07-18T03:26:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d777fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d780b59b0 0x7f7d780b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d77fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d780ad330 0x7f7d78143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40642/0 (socket says 172.21.15.47:40642) 2020-07-18T03:26:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.750+0000 7f7d77fff700 1 -- 172.21.15.47:0/4264844027 learned_addr learned my addr 172.21.15.47:0/4264844027 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:22.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d777fe700 1 -- 172.21.15.47:0/4264844027 >> v1:172.21.15.47:6789/0 conn(0x7f7d780ad330 legacy=0x7f7d78143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d777fe700 1 -- 172.21.15.47:0/4264844027 >> v1:172.21.15.18:6789/0 conn(0x7f7d780b6300 legacy=0x7f7d780b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d76ffd700 1 -- 172.21.15.47:0/4264844027 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996880062 0 0) 0x7f7d780ae000 con 0x7f7d780ad330 2020-07-18T03:26:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d777fe700 1 -- 172.21.15.47:0/4264844027 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780ad710 con 0x7f7d780b59b0 2020-07-18T03:26:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d777fe700 1 --2- 172.21.15.47:0/4264844027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780b59b0 0x7f7d780b5e10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f7d780ae4e0 tx=0x7f7d6c006dd0).ready entity=mon.1 client_cookie=ef0a09663bb9b0d0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d76ffd700 1 -- 172.21.15.47:0/4264844027 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d6c0099a0 con 0x7f7d780b59b0 2020-07-18T03:26:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d76ffd700 1 -- 172.21.15.47:0/4264844027 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c009b00 con 0x7f7d780b59b0 2020-07-18T03:26:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d76ffd700 1 -- 172.21.15.47:0/4264844027 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d6c009e60 con 0x7f7d780b59b0 2020-07-18T03:26:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 -- 172.21.15.47:0/4264844027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780b59b0 msgr2=0x7f7d780b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 --2- 172.21.15.47:0/4264844027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780b59b0 0x7f7d780b5e10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f7d780ae4e0 tx=0x7f7d6c006dd0).stop 2020-07-18T03:26:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 -- 172.21.15.47:0/4264844027 shutdown_connections 2020-07-18T03:26:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 --2- 172.21.15.47:0/4264844027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780b59b0 0x7f7d780b5e10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 -- 172.21.15.47:0/4264844027 >> 172.21.15.47:0/4264844027 conn(0x7f7d7805f5a0 msgr2=0x7f7d7805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 -- 172.21.15.47:0/4264844027 shutdown_connections 2020-07-18T03:26:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.751+0000 7f7d7e27e700 1 -- 172.21.15.47:0/4264844027 wait complete. 2020-07-18T03:26:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.753+0000 7f7d7e27e700 1 Processor -- start 2020-07-18T03:26:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.753+0000 7f7d7e27e700 1 -- start start 2020-07-18T03:26:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.753+0000 7f7d7e27e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d780639e0 0x7f7d78141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.753+0000 7f7d7e27e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d78063000 con 0x7f7d780ad330 2020-07-18T03:26:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.753+0000 7f7d7e27e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d780632a0 con 0x7f7d780b6300 2020-07-18T03:26:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.753+0000 7f7d7e27e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d78063540 con 0x7f7d780639e0 2020-07-18T03:26:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d777fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d780b6300 0x7f7d78065fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40650/0 (socket says 172.21.15.47:40650) 2020-07-18T03:26:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d777fe700 1 -- 172.21.15.47:0/2288431853 learned_addr learned my addr 172.21.15.47:0/2288431853 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7c81b700 1 --2- 172.21.15.47:0/2288431853 >> v2:172.21.15.47:3300/0 conn(0x7f7d780639e0 0x7f7d78141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478115505 0 0) 0x7f7d780632a0 con 0x7f7d780b6300 2020-07-18T03:26:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d50003240 con 0x7f7d780b6300 2020-07-18T03:26:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7c81b700 1 -- 172.21.15.47:0/2288431853 >> v1:172.21.15.47:6789/0 conn(0x7f7d780b6300 legacy=0x7f7d78065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7c81b700 1 -- 172.21.15.47:0/2288431853 >> v1:172.21.15.18:6789/0 conn(0x7f7d780ad330 legacy=0x7f7d780b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7c81b700 1 -- 172.21.15.47:0/2288431853 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78066980 con 0x7f7d780639e0 2020-07-18T03:26:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7c81b700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780639e0 0x7f7d78141be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f7d780b2910 tx=0x7f7d70007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d70010510 con 0x7f7d780639e0 2020-07-18T03:26:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d70010670 con 0x7f7d780639e0 2020-07-18T03:26:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d700107e0 con 0x7f7d780639e0 2020-07-18T03:26:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d78063e50 con 0x7f7d780639e0 2020-07-18T03:26:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.754+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d78142310 con 0x7f7d780639e0 2020-07-18T03:26:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.755+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7805b120 con 0x7f7d780639e0 2020-07-18T03:26:22.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.755+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7d70010070 con 0x7f7d780639e0 2020-07-18T03:26:22.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.756+0000 7f7d57fff700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d500503a0 0x7f7d50052860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:22.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.756+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7d700676b0 con 0x7f7d780639e0 2020-07-18T03:26:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.756+0000 7f7d77fff700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d500503a0 0x7f7d50052860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.756+0000 7f7d77fff700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d500503a0 0x7f7d50052860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f7d68005510 tx=0x7f7d680076a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.759+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7d7003fe40 con 0x7f7d780639e0 2020-07-18T03:26:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.914+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7d780690d0 con 0x7f7d780639e0 2020-07-18T03:26:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.914+0000 7f7d57fff700 1 -- 172.21.15.47:0/2288431853 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7d70043460 con 0x7f7d780639e0 2020-07-18T03:26:22.916 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d500503a0 msgr2=0x7f7d50052860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d500503a0 0x7f7d50052860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f7d68005510 tx=0x7f7d680076a0).stop 2020-07-18T03:26:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780639e0 msgr2=0x7f7d78141be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780639e0 0x7f7d78141be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f7d780b2910 tx=0x7f7d70007e40).stop 2020-07-18T03:26:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 shutdown_connections 2020-07-18T03:26:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d780639e0 0x7f7d78141be0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 --2- 172.21.15.47:0/2288431853 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d500503a0 0x7f7d50052860 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:22.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 >> 172.21.15.47:0/2288431853 conn(0x7f7d7805f5a0 msgr2=0x7f7d780b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:22.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 shutdown_connections 2020-07-18T03:26:22.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:22.916+0000 7f7d7e27e700 1 -- 172.21.15.47:0/2288431853 wait complete. 2020-07-18T03:26:22.928 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.045+0000 7f5299cd0700 1 Processor -- start 2020-07-18T03:26:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.045+0000 7f5299cd0700 1 -- start start 2020-07-18T03:26:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5294144e70 0x7f52940b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5294142430 0x7f5294148b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5294061db0 con 0x7f5294141fb0 2020-07-18T03:26:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52940b14c0 con 0x7f5294142430 2020-07-18T03:26:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52941428c0 con 0x7f5294144e70 2020-07-18T03:26:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5294144e70 0x7f52940b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5294144e70 0x7f52940b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59568/0 (socket says 172.21.15.47:59568) 2020-07-18T03:26:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 -- 172.21.15.47:0/538727328 learned_addr learned my addr 172.21.15.47:0/538727328 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5293fff700 1 --2- 172.21.15.47:0/538727328 >> v2:172.21.15.18:3300/0 conn(0x7f5294142430 0x7f5294148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 -- 172.21.15.47:0/538727328 >> v2:172.21.15.18:3300/0 conn(0x7f5294142430 msgr2=0x7f5294148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 --2- 172.21.15.47:0/538727328 >> v2:172.21.15.18:3300/0 conn(0x7f5294142430 0x7f5294148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 -- 172.21.15.47:0/538727328 >> v1:172.21.15.18:6789/0 conn(0x7f5294141fb0 legacy=0x7f5294148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 -- 172.21.15.47:0/538727328 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5294062050 con 0x7f5294144e70 2020-07-18T03:26:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52937fe700 1 --2- 172.21.15.47:0/538727328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294144e70 0x7f52940b5f60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f5284002f30 tx=0x7f5284006e90).ready entity=mon.1 client_cookie=1341f820530e477 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52927fc700 1 -- 172.21.15.47:0/538727328 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52840099c0 con 0x7f5294144e70 2020-07-18T03:26:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52927fc700 1 -- 172.21.15.47:0/538727328 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284009b20 con 0x7f5294144e70 2020-07-18T03:26:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f52927fc700 1 -- 172.21.15.47:0/538727328 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5284009e80 con 0x7f5294144e70 2020-07-18T03:26:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 -- 172.21.15.47:0/538727328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294144e70 msgr2=0x7f52940b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.046+0000 7f5299cd0700 1 --2- 172.21.15.47:0/538727328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294144e70 0x7f52940b5f60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f5284002f30 tx=0x7f5284006e90).stop 2020-07-18T03:26:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.047+0000 7f5299cd0700 1 -- 172.21.15.47:0/538727328 shutdown_connections 2020-07-18T03:26:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.047+0000 7f5299cd0700 1 --2- 172.21.15.47:0/538727328 >> v2:172.21.15.18:3300/0 conn(0x7f5294142430 0x7f5294148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.047+0000 7f5299cd0700 1 --2- 172.21.15.47:0/538727328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294144e70 0x7f52940b5f60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.047+0000 7f5299cd0700 1 -- 172.21.15.47:0/538727328 >> 172.21.15.47:0/538727328 conn(0x7f529405f5a0 msgr2=0x7f529405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.047+0000 7f5299cd0700 1 -- 172.21.15.47:0/538727328 shutdown_connections 2020-07-18T03:26:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.047+0000 7f5299cd0700 1 -- 172.21.15.47:0/538727328 wait complete. 2020-07-18T03:26:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.048+0000 7f5299cd0700 1 Processor -- start 2020-07-18T03:26:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.048+0000 7f5299cd0700 1 -- start start 2020-07-18T03:26:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5299cd0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5294141fb0 0x7f529412e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5299cd0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5294142430 0x7f5294155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5299cd0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5294146000 con 0x7f52941483a0 2020-07-18T03:26:23.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5299cd0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52941462a0 con 0x7f5294142430 2020-07-18T03:26:23.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5299cd0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5294147750 con 0x7f5294141fb0 2020-07-18T03:26:23.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5294142430 0x7f5294155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5294142430 0x7f5294155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59576/0 (socket says 172.21.15.47:59576) 2020-07-18T03:26:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 -- 172.21.15.47:0/1487467701 learned_addr learned my addr 172.21.15.47:0/1487467701 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f52937fe700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5294141fb0 0x7f529412e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 -- 172.21.15.47:0/1487467701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5294141fb0 msgr2=0x7f529412e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5294141fb0 0x7f529412e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 -- 172.21.15.47:0/1487467701 >> v1:172.21.15.18:6789/0 conn(0x7f52941483a0 legacy=0x7f52941548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 -- 172.21.15.47:0/1487467701 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940b3a10 con 0x7f5294142430 2020-07-18T03:26:23.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5292ffd700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294142430 0x7f5294155580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f5294062670 tx=0x7f5288007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f528800e4a0 con 0x7f5294142430 2020-07-18T03:26:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528800e600 con 0x7f5294142430 2020-07-18T03:26:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.049+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529412eca0 con 0x7f5294142430 2020-07-18T03:26:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.050+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f528800e960 con 0x7f5294142430 2020-07-18T03:26:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.050+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5294154fc0 con 0x7f5294142430 2020-07-18T03:26:23.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.051+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529405b120 con 0x7f5294142430 2020-07-18T03:26:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.051+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5288019030 con 0x7f5294142430 2020-07-18T03:26:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.051+0000 7f52737fe700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f526c04ffa0 0x7f526c052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.051+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5288067090 con 0x7f5294142430 2020-07-18T03:26:23.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.051+0000 7f52937fe700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f526c04ffa0 0x7f526c052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.051+0000 7f52937fe700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f526c04ffa0 0x7f526c052460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f528401c000 tx=0x7f5284015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.054+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f528803f820 con 0x7f5294142430 2020-07-18T03:26:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.211+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f529412ee30 con 0x7f5294142430 2020-07-18T03:26:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.211+0000 7f52737fe700 1 -- 172.21.15.47:0/1487467701 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52880170a0 con 0x7f5294142430 2020-07-18T03:26:23.213 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:23.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f526c04ffa0 msgr2=0x7f526c052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f526c04ffa0 0x7f526c052460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f528401c000 tx=0x7f5284015000).stop 2020-07-18T03:26:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294142430 msgr2=0x7f5294155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294142430 0x7f5294155580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f5294062670 tx=0x7f5288007c30).stop 2020-07-18T03:26:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 shutdown_connections 2020-07-18T03:26:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f526c04ffa0 0x7f526c052460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5294141fb0 0x7f529412e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 --2- 172.21.15.47:0/1487467701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5294142430 0x7f5294155580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 >> 172.21.15.47:0/1487467701 conn(0x7f529405f5a0 msgr2=0x7f52940b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 shutdown_connections 2020-07-18T03:26:23.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.213+0000 7f5299cd0700 1 -- 172.21.15.47:0/1487467701 wait complete. 2020-07-18T03:26:23.227 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.345+0000 7f2088b71700 1 Processor -- start 2020-07-18T03:26:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.345+0000 7f2088b71700 1 -- start start 2020-07-18T03:26:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2088b71700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2084061c60 0x7f20840620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2088b71700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f20840b3af0 0x7f20840b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2088b71700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20840af4b0 con 0x7f2084142a20 2020-07-18T03:26:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2088b71700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20840b6d80 con 0x7f20840b3af0 2020-07-18T03:26:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2088b71700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20840af710 con 0x7f2084061c60 2020-07-18T03:26:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2084061c60 0x7f20840620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2084061c60 0x7f20840620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59584/0 (socket says 172.21.15.47:59584) 2020-07-18T03:26:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 -- 172.21.15.47:0/1287934030 learned_addr learned my addr 172.21.15.47:0/1287934030 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2082d9d700 1 --2- 172.21.15.47:0/1287934030 >> v2:172.21.15.18:3300/0 conn(0x7f20840b3af0 0x7f20840b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 -- 172.21.15.47:0/1287934030 >> v2:172.21.15.18:3300/0 conn(0x7f20840b3af0 msgr2=0x7f20840b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 --2- 172.21.15.47:0/1287934030 >> v2:172.21.15.18:3300/0 conn(0x7f20840b3af0 0x7f20840b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 -- 172.21.15.47:0/1287934030 >> v1:172.21.15.18:6789/0 conn(0x7f2084142a20 legacy=0x7f2084142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 -- 172.21.15.47:0/1287934030 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20840af9b0 con 0x7f2084061c60 2020-07-18T03:26:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.346+0000 7f2081d9b700 1 --2- 172.21.15.47:0/1287934030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2084061c60 0x7f20840620f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f208405d1a0 tx=0x7f2074006e90).ready entity=mon.1 client_cookie=bd35cfaa3ad5160f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f208159a700 1 -- 172.21.15.47:0/1287934030 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2074009930 con 0x7f2084061c60 2020-07-18T03:26:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f208159a700 1 -- 172.21.15.47:0/1287934030 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2074009a90 con 0x7f2084061c60 2020-07-18T03:26:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f208159a700 1 -- 172.21.15.47:0/1287934030 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2074009e30 con 0x7f2084061c60 2020-07-18T03:26:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 -- 172.21.15.47:0/1287934030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2084061c60 msgr2=0x7f20840620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 --2- 172.21.15.47:0/1287934030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2084061c60 0x7f20840620f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f208405d1a0 tx=0x7f2074006e90).stop 2020-07-18T03:26:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 -- 172.21.15.47:0/1287934030 shutdown_connections 2020-07-18T03:26:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 --2- 172.21.15.47:0/1287934030 >> v2:172.21.15.18:3300/0 conn(0x7f20840b3af0 0x7f20840b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 --2- 172.21.15.47:0/1287934030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2084061c60 0x7f20840620f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 -- 172.21.15.47:0/1287934030 >> 172.21.15.47:0/1287934030 conn(0x7f208405f5a0 msgr2=0x7f208405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 -- 172.21.15.47:0/1287934030 shutdown_connections 2020-07-18T03:26:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.347+0000 7f2088b71700 1 -- 172.21.15.47:0/1287934030 wait complete. 2020-07-18T03:26:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.348+0000 7f2088b71700 1 Processor -- start 2020-07-18T03:26:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.348+0000 7f2088b71700 1 -- start start 2020-07-18T03:26:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f2088b71700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20840b3af0 0x7f208413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f2088b71700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2084159540 con 0x7f2084142a20 2020-07-18T03:26:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f2088b71700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20841131f0 con 0x7f20840b2fe0 2020-07-18T03:26:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f2088b71700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2084113490 con 0x7f20840b3af0 2020-07-18T03:26:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f208259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20840b3af0 0x7f208413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f2082d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f20840b2fe0 0x7f2084158e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40680/0 (socket says 172.21.15.47:40680) 2020-07-18T03:26:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f2082d9d700 1 -- 172.21.15.47:0/3038389435 learned_addr learned my addr 172.21.15.47:0/3038389435 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f208259c700 1 --2- 172.21.15.47:0/3038389435 >> v2:172.21.15.47:3300/0 conn(0x7f20840b3af0 0x7f208413d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59588/0 (socket says 172.21.15.47:59588) 2020-07-18T03:26:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f208259c700 1 -- 172.21.15.47:0/3038389435 >> v1:172.21.15.47:6789/0 conn(0x7f20840b2fe0 legacy=0x7f2084158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f208259c700 1 -- 172.21.15.47:0/3038389435 >> v1:172.21.15.18:6789/0 conn(0x7f2084142a20 legacy=0x7f2084158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f208259c700 1 -- 172.21.15.47:0/3038389435 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084113730 con 0x7f20840b3af0 2020-07-18T03:26:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000919599 0 0) 0x7f20841131f0 con 0x7f20840b2fe0 2020-07-18T03:26:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.349+0000 7f208259c700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b3af0 0x7f208413d6e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f2078001f50 tx=0x7f2078007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.350+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207800e4a0 con 0x7f20840b3af0 2020-07-18T03:26:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.350+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207800e600 con 0x7f20840b3af0 2020-07-18T03:26:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.350+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208413dbd0 con 0x7f20840b3af0 2020-07-18T03:26:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.350+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207800e770 con 0x7f20840b3af0 2020-07-18T03:26:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.350+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20840b1900 con 0x7f20840b3af0 2020-07-18T03:26:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.351+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208405b120 con 0x7f20840b3af0 2020-07-18T03:26:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.351+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2078010030 con 0x7f20840b3af0 2020-07-18T03:26:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.351+0000 7f20727fc700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f205804fe70 0x7f2058052330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.351+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f20780678e0 con 0x7f20840b3af0 2020-07-18T03:26:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.351+0000 7f2081d9b700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f205804fe70 0x7f2058052330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.352+0000 7f2081d9b700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f205804fe70 0x7f2058052330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f2084113110 tx=0x7f2074015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.355+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2078040070 con 0x7f20840b3af0 2020-07-18T03:26:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.520+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20840690d0 con 0x7f20840b3af0 2020-07-18T03:26:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.520+0000 7f20727fc700 1 -- 172.21.15.47:0/3038389435 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2078043690 con 0x7f20840b3af0 2020-07-18T03:26:23.522 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f205804fe70 msgr2=0x7f2058052330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f205804fe70 0x7f2058052330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f2084113110 tx=0x7f2074015000).stop 2020-07-18T03:26:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b3af0 msgr2=0x7f208413d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b3af0 0x7f208413d6e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f2078001f50 tx=0x7f2078007ac0).stop 2020-07-18T03:26:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 shutdown_connections 2020-07-18T03:26:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f205804fe70 0x7f2058052330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 --2- 172.21.15.47:0/3038389435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b3af0 0x7f208413d6e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 >> 172.21.15.47:0/3038389435 conn(0x7f208405f5a0 msgr2=0x7f2084141b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 shutdown_connections 2020-07-18T03:26:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.522+0000 7f2088b71700 1 -- 172.21.15.47:0/3038389435 wait complete. 2020-07-18T03:26:23.533 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:23.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.650+0000 7feebd249700 1 Processor -- start 2020-07-18T03:26:23.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.650+0000 7feebd249700 1 -- start start 2020-07-18T03:26:23.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feeb80ad330 0x7feeb8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feeb80b59b0 0x7feeb80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb80b6300 0x7feeb80b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb8147de0 con 0x7feeb80b59b0 2020-07-18T03:26:23.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80ae000 con 0x7feeb80b6300 2020-07-18T03:26:23.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb8148010 con 0x7feeb80ad330 2020-07-18T03:26:23.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb80b6300 0x7feeb80b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb80b6300 0x7feeb80b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59600/0 (socket says 172.21.15.47:59600) 2020-07-18T03:26:23.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 -- 172.21.15.47:0/769584481 learned_addr learned my addr 172.21.15.47:0/769584481 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:23.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb659c700 1 --2- 172.21.15.47:0/769584481 >> v2:172.21.15.18:3300/0 conn(0x7feeb80b59b0 0x7feeb80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb6d9d700 1 --2- 172.21.15.47:0/769584481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feeb80ad330 0x7feeb8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 -- 172.21.15.47:0/769584481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feeb80ad330 msgr2=0x7feeb8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 --2- 172.21.15.47:0/769584481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feeb80ad330 0x7feeb8143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 -- 172.21.15.47:0/769584481 >> v2:172.21.15.18:3300/0 conn(0x7feeb80b59b0 msgr2=0x7feeb80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 --2- 172.21.15.47:0/769584481 >> v2:172.21.15.18:3300/0 conn(0x7feeb80b59b0 0x7feeb80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 -- 172.21.15.47:0/769584481 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80ad710 con 0x7feeb80b6300 2020-07-18T03:26:23.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb759e700 1 --2- 172.21.15.47:0/769584481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb80b6300 0x7feeb80b3a10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7feea8002f30 tx=0x7feea8006e90).ready entity=mon.1 client_cookie=c8f6a85e4f7823cc server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb5d9b700 1 -- 172.21.15.47:0/769584481 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea8009730 con 0x7feeb80b6300 2020-07-18T03:26:23.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb5d9b700 1 -- 172.21.15.47:0/769584481 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8009890 con 0x7feeb80b6300 2020-07-18T03:26:23.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feeb5d9b700 1 -- 172.21.15.47:0/769584481 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea8009bf0 con 0x7feeb80b6300 2020-07-18T03:26:23.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 -- 172.21.15.47:0/769584481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb80b6300 msgr2=0x7feeb80b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- 172.21.15.47:0/769584481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb80b6300 0x7feeb80b3a10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7feea8002f30 tx=0x7feea8006e90).stop 2020-07-18T03:26:23.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 -- 172.21.15.47:0/769584481 shutdown_connections 2020-07-18T03:26:23.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- 172.21.15.47:0/769584481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb80b6300 0x7feeb80b3a10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- 172.21.15.47:0/769584481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7feeb80ad330 0x7feeb8143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 --2- 172.21.15.47:0/769584481 >> v2:172.21.15.18:3300/0 conn(0x7feeb80b59b0 0x7feeb80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.651+0000 7feebd249700 1 -- 172.21.15.47:0/769584481 >> 172.21.15.47:0/769584481 conn(0x7feeb805f5a0 msgr2=0x7feeb805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.652+0000 7feebd249700 1 -- 172.21.15.47:0/769584481 shutdown_connections 2020-07-18T03:26:23.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.652+0000 7feebd249700 1 -- 172.21.15.47:0/769584481 wait complete. 2020-07-18T03:26:23.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.653+0000 7feebd249700 1 Processor -- start 2020-07-18T03:26:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.653+0000 7feebd249700 1 -- start start 2020-07-18T03:26:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feebd249700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feeb80ad330 0x7feeb814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feebd249700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feeb814a9d0 0x7feeb80b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feebd249700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb8148210 con 0x7feeb80b59b0 2020-07-18T03:26:23.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feebd249700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb8062f20 con 0x7feeb80ad330 2020-07-18T03:26:23.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feebd249700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80631c0 con 0x7feeb814a9d0 2020-07-18T03:26:23.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feeb80b59b0 0x7feeb813a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40692/0 (socket says 172.21.15.47:40692) 2020-07-18T03:26:23.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb659c700 1 -- 172.21.15.47:0/726670960 learned_addr learned my addr 172.21.15.47:0/726670960 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:23.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb759e700 1 --2- 172.21.15.47:0/726670960 >> v2:172.21.15.47:3300/0 conn(0x7feeb814a9d0 0x7feeb80b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb6d9d700 1 --2- 172.21.15.47:0/726670960 >> v2:172.21.15.18:3300/0 conn(0x7feeb80ad330 0x7feeb814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb759e700 1 -- 172.21.15.47:0/726670960 >> v2:172.21.15.18:3300/0 conn(0x7feeb80ad330 msgr2=0x7feeb814ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb759e700 1 --2- 172.21.15.47:0/726670960 >> v2:172.21.15.18:3300/0 conn(0x7feeb80ad330 0x7feeb814ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb759e700 1 -- 172.21.15.47:0/726670960 >> v1:172.21.15.47:6789/0 conn(0x7feeb80b59b0 legacy=0x7feeb813a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb759e700 1 -- 172.21.15.47:0/726670960 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8148470 con 0x7feeb814a9d0 2020-07-18T03:26:23.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901932098 0 0) 0x7feeb8148210 con 0x7feeb80b59b0 2020-07-18T03:26:23.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeb759e700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb814a9d0 0x7feeb80b1c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7feeb8064200 tx=0x7feea8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea8011e40 con 0x7feeb814a9d0 2020-07-18T03:26:23.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8009e20 con 0x7feeb814a9d0 2020-07-18T03:26:23.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.654+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb80b2100 con 0x7feeb814a9d0 2020-07-18T03:26:23.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.655+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feea801d7c0 con 0x7feeb814a9d0 2020-07-18T03:26:23.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.655+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb8148910 con 0x7feeb814a9d0 2020-07-18T03:26:23.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.655+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb8148600 con 0x7feeb814a9d0 2020-07-18T03:26:23.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.656+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7feea8025030 con 0x7feeb814a9d0 2020-07-18T03:26:23.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.656+0000 7feeaeffd700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fee9404fe40 0x7fee94052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.656+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feea80691f0 con 0x7feeb814a9d0 2020-07-18T03:26:23.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.656+0000 7feeb6d9d700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fee9404fe40 0x7fee94052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.656+0000 7feeb6d9d700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fee9404fe40 0x7fee94052300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7feeb000b000 tx=0x7feeb00091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.659+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7feea8041900 con 0x7feeb814a9d0 2020-07-18T03:26:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.817+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feeb805b120 con 0x7feeb814a9d0 2020-07-18T03:26:23.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.817+0000 7feeaeffd700 1 -- 172.21.15.47:0/726670960 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7feea8044f20 con 0x7feeb814a9d0 2020-07-18T03:26:23.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fee9404fe40 msgr2=0x7fee94052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fee9404fe40 0x7fee94052300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7feeb000b000 tx=0x7feeb00091d0).stop 2020-07-18T03:26:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb814a9d0 msgr2=0x7feeb80b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb814a9d0 0x7feeb80b1c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7feeb8064200 tx=0x7feea8015000).stop 2020-07-18T03:26:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 shutdown_connections 2020-07-18T03:26:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feeb814a9d0 0x7feeb80b1c10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 --2- 172.21.15.47:0/726670960 >> v2:172.21.15.18:3300/0 conn(0x7feeb80ad330 0x7feeb814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 --2- 172.21.15.47:0/726670960 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fee9404fe40 0x7fee94052300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 >> 172.21.15.47:0/726670960 conn(0x7feeb805f5a0 msgr2=0x7feeb80afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 shutdown_connections 2020-07-18T03:26:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.819+0000 7feebd249700 1 -- 172.21.15.47:0/726670960 wait complete. 2020-07-18T03:26:23.826 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:23.848 INFO:tasks.cephfs_test_runner:Starting test: test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) 2020-07-18T03:26:23.850 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death' 2020-07-18T03:26:23.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.965+0000 7f780a420700 1 Processor -- start 2020-07-18T03:26:23.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.965+0000 7f780a420700 1 -- start start 2020-07-18T03:26:23.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f780a420700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f78040ad100 0x7f78040b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f780a420700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7804147a00 0x7f7804147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f780a420700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7804147330 con 0x7f780405f5a0 2020-07-18T03:26:23.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f780a420700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78040ae530 con 0x7f78040ad100 2020-07-18T03:26:23.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f780a420700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78040b57e0 con 0x7f7804147a00 2020-07-18T03:26:23.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7804147a00 0x7f7804147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7804147a00 0x7f7804147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59610/0 (socket says 172.21.15.47:59610) 2020-07-18T03:26:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 -- 172.21.15.47:0/476033971 learned_addr learned my addr 172.21.15.47:0/476033971 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f7803fff700 1 --2- 172.21.15.47:0/476033971 >> v2:172.21.15.18:3300/0 conn(0x7f78040ad100 0x7f78040b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 -- 172.21.15.47:0/476033971 >> v2:172.21.15.18:3300/0 conn(0x7f78040ad100 msgr2=0x7f78040b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 --2- 172.21.15.47:0/476033971 >> v2:172.21.15.18:3300/0 conn(0x7f78040ad100 0x7f78040b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 -- 172.21.15.47:0/476033971 >> v1:172.21.15.18:6789/0 conn(0x7f780405f5a0 legacy=0x7f780405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 -- 172.21.15.47:0/476033971 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78040b5a80 con 0x7f7804147a00 2020-07-18T03:26:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f7803fff700 1 --2- 172.21.15.47:0/476033971 >> v2:172.21.15.18:3300/0 conn(0x7f78040ad100 0x7f78040b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.966+0000 7f78037fe700 1 --2- 172.21.15.47:0/476033971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7804147a00 0x7f7804147e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f780405ca30 tx=0x7f77f8006e90).ready entity=mon.1 client_cookie=e903c8824ab63cba server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f7802ffd700 1 -- 172.21.15.47:0/476033971 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77f8009930 con 0x7f7804147a00 2020-07-18T03:26:23.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f7802ffd700 1 -- 172.21.15.47:0/476033971 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f8009a90 con 0x7f7804147a00 2020-07-18T03:26:23.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f7802ffd700 1 -- 172.21.15.47:0/476033971 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77f8009e30 con 0x7f7804147a00 2020-07-18T03:26:23.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 -- 172.21.15.47:0/476033971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7804147a00 msgr2=0x7f7804147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 --2- 172.21.15.47:0/476033971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7804147a00 0x7f7804147e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f780405ca30 tx=0x7f77f8006e90).stop 2020-07-18T03:26:23.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 -- 172.21.15.47:0/476033971 shutdown_connections 2020-07-18T03:26:23.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 --2- 172.21.15.47:0/476033971 >> v2:172.21.15.18:3300/0 conn(0x7f78040ad100 0x7f78040b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 --2- 172.21.15.47:0/476033971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7804147a00 0x7f7804147e40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:23.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 -- 172.21.15.47:0/476033971 >> 172.21.15.47:0/476033971 conn(0x7f78040b9340 msgr2=0x7f78040b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:23.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 -- 172.21.15.47:0/476033971 shutdown_connections 2020-07-18T03:26:23.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.967+0000 7f780a420700 1 -- 172.21.15.47:0/476033971 wait complete. 2020-07-18T03:26:23.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.968+0000 7f780a420700 1 Processor -- start 2020-07-18T03:26:23.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.968+0000 7f780a420700 1 -- start start 2020-07-18T03:26:23.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f780a420700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78040ad100 0x7f7804062590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f780a420700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7804061bb0 con 0x7f780405f5a0 2020-07-18T03:26:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f780a420700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7804061e50 con 0x7f7804065f70 2020-07-18T03:26:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f780a420700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78040620f0 con 0x7f78040ad100 2020-07-18T03:26:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f78037fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78040ad100 0x7f7804062590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f78037fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78040ad100 0x7f7804062590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59618/0 (socket says 172.21.15.47:59618) 2020-07-18T03:26:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f78037fe700 1 -- 172.21.15.47:0/1072891792 learned_addr learned my addr 172.21.15.47:0/1072891792 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.969+0000 7f78089bd700 1 --1- 172.21.15.47:0/1072891792 >> v1:172.21.15.47:6789/0 conn(0x7f7804065f70 0x7f780413a220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40708/0 (socket says 172.21.15.47:40708) 2020-07-18T03:26:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f78037fe700 1 -- 172.21.15.47:0/1072891792 >> v1:172.21.15.47:6789/0 conn(0x7f7804065f70 legacy=0x7f780413a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f78037fe700 1 -- 172.21.15.47:0/1072891792 >> v1:172.21.15.18:6789/0 conn(0x7f780405f5a0 legacy=0x7f78040b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714037953 0 0) 0x7f7804061e50 con 0x7f7804065f70 2020-07-18T03:26:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f78037fe700 1 -- 172.21.15.47:0/1072891792 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780413a930 con 0x7f78040ad100 2020-07-18T03:26:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f78037fe700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78040ad100 0x7f7804062590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f780405ca30 tx=0x7f77f8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77f801d5d0 con 0x7f78040ad100 2020-07-18T03:26:23.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7804062cb0 con 0x7f78040ad100 2020-07-18T03:26:23.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f801d730 con 0x7f78040ad100 2020-07-18T03:26:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77f801da90 con 0x7f78040ad100 2020-07-18T03:26:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.970+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78040630c0 con 0x7f78040ad100 2020-07-18T03:26:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.971+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78040aec80 con 0x7f78040ad100 2020-07-18T03:26:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.971+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f77f8025030 con 0x7f78040ad100 2020-07-18T03:26:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.971+0000 7f77e3fff700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77dc04fe10 0x7f77dc0522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.971+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f77f80696d0 con 0x7f78040ad100 2020-07-18T03:26:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.971+0000 7f7803fff700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77dc04fe10 0x7f77dc0522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.972+0000 7f7803fff700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77dc04fe10 0x7f77dc0522d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f78040b34f0 tx=0x7f77f4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:23.975+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f77f8041de0 con 0x7f78040ad100 2020-07-18T03:26:24.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.192+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]} v 0) v1 -- 0x7f7804057450 con 0x7f78040ad100 2020-07-18T03:26:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.222+0000 7f77e3fff700 1 -- 172.21.15.47:0/1072891792 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]}]=0 v56) v1 ==== 160+0+0 (secure 0 0 0) 0x7f77f8045400 con 0x7f78040ad100 2020-07-18T03:26:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77dc04fe10 msgr2=0x7f77dc0522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77dc04fe10 0x7f77dc0522d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f78040b34f0 tx=0x7f77f4008040).stop 2020-07-18T03:26:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78040ad100 msgr2=0x7f7804062590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78040ad100 0x7f7804062590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f780405ca30 tx=0x7f77f8015000).stop 2020-07-18T03:26:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 shutdown_connections 2020-07-18T03:26:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f77dc04fe10 0x7f77dc0522d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 --2- 172.21.15.47:0/1072891792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78040ad100 0x7f7804062590 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 >> 172.21.15.47:0/1072891792 conn(0x7f78040b9340 msgr2=0x7f78041431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 shutdown_connections 2020-07-18T03:26:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.223+0000 7f780a420700 1 -- 172.21.15.47:0/1072891792 wait complete. 2020-07-18T03:26:24.234 INFO:tasks.cephfs_test_runner:test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) ... skipped 'Require first client to be on separate server from MDSs' 2020-07-18T03:26:24.235 INFO:tasks.cephfs_test_runner:Starting test: test_abort_conn (tasks.cephfs.test_client_recovery.TestClientRecovery) 2020-07-18T03:26:24.236 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn' 2020-07-18T03:26:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.350+0000 7f7e3a536700 1 Processor -- start 2020-07-18T03:26:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.350+0000 7f7e3a536700 1 -- start start 2020-07-18T03:26:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e3a536700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7e340ad100 0x7f7e340b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e3a536700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e34147a00 0x7f7e34147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e3a536700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e34147330 con 0x7f7e3405f5a0 2020-07-18T03:26:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e3a536700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340ae530 con 0x7f7e340ad100 2020-07-18T03:26:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e3a536700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340b57e0 con 0x7f7e34147a00 2020-07-18T03:26:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e34147a00 0x7f7e34147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e38ad3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e3405f5a0 0x7f7e3405f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40714/0 (socket says 172.21.15.47:40714) 2020-07-18T03:26:24.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e34147a00 0x7f7e34147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59626/0 (socket says 172.21.15.47:59626) 2020-07-18T03:26:24.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e38ad3700 1 -- 172.21.15.47:0/923267763 learned_addr learned my addr 172.21.15.47:0/923267763 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:24.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e33fff700 1 --2- 172.21.15.47:0/923267763 >> v2:172.21.15.18:3300/0 conn(0x7f7e340ad100 0x7f7e340b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:24.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 -- 172.21.15.47:0/923267763 >> v2:172.21.15.18:3300/0 conn(0x7f7e340ad100 msgr2=0x7f7e340b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:24.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 --2- 172.21.15.47:0/923267763 >> v2:172.21.15.18:3300/0 conn(0x7f7e340ad100 0x7f7e340b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:24.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 -- 172.21.15.47:0/923267763 >> v1:172.21.15.47:6789/0 conn(0x7f7e3405f5a0 legacy=0x7f7e3405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:24.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 -- 172.21.15.47:0/923267763 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e340b5a80 con 0x7f7e34147a00 2020-07-18T03:26:24.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e33fff700 1 --2- 172.21.15.47:0/923267763 >> v2:172.21.15.18:3300/0 conn(0x7f7e340ad100 0x7f7e340b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:24.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.351+0000 7f7e337fe700 1 --2- 172.21.15.47:0/923267763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e34147a00 0x7f7e34147e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f7e28002f30 tx=0x7f7e28006dd0).ready entity=mon.1 client_cookie=29ac86bf54d35e5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:24.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e32ffd700 1 -- 172.21.15.47:0/923267763 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e280097f0 con 0x7f7e34147a00 2020-07-18T03:26:24.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e32ffd700 1 -- 172.21.15.47:0/923267763 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28009950 con 0x7f7e34147a00 2020-07-18T03:26:24.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e32ffd700 1 -- 172.21.15.47:0/923267763 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e28009cb0 con 0x7f7e34147a00 2020-07-18T03:26:24.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 -- 172.21.15.47:0/923267763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e34147a00 msgr2=0x7f7e34147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:24.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 --2- 172.21.15.47:0/923267763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e34147a00 0x7f7e34147e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f7e28002f30 tx=0x7f7e28006dd0).stop 2020-07-18T03:26:24.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 -- 172.21.15.47:0/923267763 shutdown_connections 2020-07-18T03:26:24.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 --2- 172.21.15.47:0/923267763 >> v2:172.21.15.18:3300/0 conn(0x7f7e340ad100 0x7f7e340b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:24.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 --2- 172.21.15.47:0/923267763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e34147a00 0x7f7e34147e40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:24.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 -- 172.21.15.47:0/923267763 >> 172.21.15.47:0/923267763 conn(0x7f7e340b9340 msgr2=0x7f7e340b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:24.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 -- 172.21.15.47:0/923267763 shutdown_connections 2020-07-18T03:26:24.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.352+0000 7f7e3a536700 1 -- 172.21.15.47:0/923267763 wait complete. 2020-07-18T03:26:24.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.353+0000 7f7e3a536700 1 Processor -- start 2020-07-18T03:26:24.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.353+0000 7f7e3a536700 1 -- start start 2020-07-18T03:26:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e3a536700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 0x7f7e340b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e3a536700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7e3414cac0 0x7f7e3412adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e3a536700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3414d240 con 0x7f7e340ad100 2020-07-18T03:26:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e3a536700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3414cfa0 con 0x7f7e3414cac0 2020-07-18T03:26:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e3a536700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3414d520 con 0x7f7e3405f5a0 2020-07-18T03:26:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 0x7f7e340b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e38ad3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7e3414cac0 0x7f7e3412adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 0x7f7e340b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:52230/0 (socket says 172.21.15.47:52230) 2020-07-18T03:26:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e337fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f7e340ad100 0x7f7e34145b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59484/0 (socket says 172.21.15.47:59484) 2020-07-18T03:26:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e33fff700 1 -- 172.21.15.47:0/2051766160 learned_addr learned my addr 172.21.15.47:0/2051766160 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.354+0000 7f7e38ad3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7e3414cac0 0x7f7e3412adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:45314/0 (socket says 172.21.15.47:45314) 2020-07-18T03:26:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e38ad3700 1 -- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 msgr2=0x7f7e340b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e38ad3700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 0x7f7e340b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e38ad3700 1 -- 172.21.15.47:0/2051766160 >> v1:172.21.15.18:6789/0 conn(0x7f7e340ad100 legacy=0x7f7e34145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e38ad3700 1 -- 172.21.15.47:0/2051766160 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34139320 con 0x7f7e3414cac0 2020-07-18T03:26:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736242391 0 0) 0x7f7e3414d240 con 0x7f7e340ad100 2020-07-18T03:26:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e33fff700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 0x7f7e340b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:24.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e38ad3700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e3414cac0 0x7f7e3412adc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f7e3405ca30 tx=0x7f7e2c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:24.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c005ca0 con 0x7f7e3414cac0 2020-07-18T03:26:24.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c005e00 con 0x7f7e3414cac0 2020-07-18T03:26:24.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3414a690 con 0x7f7e3414cac0 2020-07-18T03:26:24.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c007300 con 0x7f7e3414cac0 2020-07-18T03:26:24.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.355+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3412b2b0 con 0x7f7e3414cac0 2020-07-18T03:26:24.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.356+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e34057450 con 0x7f7e3414cac0 2020-07-18T03:26:24.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.356+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7e2c010030 con 0x7f7e3414cac0 2020-07-18T03:26:24.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.357+0000 7f7e13fff700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e0c04ff20 0x7f7e0c0523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:24.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.357+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7e2c067a40 con 0x7f7e3414cac0 2020-07-18T03:26:24.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.357+0000 7f7e33fff700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e0c04ff20 0x7f7e0c0523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:24.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.357+0000 7f7e33fff700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e0c04ff20 0x7f7e0c0523e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f7e2400d000 tx=0x7f7e24008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:24.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.361+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7e2c040170 con 0x7f7e3414cac0 2020-07-18T03:26:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:24.576+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]} v 0) v1 -- 0x7f7e3414d690 con 0x7f7e3414cac0 2020-07-18T03:26:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.224+0000 7f7e13fff700 1 -- 172.21.15.47:0/2051766160 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]}]=0 v57) v1 ==== 150+0+0 (secure 0 0 0) 0x7f7e2c043790 con 0x7f7e3414cac0 2020-07-18T03:26:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e0c04ff20 msgr2=0x7f7e0c0523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e0c04ff20 0x7f7e0c0523e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f7e2400d000 tx=0x7f7e24008040).stop 2020-07-18T03:26:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e3414cac0 msgr2=0x7f7e3412adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:25.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e3414cac0 0x7f7e3412adc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f7e3405ca30 tx=0x7f7e2c007de0).stop 2020-07-18T03:26:25.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 shutdown_connections 2020-07-18T03:26:25.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e0c04ff20 0x7f7e0c0523e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:25.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e3414cac0 0x7f7e3412adc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:25.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 --2- 172.21.15.47:0/2051766160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e3405f5a0 0x7f7e340b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 >> 172.21.15.47:0/2051766160 conn(0x7f7e340b9340 msgr2=0x7f7e340b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 shutdown_connections 2020-07-18T03:26:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:25.226+0000 7f7e3a536700 1 -- 172.21.15.47:0/2051766160 wait complete. 2020-07-18T03:26:25.241 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:26:25.242 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:26:25.297 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:26:25.358 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:26:25.359 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:26:25.359 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:26:25.360 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:25.403 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:26:25.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.405 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:25.461 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:26:25.461 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:26:25.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.463 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:26:25.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.542 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:26:25.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.651 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:26:25.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.691 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:26:25.746 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:25.747 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.747 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42805sec preferred_lft 42805sec 2020-07-18T03:26:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:26:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:25.749 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.749 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.749 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether a2:bd:03:fa:4a:e3 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::63:49ff:fee4:50d6/64 scope link 2020-07-18T03:26:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:25.751 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:26:25.751 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether a2:bd:03:fa:4a:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:26:25.751 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::a0bd:3ff:fefa:4ae3/64 scope link 2020-07-18T03:26:25.751 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:25.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.754 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:26:25.816 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.817 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:25.817 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.817 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.817 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.817 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.818 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.818 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.818 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.818 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.818 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.819 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether a2:bd:03:fa:4a:e3 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:25.819 INFO:teuthology.orchestra.run.smithi047.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:25.819 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether a2:bd:03:fa:4a:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:26:25.820 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:26:25.821 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:26:25.822 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:25.902 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:26:25.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.903 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:25.957 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:26:25.958 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:26:25.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:25.959 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:26:26.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.037 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.1' 2020-07-18T03:26:26.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.138 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:26:26.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.185 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:26:26.201 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:26.202 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:26.202 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:26.202 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:26.203 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:26.203 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:26.203 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:26.203 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.203 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:26.204 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.204 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:26.204 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42805sec preferred_lft 42805sec 2020-07-18T03:26:26.204 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:26:26.204 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:26.204 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:26.205 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.205 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:26.205 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.205 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:26.205 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.206 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:26.206 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:26.206 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::63:49ff:fee4:50d6/64 scope link 2020-07-18T03:26:26.206 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:26.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.207 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:26:26.272 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:26.273 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:26.273 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:26.273 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.274 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:26.274 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.274 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:26.275 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.275 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:26.275 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.275 INFO:teuthology.orchestra.run.smithi047.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:26.275 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:26.276 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:26:26.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.277 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:26:26.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.391 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:26:26.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.450 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:26:26.490 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:26:26.490 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:26.490 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:26.491 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:26.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.492 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:26.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.587 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:26.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.678 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:26.760 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi018.front.sepia.ceph.com... 2020-07-18T03:26:26.761 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:26:26.761 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:26:26.809 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:26:26.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.811 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:26:26.872 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-07-18T03:26:26.873 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' 2020-07-18T03:26:26.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.874 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:26:26.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:26.956 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:26:27.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.068 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2' 2020-07-18T03:26:27.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.110 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:26:27.164 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:27.164 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:27.164 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:27.165 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.165 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:27.165 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.165 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:27.165 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.166 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:27.166 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.166 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:27.166 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42803sec preferred_lft 42803sec 2020-07-18T03:26:27.166 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:26:27.167 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.167 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:27.167 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.167 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:27.167 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.167 INFO:teuthology.orchestra.run.smithi018.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:27.168 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether c2:13:4c:da:ad:5f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.168 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:27.168 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.168 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::804f:27ff:feee:118a/64 scope link 2020-07-18T03:26:27.168 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.169 INFO:teuthology.orchestra.run.smithi018.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:26:27.169 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether c2:13:4c:da:ad:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2020-07-18T03:26:27.169 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::c013:4cff:feda:ad5f/64 scope link 2020-07-18T03:26:27.169 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.170 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link show' 2020-07-18T03:26:27.235 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.236 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:27.236 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.236 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.236 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.237 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.237 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.237 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.237 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.237 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.238 INFO:teuthology.orchestra.run.smithi018.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.238 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether c2:13:4c:da:ad:5f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.238 INFO:teuthology.orchestra.run.smithi018.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.238 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether c2:13:4c:da:ad:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2020-07-18T03:26:27.239 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi018.front.sepia.ceph.com... 2020-07-18T03:26:27.240 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:26:27.240 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:26:27.320 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:26:27.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.321 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:26:27.376 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-07-18T03:26:27.376 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' 2020-07-18T03:26:27.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.377 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:26:27.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.455 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link delete brx.1' 2020-07-18T03:26:27.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.556 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3' 2020-07-18T03:26:27.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.601 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:26:27.618 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:27.619 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:27.619 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:27.619 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.619 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:27.619 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.620 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:27.620 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.620 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:27.620 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.620 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:27.621 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42803sec preferred_lft 42803sec 2020-07-18T03:26:27.621 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:26:27.621 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.621 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:27.621 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.622 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:27.622 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.622 INFO:teuthology.orchestra.run.smithi018.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:27.622 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.622 INFO:teuthology.orchestra.run.smithi018.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:27.622 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.623 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::804f:27ff:feee:118a/64 scope link 2020-07-18T03:26:27.623 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:27.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.624 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link show' 2020-07-18T03:26:27.704 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.704 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:27.705 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.705 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.705 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.705 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.705 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.706 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.706 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.706 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.706 INFO:teuthology.orchestra.run.smithi018.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:26:27.706 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:27.707 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:26:27.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.710 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:26:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.744 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:26:27.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.843 INFO:teuthology.orchestra.run.smithi018:> route 2020-07-18T03:26:27.860 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2020-07-18T03:26:27.860 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:27.860 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:27.860 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:27.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.861 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:27.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:27.938 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:28.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:28.019 INFO:teuthology.orchestra.run.smithi018:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:28.100 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-07-18T03:26:28.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.226+0000 7f87bee36700 1 Processor -- start 2020-07-18T03:26:28.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.226+0000 7f87bee36700 1 -- start start 2020-07-18T03:26:28.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bee36700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87b8147150 0x7f87b81445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bee36700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8060ff0 con 0x7f87b80641d0 2020-07-18T03:26:28.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bee36700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b814b540 con 0x7f87b8063d20 2020-07-18T03:26:28.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bee36700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b8147530 con 0x7f87b8147150 2020-07-18T03:26:28.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bcbd2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87b8147150 0x7f87b81445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87b7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f87b8063d20 0x7f87b814a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40730/0 (socket says 172.21.15.47:40730) 2020-07-18T03:26:28.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87b7fff700 1 -- 172.21.15.47:0/2502722207 learned_addr learned my addr 172.21.15.47:0/2502722207 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bcbd2700 1 -- 172.21.15.47:0/2502722207 >> v1:172.21.15.47:6789/0 conn(0x7f87b8063d20 legacy=0x7f87b814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bcbd2700 1 -- 172.21.15.47:0/2502722207 >> v1:172.21.15.18:6789/0 conn(0x7f87b80641d0 legacy=0x7f87b814ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bcbd2700 1 -- 172.21.15.47:0/2502722207 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8147740 con 0x7f87b8147150 2020-07-18T03:26:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.227+0000 7f87bcbd2700 1 --2- 172.21.15.47:0/2502722207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8147150 0x7f87b81445c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f87ac002f30 tx=0x7f87ac006dd0).ready entity=mon.1 client_cookie=966ecd08aef54574 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87b77fe700 1 -- 172.21.15.47:0/2502722207 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87ac0097f0 con 0x7f87b8147150 2020-07-18T03:26:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87b77fe700 1 -- 172.21.15.47:0/2502722207 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac009950 con 0x7f87b8147150 2020-07-18T03:26:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87b77fe700 1 -- 172.21.15.47:0/2502722207 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87ac009cb0 con 0x7f87b8147150 2020-07-18T03:26:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 -- 172.21.15.47:0/2502722207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8147150 msgr2=0x7f87b81445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 --2- 172.21.15.47:0/2502722207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8147150 0x7f87b81445c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f87ac002f30 tx=0x7f87ac006dd0).stop 2020-07-18T03:26:28.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 -- 172.21.15.47:0/2502722207 shutdown_connections 2020-07-18T03:26:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 --2- 172.21.15.47:0/2502722207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8147150 0x7f87b81445c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 -- 172.21.15.47:0/2502722207 >> 172.21.15.47:0/2502722207 conn(0x7f87b805f5a0 msgr2=0x7f87b805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 -- 172.21.15.47:0/2502722207 shutdown_connections 2020-07-18T03:26:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.228+0000 7f87bee36700 1 -- 172.21.15.47:0/2502722207 wait complete. 2020-07-18T03:26:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 Processor -- start 2020-07-18T03:26:28.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 -- start start 2020-07-18T03:26:28.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87b8063d20 0x7f87b80b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87b80641d0 0x7f87b81554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8156180 con 0x7f87b81559e0 2020-07-18T03:26:28.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b8156420 con 0x7f87b8063d20 2020-07-18T03:26:28.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.230+0000 7f87bee36700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b6850 con 0x7f87b80641d0 2020-07-18T03:26:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87b8063d20 0x7f87b80b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87b8063d20 0x7f87b80b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59648/0 (socket says 172.21.15.47:59648) 2020-07-18T03:26:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 -- 172.21.15.47:0/683079504 learned_addr learned my addr 172.21.15.47:0/683079504 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87b7fff700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87b80641d0 0x7f87b81554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 -- 172.21.15.47:0/683079504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87b80641d0 msgr2=0x7f87b81554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87b80641d0 0x7f87b81554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 -- 172.21.15.47:0/683079504 >> v1:172.21.15.18:6789/0 conn(0x7f87b81559e0 legacy=0x7f87b80b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 -- 172.21.15.47:0/683079504 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8130990 con 0x7f87b8063d20 2020-07-18T03:26:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bcbd2700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8063d20 0x7f87b80b6360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f87ac002f30 tx=0x7f87ac015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:28.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87ac01d5d0 con 0x7f87b8063d20 2020-07-18T03:26:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac01d730 con 0x7f87b8063d20 2020-07-18T03:26:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b80b5ae0 con 0x7f87b8063d20 2020-07-18T03:26:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87ac01da90 con 0x7f87b8063d20 2020-07-18T03:26:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.231+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b80b1ff0 con 0x7f87b8063d20 2020-07-18T03:26:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.232+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8057450 con 0x7f87b8063d20 2020-07-18T03:26:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.232+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f87ac025030 con 0x7f87b8063d20 2020-07-18T03:26:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.233+0000 7f87b4ff9700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f879004ff80 0x7f8790052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.233+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f87ac069590 con 0x7f87b8063d20 2020-07-18T03:26:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.233+0000 7f87b7fff700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f879004ff80 0x7f8790052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.233+0000 7f87b7fff700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f879004ff80 0x7f8790052440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f87a400d000 tx=0x7f87a4009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.236+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f87ac041d20 con 0x7f87b8063d20 2020-07-18T03:26:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.399+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f87b8147c60 con 0x7f87b8063d20 2020-07-18T03:26:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.400+0000 7f87b4ff9700 1 -- 172.21.15.47:0/683079504 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+34068 (secure 0 0 0) 0x7f87ac045340 con 0x7f87b8063d20 2020-07-18T03:26:28.402 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:28.403 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:26:28.403 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 24, 2020-07-18T03:26:28.403 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", 2020-07-18T03:26:28.403 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2020-07-18T03:25:24.387359+0000", 2020-07-18T03:26:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2020-07-18T03:25:45.899129+0000", 2020-07-18T03:26:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:26:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:26:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-07-18T03:26:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2020-07-18T03:26:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2020-07-18T03:26:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2020-07-18T03:26:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2020-07-18T03:26:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2020-07-18T03:26:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2020-07-18T03:26:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:26:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2020-07-18T03:26:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2020-07-18T03:26:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-07-18T03:26:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2020-07-18T03:26:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2020-07-18T03:26:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 4, 2020-07-18T03:26:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2020-07-18T03:26:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "luminous", 2020-07-18T03:26:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2020-07-18T03:26:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "pacific", 2020-07-18T03:26:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2020-07-18T03:26:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2020-07-18T03:26:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2020-07-18T03:26:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:27.725131+0000", 2020-07-18T03:26:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:26:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:26:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:26:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:26:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:26:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:26:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:26:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:26:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2020-07-18T03:26:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2020-07-18T03:26:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2020-07-18T03:26:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2020-07-18T03:26:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2020-07-18T03:26:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:26:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:26:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:26:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:26:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:26:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:26:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:26:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "15", 2020-07-18T03:26:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:26:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:26:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:26:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:26:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:26:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:26:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:26:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:26:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:26:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:26:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:26:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:26:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:26:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:26:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:26:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:26:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:26:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:26:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:26:28.430 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:26:28.430 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:26:28.430 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:26:28.431 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:26:28.431 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:26:28.431 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:26:28.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:26:28.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.432 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:26:28.432 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:26:28.432 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:26:28.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:26:28.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:26:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:26:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:26:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:26:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:26:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:26:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:26:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:26:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2020-07-18T03:26:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:26:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2020-07-18T03:26:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2020-07-18T03:26:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2020-07-18T03:26:28.436 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:37.896092+0000", 2020-07-18T03:26:28.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:26:28.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:26:28.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:26:28.436 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:26:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:26:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:26:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:26:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:26:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:26:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:26:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:26:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:26:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:26:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:26:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:26:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:26:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:26:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:26:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:26:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:26:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "18", 2020-07-18T03:26:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:26:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:26:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:26:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:26:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:26:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:26:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:26:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:26:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:26:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:26:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:26:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:26:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:26:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:26:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:26:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:26:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:26:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:26:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:26:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:26:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:26:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:26:28.456 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:26:28.456 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:26:28.456 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:26:28.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:26:28.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.457 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:26:28.457 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:26:28.457 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:26:28.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:26:28.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:26:28.458 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:26:28.458 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:26:28.458 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:26:28.459 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:26:28.459 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:26:28.459 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:26:28.459 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:26:28.459 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:26:28.460 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2020-07-18T03:26:28.460 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.460 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 3, 2020-07-18T03:26:28.460 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2020-07-18T03:26:28.461 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:41.055507+0000", 2020-07-18T03:26:28.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:26:28.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:26:28.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:26:28.461 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:26:28.462 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:26:28.462 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:26:28.462 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:26:28.462 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:26:28.462 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:26:28.463 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:26:28.463 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:26:28.463 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:26:28.463 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:26:28.463 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:26:28.463 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:26:28.464 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:26:28.464 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:26:28.464 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:26:28.464 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:26:28.464 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:26:28.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.465 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "24", 2020-07-18T03:26:28.465 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:26:28.465 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:26:28.465 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:26:28.466 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:26:28.466 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:26:28.466 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:26:28.466 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:26:28.466 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:26:28.467 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:26:28.467 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:26:28.467 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:26:28.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:26:28.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:26:28.467 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:26:28.468 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:26:28.468 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:26:28.468 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:26:28.468 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:26:28.468 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:26:28.469 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:26:28.469 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:26:28.469 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:26:28.469 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:26:28.469 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:26:28.470 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:26:28.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:26:28.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.470 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:26:28.470 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:26:28.471 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:26:28.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:26:28.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:26:28.471 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:26:28.471 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:26:28.471 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:26:28.472 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:26:28.472 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:26:28.472 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:26:28.472 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:26:28.472 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2020-07-18T03:26:28.473 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2020-07-18T03:26:28.473 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2020-07-18T03:26:28.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.473 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:26:28.473 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:26:28.474 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2020-07-18T03:26:28.474 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.475 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 4, 2020-07-18T03:26:28.475 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2020-07-18T03:26:28.476 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:43.561170+0000", 2020-07-18T03:26:28.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 5, 2020-07-18T03:26:28.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool,ec_overwrites", 2020-07-18T03:26:28.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 3, 2020-07-18T03:26:28.476 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 4, 2020-07-18T03:26:28.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 3, 2020-07-18T03:26:28.477 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 1, 2020-07-18T03:26:28.477 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:26:28.477 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:26:28.477 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:26:28.478 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:26:28.478 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:26:28.478 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:26:28.478 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:26:28.478 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:26:28.479 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:26:28.479 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:26:28.479 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:26:28.479 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:26:28.479 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:26:28.479 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:26:28.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.480 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "24", 2020-07-18T03:26:28.480 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:26:28.480 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:26:28.480 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:26:28.481 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:26:28.481 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:26:28.481 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:26:28.481 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:26:28.481 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:26:28.482 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:26:28.482 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:26:28.482 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:26:28.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:26:28.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:26:28.483 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:26:28.483 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:26:28.483 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:26:28.483 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:26:28.483 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:26:28.484 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:26:28.484 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:26:28.484 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:26:28.484 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:26:28.484 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:26:28.485 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "cephfs_data", 2020-07-18T03:26:28.485 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:26:28.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:26:28.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.485 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:26:28.486 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:26:28.486 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:26:28.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:26:28.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:26:28.486 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:26:28.487 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:26:28.487 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:26:28.487 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 8192, 2020-07-18T03:26:28.487 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:26:28.487 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:26:28.488 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:26:28.488 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:26:28.488 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:26:28.488 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2020-07-18T03:26:28.488 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:26:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2020-07-18T03:26:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:26:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", 2020-07-18T03:26:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 19, 2020-07-18T03:26:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2020-07-18T03:26:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2020-07-18T03:26:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2020-07-18T03:26:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2020-07-18T03:26:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2020-07-18T03:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2020-07-18T03:26:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2020-07-18T03:26:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2020-07-18T03:26:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:26:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6821/23865", 2020-07-18T03:26:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6825/23865", 2020-07-18T03:26:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6832/23865", 2020-07-18T03:26:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6829/23865", 2020-07-18T03:26:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:26:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", 2020-07-18T03:26:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 13, 2020-07-18T03:26:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2020-07-18T03:26:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2020-07-18T03:26:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2020-07-18T03:26:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2020-07-18T03:26:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2020-07-18T03:26:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2020-07-18T03:26:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2020-07-18T03:26:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2020-07-18T03:26:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:26:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/23853", 2020-07-18T03:26:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/23853", 2020-07-18T03:26:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/23853", 2020-07-18T03:26:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/23853", 2020-07-18T03:26:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:26:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "dc0b1a97-efca-483d-839c-37c286699283", 2020-07-18T03:26:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 22, 2020-07-18T03:26:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2020-07-18T03:26:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2020-07-18T03:26:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2020-07-18T03:26:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2020-07-18T03:26:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.538 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2020-07-18T03:26:28.539 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2020-07-18T03:26:28.540 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.540 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.540 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.540 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2020-07-18T03:26:28.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2020-07-18T03:26:28.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:26:28.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.543 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.543 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/23872", 2020-07-18T03:26:28.543 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6824/23872", 2020-07-18T03:26:28.543 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/23872", 2020-07-18T03:26:28.543 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6828/23872", 2020-07-18T03:26:28.544 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.544 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.544 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.544 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.545 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:26:28.545 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", 2020-07-18T03:26:28.545 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.545 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.546 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.546 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.546 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.546 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.546 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.546 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 22, 2020-07-18T03:26:28.547 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.547 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.547 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2020-07-18T03:26:28.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2020-07-18T03:26:28.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.549 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.549 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.550 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2020-07-18T03:26:28.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2020-07-18T03:26:28.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.552 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.552 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2020-07-18T03:26:28.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2020-07-18T03:26:28.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.554 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.554 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.555 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2020-07-18T03:26:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2020-07-18T03:26:28.557 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:26:28.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.557 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.557 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/23854", 2020-07-18T03:26:28.564 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/23854", 2020-07-18T03:26:28.564 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/23854", 2020-07-18T03:26:28.564 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/23854", 2020-07-18T03:26:28.564 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.565 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.565 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.565 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.566 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:26:28.566 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", 2020-07-18T03:26:28.566 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.566 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.566 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.567 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.567 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.567 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.567 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.567 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 22, 2020-07-18T03:26:28.568 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.568 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6808", 2020-07-18T03:26:28.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6809", 2020-07-18T03:26:28.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.571 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6810", 2020-07-18T03:26:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6811", 2020-07-18T03:26:28.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.573 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6814", 2020-07-18T03:26:28.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6815", 2020-07-18T03:26:28.576 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.576 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.576 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.576 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6812", 2020-07-18T03:26:28.577 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6813", 2020-07-18T03:26:28.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:26:28.578 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.579 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6809/24092", 2020-07-18T03:26:28.579 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6811/24092", 2020-07-18T03:26:28.579 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6815/24092", 2020-07-18T03:26:28.580 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6813/24092", 2020-07-18T03:26:28.580 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.580 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.580 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.580 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.581 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:26:28.581 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", 2020-07-18T03:26:28.581 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.581 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.582 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.582 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.582 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.582 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.582 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.583 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 22, 2020-07-18T03:26:28.583 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.583 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6800", 2020-07-18T03:26:28.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6801", 2020-07-18T03:26:28.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.585 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.586 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6802", 2020-07-18T03:26:28.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6803", 2020-07-18T03:26:28.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.588 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.588 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6806", 2020-07-18T03:26:28.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6807", 2020-07-18T03:26:28.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.591 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6804", 2020-07-18T03:26:28.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6805", 2020-07-18T03:26:28.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:26:28.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6801/24091", 2020-07-18T03:26:28.594 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6803/24091", 2020-07-18T03:26:28.594 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6807/24091", 2020-07-18T03:26:28.594 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6805/24091", 2020-07-18T03:26:28.594 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.595 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.595 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:26:28.596 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", 2020-07-18T03:26:28.596 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.596 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.596 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.596 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.597 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.597 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.597 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.597 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 22, 2020-07-18T03:26:28.597 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.598 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.598 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6816", 2020-07-18T03:26:28.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6817", 2020-07-18T03:26:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6819", 2020-07-18T03:26:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6821", 2020-07-18T03:26:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6828", 2020-07-18T03:26:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6830", 2020-07-18T03:26:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6823", 2020-07-18T03:26:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6826", 2020-07-18T03:26:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:26:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6817/24093", 2020-07-18T03:26:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6821/24093", 2020-07-18T03:26:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6830/24093", 2020-07-18T03:26:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6826/24093", 2020-07-18T03:26:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:26:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", 2020-07-18T03:26:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:26:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:26:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:26:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:26:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:26:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:26:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:26:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 16, 2020-07-18T03:26:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:26:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:26:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:26:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6818", 2020-07-18T03:26:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6820", 2020-07-18T03:26:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:26:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6822", 2020-07-18T03:26:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6824", 2020-07-18T03:26:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:26:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6829", 2020-07-18T03:26:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6831", 2020-07-18T03:26:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:26:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:26:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:26:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6825", 2020-07-18T03:26:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6827", 2020-07-18T03:26:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:26:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6820/24094", 2020-07-18T03:26:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6824/24094", 2020-07-18T03:26:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6831/24094", 2020-07-18T03:26:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6827/24094", 2020-07-18T03:26:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:26:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:26:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:26:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:26:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2020-07-18T03:26:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:26:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.933644+0000", 2020-07-18T03:26:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:26:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.929073+0000", 2020-07-18T03:26:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:26:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.974285+0000", 2020-07-18T03:26:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:26:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.909674+0000", 2020-07-18T03:26:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:26:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.935590+0000", 2020-07-18T03:26:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:26:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.916457+0000", 2020-07-18T03:26:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:26:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.990672+0000", 2020-07-18T03:26:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:26:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:26:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:26:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:26:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:26:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:26:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.948777+0000", 2020-07-18T03:26:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:26:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:26:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2020-07-18T03:26:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2020-07-18T03:26:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2020-07-18T03:26:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2020-07-18T03:26:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2020-07-18T03:26:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2020-07-18T03:26:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2020-07-18T03:26:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2020-07-18T03:26:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:26:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2020-07-18T03:26:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2020-07-18T03:26:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:26:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2020-07-18T03:26:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:26:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:26:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2020-07-18T03:26:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2020-07-18T03:26:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2020-07-18T03:26:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:26:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:26:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2020-07-18T03:26:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:26:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:26:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2020-07-18T03:26:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2020-07-18T03:26:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2020-07-18T03:26:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2020-07-18T03:26:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2020-07-18T03:26:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2020-07-18T03:26:28.648 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:26:28.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f879004ff80 msgr2=0x7f8790052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f879004ff80 0x7f8790052440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f87a400d000 tx=0x7f87a4009040).stop 2020-07-18T03:26:28.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8063d20 msgr2=0x7f87b80b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8063d20 0x7f87b80b6360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f87ac002f30 tx=0x7f87ac015000).stop 2020-07-18T03:26:28.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 shutdown_connections 2020-07-18T03:26:28.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f879004ff80 0x7f8790052440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87b8063d20 0x7f87b80b6360 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 --2- 172.21.15.47:0/683079504 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f87b80641d0 0x7f87b81554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 >> 172.21.15.47:0/683079504 conn(0x7f87b805f5a0 msgr2=0x7f87b80622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:28.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 shutdown_connections 2020-07-18T03:26:28.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.401+0000 7f87bee36700 1 -- 172.21.15.47:0/683079504 wait complete. 2020-07-18T03:26:28.653 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:28.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 Processor -- start 2020-07-18T03:26:28.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 -- start start 2020-07-18T03:26:28.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99a80ad100 0x7f99a80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99a8147a00 0x7f99a8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a8147330 con 0x7f99a805f5a0 2020-07-18T03:26:28.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80ae530 con 0x7f99a8147a00 2020-07-18T03:26:28.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.788+0000 7f99ae259700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80b57e0 con 0x7f99a80ad100 2020-07-18T03:26:28.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99a8147a00 0x7f99a8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99a8147a00 0x7f99a8147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59660/0 (socket says 172.21.15.47:59660) 2020-07-18T03:26:28.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 -- 172.21.15.47:0/3146827884 learned_addr learned my addr 172.21.15.47:0/3146827884 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:28.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a77fe700 1 --2- 172.21.15.47:0/3146827884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99a80ad100 0x7f99a80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 -- 172.21.15.47:0/3146827884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99a80ad100 msgr2=0x7f99a80b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:28.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 --2- 172.21.15.47:0/3146827884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99a80ad100 0x7f99a80b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 -- 172.21.15.47:0/3146827884 >> v1:172.21.15.18:6789/0 conn(0x7f99a805f5a0 legacy=0x7f99a805fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 -- 172.21.15.47:0/3146827884 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80b5a80 con 0x7f99a8147a00 2020-07-18T03:26:28.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a6ffd700 1 --2- 172.21.15.47:0/3146827884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a8147a00 0x7f99a8147e40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f999c002f30 tx=0x7f999c006e90).ready entity=mon.1 client_cookie=1f7803580267a304 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:28.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a67fc700 1 -- 172.21.15.47:0/3146827884 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999c009850 con 0x7f99a8147a00 2020-07-18T03:26:28.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a67fc700 1 -- 172.21.15.47:0/3146827884 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c0099b0 con 0x7f99a8147a00 2020-07-18T03:26:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99a67fc700 1 -- 172.21.15.47:0/3146827884 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999c009d10 con 0x7f99a8147a00 2020-07-18T03:26:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 -- 172.21.15.47:0/3146827884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a8147a00 msgr2=0x7f99a8147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 --2- 172.21.15.47:0/3146827884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a8147a00 0x7f99a8147e40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f999c002f30 tx=0x7f999c006e90).stop 2020-07-18T03:26:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 -- 172.21.15.47:0/3146827884 shutdown_connections 2020-07-18T03:26:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 --2- 172.21.15.47:0/3146827884 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f99a80ad100 0x7f99a80b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 --2- 172.21.15.47:0/3146827884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a8147a00 0x7f99a8147e40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 -- 172.21.15.47:0/3146827884 >> 172.21.15.47:0/3146827884 conn(0x7f99a80b9340 msgr2=0x7f99a80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 -- 172.21.15.47:0/3146827884 shutdown_connections 2020-07-18T03:26:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.789+0000 7f99ae259700 1 -- 172.21.15.47:0/3146827884 wait complete. 2020-07-18T03:26:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.791+0000 7f99ae259700 1 Processor -- start 2020-07-18T03:26:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.791+0000 7f99ae259700 1 -- start start 2020-07-18T03:26:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99ae259700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99a805f5a0 0x7f99a8061ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99ae259700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f99a81426d0 0x7f99a8138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99ae259700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a8061fc0 con 0x7f99a80ad100 2020-07-18T03:26:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99ae259700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a8139ba0 con 0x7f99a81426d0 2020-07-18T03:26:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99ae259700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a8143a90 con 0x7f99a805f5a0 2020-07-18T03:26:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99a805f5a0 0x7f99a8061ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f99a80ad100 0x7f99a813b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40752/0 (socket says 172.21.15.47:40752) 2020-07-18T03:26:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a6ffd700 1 -- 172.21.15.47:0/1392648975 learned_addr learned my addr 172.21.15.47:0/1392648975 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a77fe700 1 -- 172.21.15.47:0/1392648975 >> v2:172.21.15.18:3300/0 conn(0x7f99a81426d0 msgr2=0x7f99a8138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a77fe700 1 --2- 172.21.15.47:0/1392648975 >> v2:172.21.15.18:3300/0 conn(0x7f99a81426d0 0x7f99a8138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-07-18T03:26:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a77fe700 1 -- 172.21.15.47:0/1392648975 >> v1:172.21.15.47:6789/0 conn(0x7f99a80ad100 legacy=0x7f99a813b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a77fe700 1 -- 172.21.15.47:0/1392648975 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8146d70 con 0x7f99a805f5a0 2020-07-18T03:26:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621770693 0 0) 0x7f99a8061fc0 con 0x7f99a80ad100 2020-07-18T03:26:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99a77fe700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a805f5a0 0x7f99a8061ad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f999800c000 tx=0x7f9998006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.792+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999800ca10 con 0x7f99a805f5a0 2020-07-18T03:26:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.793+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800cb70 con 0x7f99a805f5a0 2020-07-18T03:26:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.793+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a81394e0 con 0x7f99a805f5a0 2020-07-18T03:26:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.793+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99980123f0 con 0x7f99a805f5a0 2020-07-18T03:26:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.793+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a8143710 con 0x7f99a805f5a0 2020-07-18T03:26:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.794+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a8057450 con 0x7f99a805f5a0 2020-07-18T03:26:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.794+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f99980125f0 con 0x7f99a805f5a0 2020-07-18T03:26:28.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.794+0000 7f99877fe700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f998004fed0 0x7f9980052390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:28.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.795+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f99980681e0 con 0x7f99a805f5a0 2020-07-18T03:26:28.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.795+0000 7f99a6ffd700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f998004fed0 0x7f9980052390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:28.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.795+0000 7f99a6ffd700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f998004fed0 0x7f9980052390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f999c000b80 tx=0x7f999c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:28.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:28.798+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9998040890 con 0x7f99a805f5a0 2020-07-18T03:26:29.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.010+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a813a990 con 0x7f99a805f5a0 2020-07-18T03:26:29.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.011+0000 7f99877fe700 1 -- 172.21.15.47:0/1392648975 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2124 (secure 0 0 0) 0x7f9998043eb0 con 0x7f99a805f5a0 2020-07-18T03:26:29.013 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:29.013 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:25:49.581963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4387":{"gid":4387,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.47:6835/308881804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":308881804},{"type":"v1","addr":"172.21.15.47:6835","nonce":308881804}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:26:29.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f998004fed0 msgr2=0x7f9980052390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f998004fed0 0x7f9980052390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f999c000b80 tx=0x7f999c015000).stop 2020-07-18T03:26:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a805f5a0 msgr2=0x7f99a8061ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a805f5a0 0x7f99a8061ad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f999800c000 tx=0x7f9998006dd0).stop 2020-07-18T03:26:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 shutdown_connections 2020-07-18T03:26:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 --2- 172.21.15.47:0/1392648975 >> v2:172.21.15.18:3300/0 conn(0x7f99a81426d0 0x7f99a8138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-07-18T03:26:29.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99a805f5a0 0x7f99a8061ad0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:29.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 --2- 172.21.15.47:0/1392648975 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f998004fed0 0x7f9980052390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:29.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 >> 172.21.15.47:0/1392648975 conn(0x7f99a80b9340 msgr2=0x7f99a80b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:29.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 shutdown_connections 2020-07-18T03:26:29.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.012+0000 7f99ae259700 1 -- 172.21.15.47:0/1392648975 wait complete. 2020-07-18T03:26:29.017 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2020-07-18T03:26:29.028 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-07-18T03:26:29.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.145+0000 7f23e41aa700 1 Processor -- start 2020-07-18T03:26:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.145+0000 7f23e41aa700 1 -- start start 2020-07-18T03:26:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e41aa700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 0x7f23dc142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e41aa700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0b5830 con 0x7f23dc05ffb0 2020-07-18T03:26:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e41aa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc146da0 con 0x7f23dc143330 2020-07-18T03:26:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e41aa700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0b5a60 con 0x7f23dc0aea60 2020-07-18T03:26:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e2747700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f23dc143330 0x7f23dc1471d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40762/0 (socket says 172.21.15.47:40762) 2020-07-18T03:26:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e2747700 1 -- 172.21.15.47:0/1133314476 learned_addr learned my addr 172.21.15.47:0/1133314476 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674384456 0 0) 0x7f23dc146da0 con 0x7f23dc143330 2020-07-18T03:26:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e1745700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 0x7f23dc142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.146+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c4003240 con 0x7f23dc143330 2020-07-18T03:26:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1294829244 0 0) 0x7f23c4003240 con 0x7f23dc143330 2020-07-18T03:26:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 msgr2=0x7f23dc142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:29.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 0x7f23dc142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:29.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 >> v1:172.21.15.18:6789/0 conn(0x7f23dc05ffb0 legacy=0x7f23dc0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc060390 con 0x7f23dc143330 2020-07-18T03:26:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f23d4001a90 con 0x7f23dc143330 2020-07-18T03:26:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 >> v1:172.21.15.47:6789/0 conn(0x7f23dc143330 legacy=0x7f23dc1471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 0x7f23c4003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c4003c40 0x7f23c40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c40049d0 0x7f23c4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0b5a60 con 0x7f23c40049d0 2020-07-18T03:26:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40041d0 con 0x7f23c4003c40 2020-07-18T03:26:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c400d940 con 0x7f23dc0aea60 2020-07-18T03:26:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c4003c40 0x7f23c40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1f46700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 0x7f23c4003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e2747700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c40049d0 0x7f23c4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 -- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 msgr2=0x7f23c4003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc0aea60 0x7f23c4003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 -- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c40049d0 msgr2=0x7f23c4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c40049d0 0x7f23c4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 -- 172.21.15.47:0/1133314476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4003240 con 0x7f23c4003c40 2020-07-18T03:26:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e2747700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23c40049d0 0x7f23c4004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e1745700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c4003c40 0x7f23c40044e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f23d800c000 tx=0x7f23d8008040).ready entity=mon.1 client_cookie=1181960f5f3fa297 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d800e840 con 0x7f23c4003c40 2020-07-18T03:26:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800e9a0 con 0x7f23c4003c40 2020-07-18T03:26:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e0f44700 1 -- 172.21.15.47:0/1133314476 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d800ed00 con 0x7f23c4003c40 2020-07-18T03:26:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e41aa700 1 -- 172.21.15.47:0/1133314476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c4003c40 msgr2=0x7f23c40044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e41aa700 1 --2- 172.21.15.47:0/1133314476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c4003c40 0x7f23c40044e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f23d800c000 tx=0x7f23d8008040).stop 2020-07-18T03:26:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e2747700 1 -- 172.21.15.47:0/1133314476 reap_dead start 2020-07-18T03:26:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e41aa700 1 -- 172.21.15.47:0/1133314476 shutdown_connections 2020-07-18T03:26:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.147+0000 7f23e41aa700 1 -- 172.21.15.47:0/1133314476 >> 172.21.15.47:0/1133314476 conn(0x7f23dc0b93c0 msgr2=0x7f23dc0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.148+0000 7f23e41aa700 1 -- 172.21.15.47:0/1133314476 shutdown_connections 2020-07-18T03:26:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.148+0000 7f23e41aa700 1 -- 172.21.15.47:0/1133314476 wait complete. 2020-07-18T03:26:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.149+0000 7f23e41aa700 1 Processor -- start 2020-07-18T03:26:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.149+0000 7f23e41aa700 1 -- start start 2020-07-18T03:26:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e41aa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23dc1421a0 0x7f23dc154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e41aa700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc142580 0x7f23dc065080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e41aa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0639c0 con 0x7f23dc065570 2020-07-18T03:26:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e41aa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc063c60 con 0x7f23dc1421a0 2020-07-18T03:26:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e41aa700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc150ff0 con 0x7f23dc142580 2020-07-18T03:26:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23dc1421a0 0x7f23dc154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23dc1421a0 0x7f23dc154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59682/0 (socket says 172.21.15.47:59682) 2020-07-18T03:26:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 -- 172.21.15.47:0/83749368 learned_addr learned my addr 172.21.15.47:0/83749368 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1745700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc142580 0x7f23dc065080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 -- 172.21.15.47:0/83749368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc142580 msgr2=0x7f23dc065080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc142580 0x7f23dc065080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 -- 172.21.15.47:0/83749368 >> v1:172.21.15.47:6789/0 conn(0x7f23dc065570 legacy=0x7f23dc150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878822316 0 0) 0x7f23dc0639c0 con 0x7f23dc065570 2020-07-18T03:26:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 -- 172.21.15.47:0/83749368 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0665e0 con 0x7f23dc1421a0 2020-07-18T03:26:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e1f46700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23dc1421a0 0x7f23dc154fa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f23d0007770 tx=0x7f23d0007ae0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d0002830 con 0x7f23dc1421a0 2020-07-18T03:26:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0002990 con 0x7f23dc1421a0 2020-07-18T03:26:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc063590 con 0x7f23dc1421a0 2020-07-18T03:26:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.150+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d0002cf0 con 0x7f23dc1421a0 2020-07-18T03:26:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.151+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc150d60 con 0x7f23dc1421a0 2020-07-18T03:26:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.151+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc06a5c0 con 0x7f23dc1421a0 2020-07-18T03:26:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.151+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f23d0014450 con 0x7f23dc1421a0 2020-07-18T03:26:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.152+0000 7f23cdffb700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f23b404fee0 0x7f23b40523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.152+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f23d0067880 con 0x7f23dc1421a0 2020-07-18T03:26:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.152+0000 7f23e1745700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f23b404fee0 0x7f23b40523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.152+0000 7f23e1745700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f23b404fee0 0x7f23b40523a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f23d801c000 tx=0x7f23d801a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.155+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f23d0040060 con 0x7f23dc1421a0 2020-07-18T03:26:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:29.367+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f23dc057450 con 0x7f23dc1421a0 2020-07-18T03:26:30.230 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:26:30.229+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:26:30.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.234+0000 7f23cdffb700 1 -- 172.21.15.47:0/83749368 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f23d0043680 con 0x7f23dc1421a0 2020-07-18T03:26:30.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f23b404fee0 msgr2=0x7f23b40523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f23b404fee0 0x7f23b40523a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f23d801c000 tx=0x7f23d801a000).stop 2020-07-18T03:26:30.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23dc1421a0 msgr2=0x7f23dc154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23dc1421a0 0x7f23dc154fa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f23d0007770 tx=0x7f23d0007ae0).stop 2020-07-18T03:26:30.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 shutdown_connections 2020-07-18T03:26:30.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f23b404fee0 0x7f23b40523a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23dc1421a0 0x7f23dc154fa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 --2- 172.21.15.47:0/83749368 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f23dc142580 0x7f23dc065080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 >> 172.21.15.47:0/83749368 conn(0x7f23dc0b93c0 msgr2=0x7f23dc0b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:30.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 shutdown_connections 2020-07-18T03:26:30.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.236+0000 7f23e41aa700 1 -- 172.21.15.47:0/83749368 wait complete. 2020-07-18T03:26:30.240 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-07-18T03:26:30.251 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:30.260 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:26:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.375+0000 7fbd1f910700 1 Processor -- start 2020-07-18T03:26:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.375+0000 7fbd1f910700 1 -- start start 2020-07-18T03:26:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1f910700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd18147a00 0x7fbd18147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1f910700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd18147330 con 0x7fbd181433e0 2020-07-18T03:26:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1f910700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd180b57e0 con 0x7fbd180ad100 2020-07-18T03:26:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1f910700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180ae530 con 0x7fbd18147a00 2020-07-18T03:26:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1ceab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd18147a00 0x7fbd18147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1d6ac700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd180ad100 0x7fbd180b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40790/0 (socket says 172.21.15.47:40790) 2020-07-18T03:26:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1d6ac700 1 -- 172.21.15.47:0/2463790380 learned_addr learned my addr 172.21.15.47:0/2463790380 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1ceab700 1 -- 172.21.15.47:0/2463790380 >> v1:172.21.15.47:6789/0 conn(0x7fbd180ad100 legacy=0x7fbd180b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1ceab700 1 -- 172.21.15.47:0/2463790380 >> v1:172.21.15.18:6789/0 conn(0x7fbd181433e0 legacy=0x7fbd18143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd07fff700 1 -- 172.21.15.47:0/2463790380 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976346039 0 0) 0x7fbd180b57e0 con 0x7fbd180ad100 2020-07-18T03:26:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1ceab700 1 -- 172.21.15.47:0/2463790380 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180b5a80 con 0x7fbd18147a00 2020-07-18T03:26:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1ceab700 1 --2- 172.21.15.47:0/2463790380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd18147a00 0x7fbd18147e40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbd1805ca30 tx=0x7fbd08006dd0).ready entity=mon.1 client_cookie=51115dcc52061c5e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd07fff700 1 -- 172.21.15.47:0/2463790380 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd080097b0 con 0x7fbd18147a00 2020-07-18T03:26:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd07fff700 1 -- 172.21.15.47:0/2463790380 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd08009910 con 0x7fbd18147a00 2020-07-18T03:26:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd07fff700 1 -- 172.21.15.47:0/2463790380 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd08009c70 con 0x7fbd18147a00 2020-07-18T03:26:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1f910700 1 -- 172.21.15.47:0/2463790380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd18147a00 msgr2=0x7fbd18147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.376+0000 7fbd1f910700 1 --2- 172.21.15.47:0/2463790380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd18147a00 0x7fbd18147e40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbd1805ca30 tx=0x7fbd08006dd0).stop 2020-07-18T03:26:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.377+0000 7fbd1f910700 1 -- 172.21.15.47:0/2463790380 shutdown_connections 2020-07-18T03:26:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.377+0000 7fbd1f910700 1 --2- 172.21.15.47:0/2463790380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd18147a00 0x7fbd18147e40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.377+0000 7fbd1f910700 1 -- 172.21.15.47:0/2463790380 >> 172.21.15.47:0/2463790380 conn(0x7fbd180b9340 msgr2=0x7fbd180b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.377+0000 7fbd1f910700 1 -- 172.21.15.47:0/2463790380 shutdown_connections 2020-07-18T03:26:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.377+0000 7fbd1f910700 1 -- 172.21.15.47:0/2463790380 wait complete. 2020-07-18T03:26:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.378+0000 7fbd1f910700 1 Processor -- start 2020-07-18T03:26:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.378+0000 7fbd1f910700 1 -- start start 2020-07-18T03:26:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1f910700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd180ad100 0x7fbd181573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1f910700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbd18066a20 0x7fbd18146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1f910700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd18064200 con 0x7fbd181433e0 2020-07-18T03:26:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1f910700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd181578c0 con 0x7fbd18066a20 2020-07-18T03:26:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1f910700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd18063cc0 con 0x7fbd180ad100 2020-07-18T03:26:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1d6ac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd180ad100 0x7fbd181573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1ceab700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd181433e0 0x7fbd180655e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40798/0 (socket says 172.21.15.47:40798) 2020-07-18T03:26:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1ceab700 1 -- 172.21.15.47:0/3412920186 learned_addr learned my addr 172.21.15.47:0/3412920186 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1dead700 1 --2- 172.21.15.47:0/3412920186 >> v2:172.21.15.18:3300/0 conn(0x7fbd18066a20 0x7fbd18146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1d6ac700 1 -- 172.21.15.47:0/3412920186 >> v2:172.21.15.18:3300/0 conn(0x7fbd18066a20 msgr2=0x7fbd18146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1d6ac700 1 --2- 172.21.15.47:0/3412920186 >> v2:172.21.15.18:3300/0 conn(0x7fbd18066a20 0x7fbd18146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1d6ac700 1 -- 172.21.15.47:0/3412920186 >> v1:172.21.15.47:6789/0 conn(0x7fbd181433e0 legacy=0x7fbd180655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1d6ac700 1 -- 172.21.15.47:0/3412920186 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd18063f60 con 0x7fbd180ad100 2020-07-18T03:26:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554009392 0 0) 0x7fbd18064200 con 0x7fbd181433e0 2020-07-18T03:26:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1dead700 1 --2- 172.21.15.47:0/3412920186 >> v2:172.21.15.18:3300/0 conn(0x7fbd18066a20 0x7fbd18146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd1d6ac700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180ad100 0x7fbd181573d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fbd10001210 tx=0x7fbd10007cc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.379+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd10005b80 con 0x7fbd180ad100 2020-07-18T03:26:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.380+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd10005ce0 con 0x7fbd180ad100 2020-07-18T03:26:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.380+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd18146c50 con 0x7fbd180ad100 2020-07-18T03:26:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.380+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd10007300 con 0x7fbd180ad100 2020-07-18T03:26:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.380+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd180661c0 con 0x7fbd180ad100 2020-07-18T03:26:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.381+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd18063b20 con 0x7fbd180ad100 2020-07-18T03:26:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.381+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbd10010030 con 0x7fbd180ad100 2020-07-18T03:26:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.381+0000 7fbd057fa700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbcf004fe60 0x7fbcf0052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.381+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fbd10067d30 con 0x7fbd180ad100 2020-07-18T03:26:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.381+0000 7fbd1ceab700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbcf004fe60 0x7fbcf0052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.381+0000 7fbd1ceab700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbcf004fe60 0x7fbcf0052320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fbd08006a00 tx=0x7fbd08015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.385+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbd18063b20 con 0x7fbd180ad100 2020-07-18T03:26:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.592+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd18063b20 con 0x7fbd180ad100 2020-07-18T03:26:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.592+0000 7fbd057fa700 1 -- 172.21.15.47:0/3412920186 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+1767 (secure 0 0 0) 0x7fbd18063b20 con 0x7fbd180ad100 2020-07-18T03:26:30.594 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:30.594 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/642174240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":642174240},{"type":"v1","addr":"172.21.15.18:6833","nonce":642174240}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:25:45.899311+0000","modified":"2020-07-18T03:26:30.230029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-07-18T03:26:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbcf004fe60 msgr2=0x7fbcf0052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbcf004fe60 0x7fbcf0052320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fbd08006a00 tx=0x7fbd08015000).stop 2020-07-18T03:26:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180ad100 msgr2=0x7fbd181573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180ad100 0x7fbd181573d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fbd10001210 tx=0x7fbd10007cc0).stop 2020-07-18T03:26:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 shutdown_connections 2020-07-18T03:26:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 --2- 172.21.15.47:0/3412920186 >> v2:172.21.15.18:3300/0 conn(0x7fbd18066a20 0x7fbd18146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd180ad100 0x7fbd181573d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 --2- 172.21.15.47:0/3412920186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbcf004fe60 0x7fbcf0052320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 >> 172.21.15.47:0/3412920186 conn(0x7fbd180b9340 msgr2=0x7fbd180b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 shutdown_connections 2020-07-18T03:26:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.594+0000 7fbd1f910700 1 -- 172.21.15.47:0/3412920186 wait complete. 2020-07-18T03:26:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2020-07-18T03:26:30.605 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-07-18T03:26:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 Processor -- start 2020-07-18T03:26:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 -- start start 2020-07-18T03:26:30.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40ad330 0x7f05e4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e40b59b0 0x7f05e40b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f05e40b6300 0x7f05e40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e4147de0 con 0x7f05e40b6300 2020-07-18T03:26:30.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e40ae000 con 0x7f05e40b59b0 2020-07-18T03:26:30.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.761+0000 7f05e885e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4148010 con 0x7f05e40ad330 2020-07-18T03:26:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e40b59b0 0x7f05e40b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e40b59b0 0x7f05e40b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59720/0 (socket says 172.21.15.47:59720) 2020-07-18T03:26:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 -- 172.21.15.47:0/3503385596 learned_addr learned my addr 172.21.15.47:0/3503385596 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e2d9d700 1 --2- 172.21.15.47:0/3503385596 >> v2:172.21.15.18:3300/0 conn(0x7f05e40b6300 0x7f05e40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 -- 172.21.15.47:0/3503385596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40ad330 msgr2=0x7f05e4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e259c700 1 --2- 172.21.15.47:0/3503385596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40ad330 0x7f05e4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 --2- 172.21.15.47:0/3503385596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40ad330 0x7f05e4143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 -- 172.21.15.47:0/3503385596 >> v2:172.21.15.18:3300/0 conn(0x7f05e40b6300 msgr2=0x7f05e40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 --2- 172.21.15.47:0/3503385596 >> v2:172.21.15.18:3300/0 conn(0x7f05e40b6300 0x7f05e40b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 -- 172.21.15.47:0/3503385596 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40ad710 con 0x7f05e40b59b0 2020-07-18T03:26:30.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e2d9d700 1 --2- 172.21.15.47:0/3503385596 >> v2:172.21.15.18:3300/0 conn(0x7f05e40b6300 0x7f05e40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:30.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e1d9b700 1 --2- 172.21.15.47:0/3503385596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40b59b0 0x7f05e40b5e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f05d4002f30 tx=0x7f05d4006e90).ready entity=mon.1 client_cookie=492504e69636afc1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:30.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e159a700 1 -- 172.21.15.47:0/3503385596 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d4009920 con 0x7f05e40b59b0 2020-07-18T03:26:30.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e159a700 1 -- 172.21.15.47:0/3503385596 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4009a80 con 0x7f05e40b59b0 2020-07-18T03:26:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e159a700 1 -- 172.21.15.47:0/3503385596 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d4009de0 con 0x7f05e40b59b0 2020-07-18T03:26:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 -- 172.21.15.47:0/3503385596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40b59b0 msgr2=0x7f05e40b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 --2- 172.21.15.47:0/3503385596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40b59b0 0x7f05e40b5e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f05d4002f30 tx=0x7f05d4006e90).stop 2020-07-18T03:26:30.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 -- 172.21.15.47:0/3503385596 shutdown_connections 2020-07-18T03:26:30.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 --2- 172.21.15.47:0/3503385596 >> v2:172.21.15.18:3300/0 conn(0x7f05e40b6300 0x7f05e40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 --2- 172.21.15.47:0/3503385596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40ad330 0x7f05e4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 --2- 172.21.15.47:0/3503385596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40b59b0 0x7f05e40b5e10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 -- 172.21.15.47:0/3503385596 >> 172.21.15.47:0/3503385596 conn(0x7f05e405f5a0 msgr2=0x7f05e405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:30.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 -- 172.21.15.47:0/3503385596 shutdown_connections 2020-07-18T03:26:30.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.762+0000 7f05e885e700 1 -- 172.21.15.47:0/3503385596 wait complete. 2020-07-18T03:26:30.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.764+0000 7f05e885e700 1 Processor -- start 2020-07-18T03:26:30.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.764+0000 7f05e885e700 1 -- start start 2020-07-18T03:26:30.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e885e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e40ad330 0x7f05e415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e885e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40b32c0 0x7f05e40665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e885e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e415a520 con 0x7f05e40b6300 2020-07-18T03:26:30.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e885e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e4159650 con 0x7f05e40ad330 2020-07-18T03:26:30.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e885e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41598f0 con 0x7f05e40b32c0 2020-07-18T03:26:30.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e40ad330 0x7f05e415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e40ad330 0x7f05e415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59726/0 (socket says 172.21.15.47:59726) 2020-07-18T03:26:30.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e1d9b700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f05e40b6300 0x7f05e4061d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59576/0 (socket says 172.21.15.47:59576) 2020-07-18T03:26:30.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e259c700 1 -- 172.21.15.47:0/3280942424 learned_addr learned my addr 172.21.15.47:0/3280942424 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:30.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05e2d9d700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40b32c0 0x7f05e40665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.765+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720026633 0 0) 0x7f05e415a520 con 0x7f05e40b6300 2020-07-18T03:26:30.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c0003240 con 0x7f05e40b6300 2020-07-18T03:26:30.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e259c700 1 -- 172.21.15.47:0/3280942424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40b32c0 msgr2=0x7f05e40665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e259c700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40b32c0 0x7f05e40665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:30.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e259c700 1 -- 172.21.15.47:0/3280942424 >> v1:172.21.15.18:6789/0 conn(0x7f05e40b6300 legacy=0x7f05e4061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:30.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e259c700 1 -- 172.21.15.47:0/3280942424 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4159b90 con 0x7f05e40ad330 2020-07-18T03:26:30.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e259c700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40ad330 0x7f05e415a030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f05cc007000 tx=0x7f05cc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:30.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05cc010580 con 0x7f05e40ad330 2020-07-18T03:26:30.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc0106e0 con 0x7f05e40ad330 2020-07-18T03:26:30.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e4064b80 con 0x7f05e40ad330 2020-07-18T03:26:30.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05cc010a40 con 0x7f05e40ad330 2020-07-18T03:26:30.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.766+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e4066a90 con 0x7f05e40ad330 2020-07-18T03:26:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.767+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e4159d20 con 0x7f05e40ad330 2020-07-18T03:26:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.767+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f05cc010cb0 con 0x7f05e40ad330 2020-07-18T03:26:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.767+0000 7f05da7fc700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f05c00503b0 0x7f05c0052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.767+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f05cc067d50 con 0x7f05e40ad330 2020-07-18T03:26:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.767+0000 7f05e1d9b700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f05c00503b0 0x7f05c0052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.768+0000 7f05e1d9b700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f05c00503b0 0x7f05c0052870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f05d4006f90 tx=0x7f05d4015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.771+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f05cc040400 con 0x7f05e40ad330 2020-07-18T03:26:30.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:30.980+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05e4148a80 con 0x7f05e40ad330 2020-07-18T03:26:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.373+0000 7f05da7fc700 1 -- 172.21.15.47:0/3280942424 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05cc043a20 con 0x7f05e40ad330 2020-07-18T03:26:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f05c00503b0 msgr2=0x7f05c0052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f05c00503b0 0x7f05c0052870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f05d4006f90 tx=0x7f05d4015000).stop 2020-07-18T03:26:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40ad330 msgr2=0x7f05e415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40ad330 0x7f05e415a030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f05cc007000 tx=0x7f05cc008040).stop 2020-07-18T03:26:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 shutdown_connections 2020-07-18T03:26:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f05c00503b0 0x7f05c0052870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f05e40b32c0 0x7f05e40665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 --2- 172.21.15.47:0/3280942424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e40ad330 0x7f05e415a030 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 >> 172.21.15.47:0/3280942424 conn(0x7f05e405f5a0 msgr2=0x7f05e40b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 shutdown_connections 2020-07-18T03:26:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.375+0000 7f05e885e700 1 -- 172.21.15.47:0/3280942424 wait complete. 2020-07-18T03:26:31.394 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-07-18T03:26:31.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.517+0000 7f5fd8cac700 1 Processor -- start 2020-07-18T03:26:31.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.517+0000 7f5fd8cac700 1 -- start start 2020-07-18T03:26:31.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd8cac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fd40b59b0 0x7f5fd40b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:31.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd8cac700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:31.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd8cac700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd4147de0 con 0x7f5fd40ad330 2020-07-18T03:26:31.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd8cac700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd40ae000 con 0x7f5fd40b6300 2020-07-18T03:26:31.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd8cac700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd4148010 con 0x7f5fd40b59b0 2020-07-18T03:26:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fd40b59b0 0x7f5fd40b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fd40b59b0 0x7f5fd40b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59734/0 (socket says 172.21.15.47:59734) 2020-07-18T03:26:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.518+0000 7f5fd1d9b700 1 -- 172.21.15.47:0/4232404619 learned_addr learned my addr 172.21.15.47:0/4232404619 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd2d9d700 1 --2- 172.21.15.47:0/4232404619 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd1d9b700 1 -- 172.21.15.47:0/4232404619 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 msgr2=0x7f5fd40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd1d9b700 1 --2- 172.21.15.47:0/4232404619 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd1d9b700 1 -- 172.21.15.47:0/4232404619 >> v1:172.21.15.18:6789/0 conn(0x7f5fd40ad330 legacy=0x7f5fd4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd1d9b700 1 -- 172.21.15.47:0/4232404619 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40ad710 con 0x7f5fd40b59b0 2020-07-18T03:26:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd2d9d700 1 --2- 172.21.15.47:0/4232404619 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd1d9b700 1 --2- 172.21.15.47:0/4232404619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40b59b0 0x7f5fd40b5e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f5fd40ae4e0 tx=0x7f5fc8006e90).ready entity=mon.1 client_cookie=f82b11d2c695f0b0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd159a700 1 -- 172.21.15.47:0/4232404619 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc8009930 con 0x7f5fd40b59b0 2020-07-18T03:26:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd159a700 1 -- 172.21.15.47:0/4232404619 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc8009a90 con 0x7f5fd40b59b0 2020-07-18T03:26:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd159a700 1 -- 172.21.15.47:0/4232404619 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc8009e30 con 0x7f5fd40b59b0 2020-07-18T03:26:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 -- 172.21.15.47:0/4232404619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40b59b0 msgr2=0x7f5fd40b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/4232404619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40b59b0 0x7f5fd40b5e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f5fd40ae4e0 tx=0x7f5fc8006e90).stop 2020-07-18T03:26:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 -- 172.21.15.47:0/4232404619 shutdown_connections 2020-07-18T03:26:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/4232404619 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/4232404619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40b59b0 0x7f5fd40b5e10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 -- 172.21.15.47:0/4232404619 >> 172.21.15.47:0/4232404619 conn(0x7f5fd405f5a0 msgr2=0x7f5fd405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 -- 172.21.15.47:0/4232404619 shutdown_connections 2020-07-18T03:26:31.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.519+0000 7f5fd8cac700 1 -- 172.21.15.47:0/4232404619 wait complete. 2020-07-18T03:26:31.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.520+0000 7f5fd8cac700 1 Processor -- start 2020-07-18T03:26:31.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd8cac700 1 -- start start 2020-07-18T03:26:31.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd8cac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fd40ad330 0x7f5fd413a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:31.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd8cac700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:31.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd8cac700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd413ca50 con 0x7f5fd4146e80 2020-07-18T03:26:31.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd8cac700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd413aa30 con 0x7f5fd40b6300 2020-07-18T03:26:31.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd8cac700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd413bbb0 con 0x7f5fd40ad330 2020-07-18T03:26:31.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fd40ad330 0x7f5fd413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:31.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5fd40ad330 0x7f5fd413a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59736/0 (socket says 172.21.15.47:59736) 2020-07-18T03:26:31.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd259c700 1 -- 172.21.15.47:0/998927830 learned_addr learned my addr 172.21.15.47:0/998927830 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:31.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.521+0000 7f5fd1d9b700 1 --2- 172.21.15.47:0/998927830 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:31.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd259c700 1 -- 172.21.15.47:0/998927830 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 msgr2=0x7f5fd40b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:31.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd259c700 1 --2- 172.21.15.47:0/998927830 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:31.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd259c700 1 -- 172.21.15.47:0/998927830 >> v1:172.21.15.18:6789/0 conn(0x7f5fd4146e80 legacy=0x7f5fd4143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:31.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd259c700 1 -- 172.21.15.47:0/998927830 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd413bd20 con 0x7f5fd40ad330 2020-07-18T03:26:31.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd1d9b700 1 --2- 172.21.15.47:0/998927830 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:31.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd259c700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40ad330 0x7f5fd413a540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f5fcc0022d0 tx=0x7f5fcc007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:31.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fcc00a7b0 con 0x7f5fd40ad330 2020-07-18T03:26:31.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc00a910 con 0x7f5fd40ad330 2020-07-18T03:26:31.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd40b9c10 con 0x7f5fd40ad330 2020-07-18T03:26:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fcc00ac70 con 0x7f5fd40ad330 2020-07-18T03:26:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.522+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd413c0e0 con 0x7f5fd40ad330 2020-07-18T03:26:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.523+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd405b120 con 0x7f5fd40ad330 2020-07-18T03:26:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.523+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5fcc021070 con 0x7f5fd40ad330 2020-07-18T03:26:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.523+0000 7f5fc27fc700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5fa8050010 0x7f5fa80524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.523+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f5fcc06a190 con 0x7f5fd40ad330 2020-07-18T03:26:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.523+0000 7f5fd1d9b700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5fa8050010 0x7f5fa80524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.524+0000 7f5fd1d9b700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5fa8050010 0x7f5fa80524d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5fc801a000 tx=0x7f5fc8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:31.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.527+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5fcc0428c0 con 0x7f5fd40ad330 2020-07-18T03:26:31.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:31.687+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fd40690d0 con 0x7f5fd40ad330 2020-07-18T03:26:32.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.377+0000 7f5fc27fc700 1 -- 172.21.15.47:0/998927830 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5fcc015070 con 0x7f5fd40ad330 2020-07-18T03:26:32.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5fa8050010 msgr2=0x7f5fa80524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:32.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5fa8050010 0x7f5fa80524d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5fc801a000 tx=0x7f5fc8015000).stop 2020-07-18T03:26:32.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40ad330 msgr2=0x7f5fd413a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:32.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40ad330 0x7f5fd413a540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f5fcc0022d0 tx=0x7f5fcc007f90).stop 2020-07-18T03:26:32.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 shutdown_connections 2020-07-18T03:26:32.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5fa8050010 0x7f5fa80524d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/998927830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fd40ad330 0x7f5fd413a540 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 --2- 172.21.15.47:0/998927830 >> v2:172.21.15.18:3300/0 conn(0x7f5fd40b6300 0x7f5fd40b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 >> 172.21.15.47:0/998927830 conn(0x7f5fd405f5a0 msgr2=0x7f5fd4148c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:32.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 shutdown_connections 2020-07-18T03:26:32.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.380+0000 7f5fd8cac700 1 -- 172.21.15.47:0/998927830 wait complete. 2020-07-18T03:26:32.388 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2020-07-18T03:26:32.394 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-07-18T03:26:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.521+0000 7f341b8d4700 1 Processor -- start 2020-07-18T03:26:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.522+0000 7f341b8d4700 1 -- start start 2020-07-18T03:26:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.522+0000 7f341b8d4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3414148570 0x7f3414148a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f341b8d4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f341414b430 0x7f341414b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f341b8d4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140b5bb0 con 0x7f34140b68b0 2020-07-18T03:26:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f341b8d4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3414148f10 con 0x7f3414148570 2020-07-18T03:26:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f341b8d4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341414f960 con 0x7f341414b430 2020-07-18T03:26:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3414148570 0x7f3414148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3414148570 0x7f3414148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59744/0 (socket says 172.21.15.47:59744) 2020-07-18T03:26:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 -- 172.21.15.47:0/3245899012 learned_addr learned my addr 172.21.15.47:0/3245899012 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3419e71700 1 --2- 172.21.15.47:0/3245899012 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f341414b430 0x7f341414b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 -- 172.21.15.47:0/3245899012 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f341414b430 msgr2=0x7f341414b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 --2- 172.21.15.47:0/3245899012 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f341414b430 0x7f341414b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 -- 172.21.15.47:0/3245899012 >> v1:172.21.15.18:6789/0 conn(0x7f34140b68b0 legacy=0x7f34140b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 -- 172.21.15.47:0/3245899012 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341414ef80 con 0x7f3414148570 2020-07-18T03:26:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3418e6f700 1 --2- 172.21.15.47:0/3245899012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414148570 0x7f3414148a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f3410002f30 tx=0x7f3410006e90).ready entity=mon.1 client_cookie=d2e10d6d1ef0114e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.523+0000 7f3403fff700 1 -- 172.21.15.47:0/3245899012 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3410009970 con 0x7f3414148570 2020-07-18T03:26:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f3403fff700 1 -- 172.21.15.47:0/3245899012 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410009ad0 con 0x7f3414148570 2020-07-18T03:26:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f3403fff700 1 -- 172.21.15.47:0/3245899012 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3410009e30 con 0x7f3414148570 2020-07-18T03:26:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 -- 172.21.15.47:0/3245899012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414148570 msgr2=0x7f3414148a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 --2- 172.21.15.47:0/3245899012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414148570 0x7f3414148a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f3410002f30 tx=0x7f3410006e90).stop 2020-07-18T03:26:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 -- 172.21.15.47:0/3245899012 shutdown_connections 2020-07-18T03:26:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 --2- 172.21.15.47:0/3245899012 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f341414b430 0x7f341414b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 --2- 172.21.15.47:0/3245899012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414148570 0x7f3414148a20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 -- 172.21.15.47:0/3245899012 >> 172.21.15.47:0/3245899012 conn(0x7f341405f5a0 msgr2=0x7f341405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 -- 172.21.15.47:0/3245899012 shutdown_connections 2020-07-18T03:26:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.524+0000 7f341b8d4700 1 -- 172.21.15.47:0/3245899012 wait complete. 2020-07-18T03:26:32.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.525+0000 7f341b8d4700 1 Processor -- start 2020-07-18T03:26:32.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.525+0000 7f341b8d4700 1 -- start start 2020-07-18T03:26:32.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f341b8d4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f34140b68b0 0x7f341414cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f341b8d4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 0x7f34141429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f341b8d4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140b0950 con 0x7f341414b430 2020-07-18T03:26:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f341b8d4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140b0bf0 con 0x7f34140b68b0 2020-07-18T03:26:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f341b8d4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34140b0e90 con 0x7f34140b04b0 2020-07-18T03:26:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f3419e71700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 0x7f34141429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f3419670700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f34140b68b0 0x7f341414cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f3419e71700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 0x7f34141429f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:52352/0 (socket says 172.21.15.47:52352) 2020-07-18T03:26:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.526+0000 7f3419e71700 1 -- 172.21.15.47:0/2665388179 learned_addr learned my addr 172.21.15.47:0/2665388179 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f3419e71700 1 -- 172.21.15.47:0/2665388179 >> v2:172.21.15.18:3300/0 conn(0x7f34140b68b0 msgr2=0x7f341414cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f3419e71700 1 --2- 172.21.15.47:0/2665388179 >> v2:172.21.15.18:3300/0 conn(0x7f34140b68b0 0x7f341414cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f3419e71700 1 -- 172.21.15.47:0/2665388179 >> v1:172.21.15.18:6789/0 conn(0x7f341414b430 legacy=0x7f34140b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f3419e71700 1 -- 172.21.15.47:0/2665388179 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414143740 con 0x7f34140b04b0 2020-07-18T03:26:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f3419670700 1 --2- 172.21.15.47:0/2665388179 >> v2:172.21.15.18:3300/0 conn(0x7f34140b68b0 0x7f341414cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f3419e71700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 0x7f34141429f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f340c00a000 tx=0x7f340c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c009a90 con 0x7f34140b04b0 2020-07-18T03:26:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c009bf0 con 0x7f34140b04b0 2020-07-18T03:26:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34140b7180 con 0x7f34140b04b0 2020-07-18T03:26:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c013480 con 0x7f34140b04b0 2020-07-18T03:26:32.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.527+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3414142ee0 con 0x7f34140b04b0 2020-07-18T03:26:32.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.528+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34140b41c0 con 0x7f34140b04b0 2020-07-18T03:26:32.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.529+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f340c013680 con 0x7f34140b04b0 2020-07-18T03:26:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.529+0000 7f34017fa700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33ec04ff00 0x7f33ec0523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.529+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f340c068700 con 0x7f34140b04b0 2020-07-18T03:26:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.529+0000 7f3419670700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33ec04ff00 0x7f33ec0523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.529+0000 7f3419670700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33ec04ff00 0x7f33ec0523c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f340400d000 tx=0x7f3404009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.533+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f340c044e60 con 0x7f34140b04b0 2020-07-18T03:26:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:32.691+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f341405b120 con 0x7f34140b04b0 2020-07-18T03:26:33.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.380+0000 7f34017fa700 1 -- 172.21.15.47:0/2665388179 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7f340c0449a0 con 0x7f34140b04b0 2020-07-18T03:26:33.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33ec04ff00 msgr2=0x7f33ec0523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33ec04ff00 0x7f33ec0523c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f340400d000 tx=0x7f3404009040).stop 2020-07-18T03:26:33.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 msgr2=0x7f34141429f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 0x7f34141429f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f340c00a000 tx=0x7f340c008040).stop 2020-07-18T03:26:33.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 shutdown_connections 2020-07-18T03:26:33.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33ec04ff00 0x7f33ec0523c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 --2- 172.21.15.47:0/2665388179 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f34140b04b0 0x7f34141429f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 --2- 172.21.15.47:0/2665388179 >> v2:172.21.15.18:3300/0 conn(0x7f34140b68b0 0x7f341414cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.382+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 >> 172.21.15.47:0/2665388179 conn(0x7f341405f5a0 msgr2=0x7f34140b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:33.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.384+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 shutdown_connections 2020-07-18T03:26:33.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.384+0000 7f341b8d4700 1 -- 172.21.15.47:0/2665388179 wait complete. 2020-07-18T03:26:33.391 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2020-07-18T03:26:33.400 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:26:33.400 INFO:tasks.ceph.mds.a:Stopping old one... 2020-07-18T03:26:33.400 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:26:33.401 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:26:33.401 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:26:33.401 INFO:tasks.ceph.mds.b:Stopping old one... 2020-07-18T03:26:33.402 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:26:33.402 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:26:33.418 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:26:33.419 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:26:33.423 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:26:33.423 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:26:33.452 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:26:33.453 DEBUG:teuthology.parallel:result is None 2020-07-18T03:26:33.465 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:26:33.466 DEBUG:teuthology.parallel:result is None 2020-07-18T03:26:33.466 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-07-18T03:26:33.542 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:26:33.552 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:26:33.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.594+0000 7f4752d63700 1 Processor -- start 2020-07-18T03:26:33.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.594+0000 7f4752d63700 1 -- start start 2020-07-18T03:26:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f4752d63700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f474c0b7110 0x7f474c144210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f4752d63700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c144b50 con 0x7f474c0b44c0 2020-07-18T03:26:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f4752d63700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c147450 con 0x7f474c144700 2020-07-18T03:26:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f4752d63700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c1476f0 con 0x7f474c0b7110 2020-07-18T03:26:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f4751300700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f474c144700 0x7f474c0616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40862/0 (socket says 172.21.15.47:40862) 2020-07-18T03:26:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f4751300700 1 -- 172.21.15.47:0/1468718894 learned_addr learned my addr 172.21.15.47:0/1468718894 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474bfff700 1 --2- 172.21.15.47:0/1468718894 >> v2:172.21.15.18:3300/0 conn(0x7f474c0b7110 0x7f474c144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009918191 0 0) 0x7f474c147450 con 0x7f474c144700 2020-07-18T03:26:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4730003240 con 0x7f474c144700 2020-07-18T03:26:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198530475 0 0) 0x7f474c144b50 con 0x7f474c0b44c0 2020-07-18T03:26:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f474c147450 con 0x7f474c0b44c0 2020-07-18T03:26:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1655272408 0 0) 0x7f4730003240 con 0x7f474c144700 2020-07-18T03:26:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 >> v2:172.21.15.18:3300/0 conn(0x7f474c0b7110 msgr2=0x7f474c144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 --2- 172.21.15.47:0/1468718894 >> v2:172.21.15.18:3300/0 conn(0x7f474c0b7110 0x7f474c144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 >> v1:172.21.15.18:6789/0 conn(0x7f474c0b44c0 legacy=0x7f474c0be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.595+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c147990 con 0x7f474c144700 2020-07-18T03:26:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4744001a50 con 0x7f474c144700 2020-07-18T03:26:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 >> v1:172.21.15.47:6789/0 conn(0x7f474c144700 legacy=0x7f474c0616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47300038c0 0x7f4730003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47300044e0 0x7f4730004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4730004e50 0x7f473000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c1476f0 con 0x7f47300038c0 2020-07-18T03:26:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473000bc20 con 0x7f47300044e0 2020-07-18T03:26:33.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473000bdb0 con 0x7f4730004e50 2020-07-18T03:26:33.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47300044e0 0x7f4730004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4751300700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4730004e50 0x7f473000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4750aff700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47300038c0 0x7f4730003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 -- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4730004e50 msgr2=0x7f473000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4730004e50 0x7f473000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 -- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47300038c0 msgr2=0x7f4730003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47300038c0 0x7f4730003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4751300700 1 -- 172.21.15.47:0/1468718894 reap_dead start 2020-07-18T03:26:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 -- 172.21.15.47:0/1468718894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730003240 con 0x7f47300044e0 2020-07-18T03:26:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474bfff700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47300044e0 0x7f4730004960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f473800c000 tx=0x7f4738008040).ready entity=mon.1 client_cookie=45e017f6df0b5f06 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f473800b2d0 con 0x7f47300044e0 2020-07-18T03:26:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473800e8a0 con 0x7f47300044e0 2020-07-18T03:26:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f474b7fe700 1 -- 172.21.15.47:0/1468718894 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f473800ec00 con 0x7f47300044e0 2020-07-18T03:26:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 -- 172.21.15.47:0/1468718894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47300044e0 msgr2=0x7f4730004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47300044e0 0x7f4730004960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f473800c000 tx=0x7f4738008040).stop 2020-07-18T03:26:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 -- 172.21.15.47:0/1468718894 shutdown_connections 2020-07-18T03:26:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 --2- 172.21.15.47:0/1468718894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47300044e0 0x7f4730004960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 -- 172.21.15.47:0/1468718894 >> 172.21.15.47:0/1468718894 conn(0x7f474c060070 msgr2=0x7f474c0bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 -- 172.21.15.47:0/1468718894 shutdown_connections 2020-07-18T03:26:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.596+0000 7f4752d63700 1 -- 172.21.15.47:0/1468718894 wait complete. 2020-07-18T03:26:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.598+0000 7f4752d63700 1 Processor -- start 2020-07-18T03:26:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.598+0000 7f4752d63700 1 -- start start 2020-07-18T03:26:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4752d63700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f474c155f30 0x7f474c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4752d63700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c0b1970 con 0x7f474c141c30 2020-07-18T03:26:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4752d63700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c0ad020 con 0x7f474c0ae4d0 2020-07-18T03:26:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4752d63700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c0b1790 con 0x7f474c155f30 2020-07-18T03:26:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f474bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f474c0ae4d0 0x7f474c155820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40874/0 (socket says 172.21.15.47:40874) 2020-07-18T03:26:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f474bfff700 1 -- 172.21.15.47:0/3920372273 learned_addr learned my addr 172.21.15.47:0/3920372273 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4751300700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f474c155f30 0x7f474c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207843985 0 0) 0x7f474c0ad020 con 0x7f474c0ae4d0 2020-07-18T03:26:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4724003240 con 0x7f474c0ae4d0 2020-07-18T03:26:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937407199 0 0) 0x7f474c0b1970 con 0x7f474c141c30 2020-07-18T03:26:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f474c0ad020 con 0x7f474c141c30 2020-07-18T03:26:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4751300700 1 -- 172.21.15.47:0/3920372273 >> v1:172.21.15.47:6789/0 conn(0x7f474c0ae4d0 legacy=0x7f474c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4751300700 1 -- 172.21.15.47:0/3920372273 >> v1:172.21.15.18:6789/0 conn(0x7f474c141c30 legacy=0x7f474c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4751300700 1 -- 172.21.15.47:0/3920372273 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c142010 con 0x7f474c155f30 2020-07-18T03:26:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.599+0000 7f4751300700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f474c155f30 0x7f474c15f8a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4744000b20 tx=0x7f474c067eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.600+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474400fc30 con 0x7f474c155f30 2020-07-18T03:26:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.600+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474400fd90 con 0x7f474c155f30 2020-07-18T03:26:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.600+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c0b1380 con 0x7f474c155f30 2020-07-18T03:26:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.600+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4744012540 con 0x7f474c155f30 2020-07-18T03:26:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.600+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c15fd90 con 0x7f474c155f30 2020-07-18T03:26:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.601+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474c0b9ea0 con 0x7f474c155f30 2020-07-18T03:26:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.601+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4744001570 con 0x7f474c155f30 2020-07-18T03:26:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.601+0000 7f4748ff9700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47240504c0 0x7f4724052980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.601+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4744067100 con 0x7f474c155f30 2020-07-18T03:26:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.601+0000 7f4750aff700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47240504c0 0x7f4724052980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.601+0000 7f4750aff700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47240504c0 0x7f4724052980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f474000b000 tx=0x7f4740007b70).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.605+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f474403faf0 con 0x7f474c155f30 2020-07-18T03:26:33.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:33.779+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f474c057450 con 0x7f474c155f30 2020-07-18T03:26:34.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.577+0000 7f4748ff9700 1 -- 172.21.15.47:0/3920372273 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4744019080 con 0x7f474c155f30 2020-07-18T03:26:34.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47240504c0 msgr2=0x7f4724052980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47240504c0 0x7f4724052980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f474000b000 tx=0x7f4740007b70).stop 2020-07-18T03:26:34.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f474c155f30 msgr2=0x7f474c15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f474c155f30 0x7f474c15f8a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4744000b20 tx=0x7f474c067eb0).stop 2020-07-18T03:26:34.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 shutdown_connections 2020-07-18T03:26:34.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f474c155f30 0x7f474c15f8a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 --2- 172.21.15.47:0/3920372273 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47240504c0 0x7f4724052980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 >> 172.21.15.47:0/3920372273 conn(0x7f474c060070 msgr2=0x7f474c0b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 shutdown_connections 2020-07-18T03:26:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.579+0000 7f4752d63700 1 -- 172.21.15.47:0/3920372273 wait complete. 2020-07-18T03:26:34.582 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2020-07-18T03:26:34.594 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2020-07-18T03:26:34.595 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-07-18T03:26:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.716+0000 7f9d165b2700 1 Processor -- start 2020-07-18T03:26:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.716+0000 7f9d165b2700 1 -- start start 2020-07-18T03:26:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d165b2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9d100ad100 0x7f9d100b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d165b2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d10147a00 0x7f9d10147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d165b2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d10147330 con 0x7f9d1005f5a0 2020-07-18T03:26:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d165b2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100ae530 con 0x7f9d100ad100 2020-07-18T03:26:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d165b2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100b57e0 con 0x7f9d10147a00 2020-07-18T03:26:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d0f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d10147a00 0x7f9d10147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d14b4f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d1005f5a0 0x7f9d1005f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40880/0 (socket says 172.21.15.47:40880) 2020-07-18T03:26:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d14b4f700 1 -- 172.21.15.47:0/774936118 learned_addr learned my addr 172.21.15.47:0/774936118 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/774936118 >> v2:172.21.15.47:3300/0 conn(0x7f9d10147a00 0x7f9d10147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59792/0 (socket says 172.21.15.47:59792) 2020-07-18T03:26:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.717+0000 7f9d0ffff700 1 --2- 172.21.15.47:0/774936118 >> v2:172.21.15.18:3300/0 conn(0x7f9d100ad100 0x7f9d100b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0f7fe700 1 -- 172.21.15.47:0/774936118 >> v2:172.21.15.18:3300/0 conn(0x7f9d100ad100 msgr2=0x7f9d100b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/774936118 >> v2:172.21.15.18:3300/0 conn(0x7f9d100ad100 0x7f9d100b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0f7fe700 1 -- 172.21.15.47:0/774936118 >> v1:172.21.15.47:6789/0 conn(0x7f9d1005f5a0 legacy=0x7f9d1005f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0f7fe700 1 -- 172.21.15.47:0/774936118 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100b5a80 con 0x7f9d10147a00 2020-07-18T03:26:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0effd700 1 -- 172.21.15.47:0/774936118 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224533565 0 0) 0x7f9d10147330 con 0x7f9d1005f5a0 2020-07-18T03:26:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/774936118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d10147a00 0x7f9d10147e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f9d04002f30 tx=0x7f9d04006dd0).ready entity=mon.1 client_cookie=605422cfe623e988 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0effd700 1 -- 172.21.15.47:0/774936118 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d040097b0 con 0x7f9d10147a00 2020-07-18T03:26:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0effd700 1 -- 172.21.15.47:0/774936118 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04009910 con 0x7f9d10147a00 2020-07-18T03:26:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d0effd700 1 -- 172.21.15.47:0/774936118 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d04009c70 con 0x7f9d10147a00 2020-07-18T03:26:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 -- 172.21.15.47:0/774936118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d10147a00 msgr2=0x7f9d10147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 --2- 172.21.15.47:0/774936118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d10147a00 0x7f9d10147e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f9d04002f30 tx=0x7f9d04006dd0).stop 2020-07-18T03:26:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 -- 172.21.15.47:0/774936118 shutdown_connections 2020-07-18T03:26:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 --2- 172.21.15.47:0/774936118 >> v2:172.21.15.18:3300/0 conn(0x7f9d100ad100 0x7f9d100b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 --2- 172.21.15.47:0/774936118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d10147a00 0x7f9d10147e40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 -- 172.21.15.47:0/774936118 >> 172.21.15.47:0/774936118 conn(0x7f9d100b9340 msgr2=0x7f9d100b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 -- 172.21.15.47:0/774936118 shutdown_connections 2020-07-18T03:26:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.718+0000 7f9d165b2700 1 -- 172.21.15.47:0/774936118 wait complete. 2020-07-18T03:26:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.720+0000 7f9d165b2700 1 Processor -- start 2020-07-18T03:26:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.720+0000 7f9d165b2700 1 -- start start 2020-07-18T03:26:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d165b2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9d1005f5a0 0x7f9d1014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d165b2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9d10144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d165b2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1014d200 con 0x7f9d100ad100 2020-07-18T03:26:34.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d165b2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1014d4a0 con 0x7f9d1005f5a0 2020-07-18T03:26:34.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d165b2700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1014d740 con 0x7f9d1014dbe0 2020-07-18T03:26:34.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d0f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d100ad100 0x7f9d100ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40886/0 (socket says 172.21.15.47:40886) 2020-07-18T03:26:34.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d0f7fe700 1 -- 172.21.15.47:0/3459054235 learned_addr learned my addr 172.21.15.47:0/3459054235 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:34.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195497001 0 0) 0x7f9d1014d200 con 0x7f9d100ad100 2020-07-18T03:26:34.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d0ffff700 1 --2- 172.21.15.47:0/3459054235 >> v2:172.21.15.18:3300/0 conn(0x7f9d1005f5a0 0x7f9d1014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce8003240 con 0x7f9d100ad100 2020-07-18T03:26:34.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d14b4f700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9d10144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 426362584 0 0) 0x7f9ce8003240 con 0x7f9d100ad100 2020-07-18T03:26:34.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d1014d200 con 0x7f9d100ad100 2020-07-18T03:26:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f9d040097b0 con 0x7f9d100ad100 2020-07-18T03:26:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 msgr2=0x7f9d10144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9d10144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 >> v2:172.21.15.18:3300/0 conn(0x7f9d1005f5a0 msgr2=0x7f9d1014c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 --2- 172.21.15.47:0/3459054235 >> v2:172.21.15.18:3300/0 conn(0x7f9d1005f5a0 0x7f9d1014c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 >> v1:172.21.15.47:6789/0 conn(0x7f9d100ad100 legacy=0x7f9d100ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8003800 0x7f9ce8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9ce80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce80049d0 0x7f9ce8004e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1014d4a0 con 0x7f9ce80049d0 2020-07-18T03:26:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1014d740 con 0x7f9ce8003800 2020-07-18T03:26:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80041e0 con 0x7f9d1014dbe0 2020-07-18T03:26:34.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.721+0000 7f9d0ffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8003800 0x7f9ce8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9ce80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d14b4f700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce80049d0 0x7f9ce8004e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0ffff700 1 -- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 msgr2=0x7f9ce80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0ffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9ce80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0ffff700 1 -- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce80049d0 msgr2=0x7f9ce8004e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0ffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce80049d0 0x7f9ce8004e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0ffff700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1014ccc0 con 0x7f9ce8003800 2020-07-18T03:26:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9d1014dbe0 0x7f9ce80044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1014ab60 con 0x7f9ce8003800 2020-07-18T03:26:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1014e230 con 0x7f9ce8003800 2020-07-18T03:26:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9d0ffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8003800 0x7f9ce8003c20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f9d0000b000 tx=0x7f9d00006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d00018070 con 0x7f9ce8003800 2020-07-18T03:26:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0000fca0 con 0x7f9ce8003800 2020-07-18T03:26:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.722+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d00014440 con 0x7f9ce8003800 2020-07-18T03:26:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.723+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1014d070 con 0x7f9ce8003800 2020-07-18T03:26:34.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.723+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9d000145a0 con 0x7f9ce8003800 2020-07-18T03:26:34.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.723+0000 7f9ceffff700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9ce8058170 0x7f9ce805a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:34.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.723+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9d00068300 con 0x7f9ce8003800 2020-07-18T03:26:34.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.723+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9ce8058170 0x7f9ce805a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:34.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.724+0000 7f9d0f7fe700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9ce8058170 0x7f9ce805a630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f9d04019000 tx=0x7f9d04015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:34.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.727+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9d00040d70 con 0x7f9ce8003800 2020-07-18T03:26:34.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.941+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9d1014d910 con 0x7f9ce8003800 2020-07-18T03:26:34.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.941+0000 7f9ceffff700 1 -- 172.21.15.47:0/3459054235 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+5917 (secure 0 0 0) 0x7f9d00044390 con 0x7f9ce8003800 2020-07-18T03:26:34.943 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:34.944 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:26:34.944 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2020-07-18T03:26:34.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.944 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2020-07-18T03:26:34.945 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g==", 2020-07-18T03:26:34.945 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.945 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:26:34.945 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.946 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.946 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.946 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.947 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2020-07-18T03:26:34.947 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ==", 2020-07-18T03:26:34.948 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.948 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:26:34.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.951 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2020-07-18T03:26:34.951 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA==", 2020-07-18T03:26:34.951 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.954 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2020-07-18T03:26:34.954 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ==", 2020-07-18T03:26:34.955 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.955 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.956 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2020-07-18T03:26:34.956 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA==", 2020-07-18T03:26:34.956 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.957 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.958 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2020-07-18T03:26:34.958 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ==", 2020-07-18T03:26:34.958 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.958 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.959 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.959 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2020-07-18T03:26:34.960 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ==", 2020-07-18T03:26:34.960 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.960 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.960 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.960 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2020-07-18T03:26:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA==", 2020-07-18T03:26:34.962 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.962 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.962 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.962 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.963 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2020-07-18T03:26:34.963 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg==", 2020-07-18T03:26:34.963 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.965 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2020-07-18T03:26:34.965 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg==", 2020-07-18T03:26:34.965 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.966 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.966 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2020-07-18T03:26:34.967 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw==", 2020-07-18T03:26:34.967 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.967 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:26:34.967 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2020-07-18T03:26:34.967 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2020-07-18T03:26:34.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2020-07-18T03:26:34.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.968 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2020-07-18T03:26:34.968 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g==", 2020-07-18T03:26:34.969 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:26:34.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2020-07-18T03:26:34.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2020-07-18T03:26:34.969 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2020-07-18T03:26:34.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.970 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2020-07-18T03:26:34.970 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w==", 2020-07-18T03:26:34.971 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:26:34.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2020-07-18T03:26:34.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2020-07-18T03:26:34.971 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2020-07-18T03:26:34.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.972 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2020-07-18T03:26:34.972 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog==", 2020-07-18T03:26:34.972 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.973 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:26:34.973 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2020-07-18T03:26:34.973 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2020-07-18T03:26:34.973 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2020-07-18T03:26:34.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.974 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2020-07-18T03:26:34.974 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfi9GyDxAACUH4y/92DOhKqZyh9ejEnA==", 2020-07-18T03:26:34.974 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.975 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:26:34.975 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:26:34.975 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:26:34.975 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.977 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mds", 2020-07-18T03:26:34.978 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCkaxJfyqwfFxAAO8fvTAoKAFWg25V+PJCmtw==", 2020-07-18T03:26:34.978 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.978 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mds" 2020-07-18T03:26:34.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.979 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mgr", 2020-07-18T03:26:34.979 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCkaxJfUbgfFxAA7JT7XAIOId4fPzFU3gwlcQ==", 2020-07-18T03:26:34.979 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.979 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mgr" 2020-07-18T03:26:34.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.980 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-osd", 2020-07-18T03:26:34.980 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCkaxJfycQfFxAAkLZ/h1a2e41K2RdSc7yenA==", 2020-07-18T03:26:34.981 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.981 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-osd" 2020-07-18T03:26:34.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.981 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd", 2020-07-18T03:26:34.982 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCkaxJfhM0fFxAAmmd67khfDhK7iE8OkdJUug==", 2020-07-18T03:26:34.982 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.982 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd" 2020-07-18T03:26:34.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.983 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-07-18T03:26:34.983 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCkaxJfEtYfFxAAl2IwJpknYHbJrnNLc0Qtng==", 2020-07-18T03:26:34.983 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.983 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-07-18T03:26:34.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.984 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rgw", 2020-07-18T03:26:34.984 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCkaxJfMN8fFxAAlFDU5sqPgGeYrNSELGli9g==", 2020-07-18T03:26:34.985 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rgw" 2020-07-18T03:26:34.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.986 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2020-07-18T03:26:34.986 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ==", 2020-07-18T03:26:34.986 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.986 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:26:34.987 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:26:34.987 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:34.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:34.987 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2020-07-18T03:26:34.988 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw==", 2020-07-18T03:26:34.988 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:26:34.988 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:26:34.988 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:26:34.988 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:26:34.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:34.989 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:26:34.989 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:26:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9ce8058170 msgr2=0x7f9ce805a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9ce8058170 0x7f9ce805a630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f9d04019000 tx=0x7f9d04015000).stop 2020-07-18T03:26:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8003800 msgr2=0x7f9ce8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 --2- 172.21.15.47:0/3459054235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8003800 0x7f9ce8003c20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f9d0000b000 tx=0x7f9d00006dd0).stop 2020-07-18T03:26:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d14b4f700 1 -- 172.21.15.47:0/3459054235 reap_dead start 2020-07-18T03:26:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 shutdown_connections 2020-07-18T03:26:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 >> 172.21.15.47:0/3459054235 conn(0x7f9d100b9340 msgr2=0x7f9d100b6360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 shutdown_connections 2020-07-18T03:26:34.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:34.943+0000 7f9d165b2700 1 -- 172.21.15.47:0/3459054235 wait complete. 2020-07-18T03:26:34.993 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mds 2020-07-18T03:26:35.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd87ad58700 1 Processor -- start 2020-07-18T03:26:35.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd87ad58700 1 -- start start 2020-07-18T03:26:35.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd87ad58700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd87405ffb0 0x7fd8740625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd87ad58700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8740b5830 con 0x7fd8740aea60 2020-07-18T03:26:35.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd87ad58700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd874146da0 con 0x7fd8741471d0 2020-07-18T03:26:35.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd87ad58700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd874147d70 con 0x7fd87405ffb0 2020-07-18T03:26:35.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd8792f5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd8741471d0 0x7fd874147660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40902/0 (socket says 172.21.15.47:40902) 2020-07-18T03:26:35.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.109+0000 7fd8792f5700 1 -- 172.21.15.47:0/3756573762 learned_addr learned my addr 172.21.15.47:0/3756573762 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:35.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd878af4700 1 --2- 172.21.15.47:0/3756573762 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd87405ffb0 0x7fd8740625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651759265 0 0) 0x7fd874146da0 con 0x7fd8741471d0 2020-07-18T03:26:35.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd860003240 con 0x7fd8741471d0 2020-07-18T03:26:35.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207153391 0 0) 0x7fd8740b5830 con 0x7fd8740aea60 2020-07-18T03:26:35.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd874146da0 con 0x7fd8740aea60 2020-07-18T03:26:35.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd878af4700 1 -- 172.21.15.47:0/3756573762 >> v1:172.21.15.47:6789/0 conn(0x7fd8741471d0 legacy=0x7fd874147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd878af4700 1 -- 172.21.15.47:0/3756573762 >> v1:172.21.15.18:6789/0 conn(0x7fd8740aea60 legacy=0x7fd874142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd878af4700 1 -- 172.21.15.47:0/3756573762 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740b5a60 con 0x7fd87405ffb0 2020-07-18T03:26:35.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd878af4700 1 --2- 172.21.15.47:0/3756573762 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd87405ffb0 0x7fd8740625d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd864002f30 tx=0x7fd864006dd0).ready entity=mon.2 client_cookie=56dfcf89a3e6b9f4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8640096e0 con 0x7fd87405ffb0 2020-07-18T03:26:35.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864009840 con 0x7fd87405ffb0 2020-07-18T03:26:35.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd8737fe700 1 -- 172.21.15.47:0/3756573762 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd864009ba0 con 0x7fd87405ffb0 2020-07-18T03:26:35.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 -- 172.21.15.47:0/3756573762 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd87405ffb0 msgr2=0x7fd8740625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 --2- 172.21.15.47:0/3756573762 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd87405ffb0 0x7fd8740625d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd864002f30 tx=0x7fd864006dd0).stop 2020-07-18T03:26:35.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 -- 172.21.15.47:0/3756573762 shutdown_connections 2020-07-18T03:26:35.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 --2- 172.21.15.47:0/3756573762 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd87405ffb0 0x7fd8740625d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 -- 172.21.15.47:0/3756573762 >> 172.21.15.47:0/3756573762 conn(0x7fd8740b93c0 msgr2=0x7fd8740b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:35.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 -- 172.21.15.47:0/3756573762 shutdown_connections 2020-07-18T03:26:35.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.110+0000 7fd87ad58700 1 -- 172.21.15.47:0/3756573762 wait complete. 2020-07-18T03:26:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.112+0000 7fd87ad58700 1 Processor -- start 2020-07-18T03:26:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.112+0000 7fd87ad58700 1 -- start start 2020-07-18T03:26:35.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd87ad58700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8741471d0 0x7fd874145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd87ad58700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8741435d0 con 0x7fd874143fb0 2020-07-18T03:26:35.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd87ad58700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd874143870 con 0x7fd8740aea60 2020-07-18T03:26:35.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd87ad58700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd874143b10 con 0x7fd8741471d0 2020-07-18T03:26:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd873fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8741471d0 0x7fd874145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd878af4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd8740aea60 0x7fd8740b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40904/0 (socket says 172.21.15.47:40904) 2020-07-18T03:26:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd878af4700 1 -- 172.21.15.47:0/1017390840 learned_addr learned my addr 172.21.15.47:0/1017390840 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd873fff700 1 -- 172.21.15.47:0/1017390840 >> v1:172.21.15.47:6789/0 conn(0x7fd8740aea60 legacy=0x7fd8740b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd873fff700 1 -- 172.21.15.47:0/1017390840 >> v1:172.21.15.18:6789/0 conn(0x7fd874143fb0 legacy=0x7fd87413ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926849109 0 0) 0x7fd874143870 con 0x7fd8740aea60 2020-07-18T03:26:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd873fff700 1 -- 172.21.15.47:0/1017390840 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87413b470 con 0x7fd8741471d0 2020-07-18T03:26:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd873fff700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8741471d0 0x7fd874145610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fd8680010d0 tx=0x7fd868007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd86800e4a0 con 0x7fd8741471d0 2020-07-18T03:26:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86800e600 con 0x7fd8741471d0 2020-07-18T03:26:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87413b6c0 con 0x7fd8741471d0 2020-07-18T03:26:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd86800e960 con 0x7fd8741471d0 2020-07-18T03:26:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.113+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd874143100 con 0x7fd8741471d0 2020-07-18T03:26:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.114+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874057450 con 0x7fd8741471d0 2020-07-18T03:26:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.115+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd868019030 con 0x7fd8741471d0 2020-07-18T03:26:35.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.115+0000 7fd870ff9700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd84c04fe10 0x7fd84c0522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.115+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd868066dc0 con 0x7fd8741471d0 2020-07-18T03:26:35.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.115+0000 7fd878af4700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd84c04fe10 0x7fd84c0522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.115+0000 7fd878af4700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd84c04fe10 0x7fd84c0522d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fd86400c000 tx=0x7fd864016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.118+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd86803f890 con 0x7fd8741471d0 2020-07-18T03:26:35.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.334+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd874068190 con 0x7fd8741471d0 2020-07-18T03:26:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.338+0000 7fd870ff9700 1 -- 172.21.15.47:0/1017390840 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd868042eb0 con 0x7fd8741471d0 2020-07-18T03:26:35.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd84c04fe10 msgr2=0x7fd84c0522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd84c04fe10 0x7fd84c0522d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fd86400c000 tx=0x7fd864016000).stop 2020-07-18T03:26:35.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8741471d0 msgr2=0x7fd874145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8741471d0 0x7fd874145610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fd8680010d0 tx=0x7fd868007a00).stop 2020-07-18T03:26:35.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 shutdown_connections 2020-07-18T03:26:35.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd84c04fe10 0x7fd84c0522d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 --2- 172.21.15.47:0/1017390840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8741471d0 0x7fd874145610 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 >> 172.21.15.47:0/1017390840 conn(0x7fd8740b93c0 msgr2=0x7fd87405f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:35.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 shutdown_connections 2020-07-18T03:26:35.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.340+0000 7fd87ad58700 1 -- 172.21.15.47:0/1017390840 wait complete. 2020-07-18T03:26:35.343 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2020-07-18T03:26:35.351 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mgr 2020-07-18T03:26:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.467+0000 7f4f6026d700 1 Processor -- start 2020-07-18T03:26:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.467+0000 7f4f6026d700 1 -- start start 2020-07-18T03:26:35.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.467+0000 7f4f6026d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 0x7f4f580b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.467+0000 7f4f6026d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f58145500 con 0x7f4f58145120 2020-07-18T03:26:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.467+0000 7f4f6026d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f58145770 con 0x7f4f58148e30 2020-07-18T03:26:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.467+0000 7f4f6026d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58146150 con 0x7f4f581492e0 2020-07-18T03:26:35.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d808700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4f58148e30 0x7f4f580b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40914/0 (socket says 172.21.15.47:40914) 2020-07-18T03:26:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d808700 1 -- 172.21.15.47:0/3786185124 learned_addr learned my addr 172.21.15.47:0/3786185124 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5e80a700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 0x7f4f580b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933378068 0 0) 0x7f4f58145770 con 0x7f4f58148e30 2020-07-18T03:26:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f40003240 con 0x7f4f58148e30 2020-07-18T03:26:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4156796431 0 0) 0x7f4f40003240 con 0x7f4f58148e30 2020-07-18T03:26:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 msgr2=0x7f4f580b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 0x7f4f580b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 >> v1:172.21.15.18:6789/0 conn(0x7f4f58145120 legacy=0x7f4f58061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f58145a10 con 0x7f4f58148e30 2020-07-18T03:26:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4f540016d0 con 0x7f4f58148e30 2020-07-18T03:26:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 >> v1:172.21.15.47:6789/0 conn(0x7f4f58148e30 legacy=0x7f4f580b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f40003730 0x7f4f40003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f40004070 0x7f4f40004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 0x7f4f40004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58146150 con 0x7f4f40003730 2020-07-18T03:26:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400050e0 con 0x7f4f40004070 2020-07-18T03:26:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400052a0 con 0x7f4f581492e0 2020-07-18T03:26:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5d808700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f40004070 0x7f4f40004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5e009700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f40003730 0x7f4f40003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.468+0000 7f4f5e80a700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 0x7f4f40004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d808700 1 -- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 msgr2=0x7f4f40004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d808700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f581492e0 0x7f4f40004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d808700 1 -- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f40003730 msgr2=0x7f4f40003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:35.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d808700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f40003730 0x7f4f40003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d808700 1 -- 172.21.15.47:0/3786185124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f40003240 con 0x7f4f40004070 2020-07-18T03:26:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d808700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f40004070 0x7f4f40004540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4f540013e0 tx=0x7f4f54007780).ready entity=mon.1 client_cookie=2a4dc4930612ba2f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f540070d0 con 0x7f4f40004070 2020-07-18T03:26:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54007230 con 0x7f4f40004070 2020-07-18T03:26:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5d007700 1 -- 172.21.15.47:0/3786185124 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f540055f0 con 0x7f4f40004070 2020-07-18T03:26:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f6026d700 1 -- 172.21.15.47:0/3786185124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f40004070 msgr2=0x7f4f40004540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f6026d700 1 --2- 172.21.15.47:0/3786185124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f40004070 0x7f4f40004540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4f540013e0 tx=0x7f4f54007780).stop 2020-07-18T03:26:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f5e80a700 1 -- 172.21.15.47:0/3786185124 reap_dead start 2020-07-18T03:26:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f6026d700 1 -- 172.21.15.47:0/3786185124 shutdown_connections 2020-07-18T03:26:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f6026d700 1 -- 172.21.15.47:0/3786185124 >> 172.21.15.47:0/3786185124 conn(0x7f4f580b93c0 msgr2=0x7f4f580b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f6026d700 1 -- 172.21.15.47:0/3786185124 shutdown_connections 2020-07-18T03:26:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.469+0000 7f4f6026d700 1 -- 172.21.15.47:0/3786185124 wait complete. 2020-07-18T03:26:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f6026d700 1 Processor -- start 2020-07-18T03:26:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f6026d700 1 -- start start 2020-07-18T03:26:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f6026d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4f580630e0 0x7f4f5815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f6026d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f58063a90 con 0x7f4f58141580 2020-07-18T03:26:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f6026d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f58063d30 con 0x7f4f58141960 2020-07-18T03:26:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f6026d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f58063880 con 0x7f4f580630e0 2020-07-18T03:26:35.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f5d808700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4f58141960 0x7f4f58153780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40928/0 (socket says 172.21.15.47:40928) 2020-07-18T03:26:35.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.471+0000 7f4f5d808700 1 -- 172.21.15.47:0/426067824 learned_addr learned my addr 172.21.15.47:0/426067824 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:35.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279622814 0 0) 0x7f4f58063d30 con 0x7f4f58141960 2020-07-18T03:26:35.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f30003240 con 0x7f4f58141960 2020-07-18T03:26:35.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e80a700 1 --2- 172.21.15.47:0/426067824 >> v2:172.21.15.18:3300/0 conn(0x7f4f580630e0 0x7f4f5815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 138915576 0 0) 0x7f4f30003240 con 0x7f4f58141960 2020-07-18T03:26:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f58063d30 con 0x7f4f58141960 2020-07-18T03:26:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4f540070d0 con 0x7f4f58141960 2020-07-18T03:26:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 >> v2:172.21.15.18:3300/0 conn(0x7f4f580630e0 msgr2=0x7f4f5815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 --2- 172.21.15.47:0/426067824 >> v2:172.21.15.18:3300/0 conn(0x7f4f580630e0 0x7f4f5815d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 >> v1:172.21.15.47:6789/0 conn(0x7f4f58141960 legacy=0x7f4f58153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 >> v1:172.21.15.18:6789/0 conn(0x7f4f58141580 legacy=0x7f4f58153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f30003800 0x7f4f30003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f300044e0 0x7f4f30004940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30004e30 0x7f4f3000b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58063880 con 0x7f4f30004e30 2020-07-18T03:26:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f30005310 con 0x7f4f30003800 2020-07-18T03:26:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3000bc20 con 0x7f4f300044e0 2020-07-18T03:26:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f30003800 0x7f4f30003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5d808700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f300044e0 0x7f4f30004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e80a700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30004e30 0x7f4f3000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 -- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f300044e0 msgr2=0x7f4f30004940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f300044e0 0x7f4f30004940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 -- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30004e30 msgr2=0x7f4f3000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30004e30 0x7f4f3000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f58063550 con 0x7f4f30003800 2020-07-18T03:26:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e80a700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30004e30 0x7f4f3000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5d808700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f300044e0 0x7f4f30004940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e80a700 1 -- 172.21.15.47:0/426067824 reap_dead start 2020-07-18T03:26:35.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f580641d0 con 0x7f4f30003800 2020-07-18T03:26:35.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f581568d0 con 0x7f4f30003800 2020-07-18T03:26:35.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.472+0000 7f4f5e009700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f30003800 0x7f4f30003c20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f4f58062ff0 tx=0x7f4f4c007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.473+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f4c002e60 con 0x7f4f30003800 2020-07-18T03:26:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.473+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c007020 con 0x7f4f30003800 2020-07-18T03:26:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.473+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f4c007380 con 0x7f4f30003800 2020-07-18T03:26:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.473+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f58057450 con 0x7f4f30003800 2020-07-18T03:26:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.474+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4f4c00f840 con 0x7f4f30003800 2020-07-18T03:26:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.474+0000 7f4f4a7fc700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4f30053ad0 0x7f4f30055f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.474+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4f4c067750 con 0x7f4f30003800 2020-07-18T03:26:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.474+0000 7f4f5d808700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4f30053ad0 0x7f4f30055f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.474+0000 7f4f5d808700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4f30053ad0 0x7f4f30055f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f4f54018000 tx=0x7f4f54015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.477+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4f4c0401a0 con 0x7f4f30003800 2020-07-18T03:26:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.696+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4f5805d9c0 con 0x7f4f30003800 2020-07-18T03:26:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.700+0000 7f4f4a7fc700 1 -- 172.21.15.47:0/426067824 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4f4c0437c0 con 0x7f4f30003800 2020-07-18T03:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4f30053ad0 msgr2=0x7f4f30055f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4f30053ad0 0x7f4f30055f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f4f54018000 tx=0x7f4f54015000).stop 2020-07-18T03:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f30003800 msgr2=0x7f4f30003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f30003800 0x7f4f30003c20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f4f58062ff0 tx=0x7f4f4c007ac0).stop 2020-07-18T03:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 shutdown_connections 2020-07-18T03:26:35.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4f30053ad0 0x7f4f30055f90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 --2- 172.21.15.47:0/426067824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f30003800 0x7f4f30003c20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 >> 172.21.15.47:0/426067824 conn(0x7f4f580b93c0 msgr2=0x7f4f580b7940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 shutdown_connections 2020-07-18T03:26:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.702+0000 7f4f6026d700 1 -- 172.21.15.47:0/426067824 wait complete. 2020-07-18T03:26:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2020-07-18T03:26:35.714 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-osd 2020-07-18T03:26:35.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 Processor -- start 2020-07-18T03:26:35.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 -- start start 2020-07-18T03:26:35.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e8148e30 0x7f62e81492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62e80b0500 0x7f62e80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e8145500 con 0x7f62e8145120 2020-07-18T03:26:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e8146150 con 0x7f62e80b0500 2020-07-18T03:26:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62ee5a6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e8145770 con 0x7f62e8148e30 2020-07-18T03:26:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62dffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e8148e30 0x7f62e81492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62dffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e8148e30 0x7f62e81492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59852/0 (socket says 172.21.15.47:59852) 2020-07-18T03:26:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.832+0000 7f62dffff700 1 -- 172.21.15.47:0/1860581246 learned_addr learned my addr 172.21.15.47:0/1860581246 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:35.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ecb43700 1 --2- 172.21.15.47:0/1860581246 >> v2:172.21.15.18:3300/0 conn(0x7f62e80b0500 0x7f62e80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62dffff700 1 -- 172.21.15.47:0/1860581246 >> v2:172.21.15.18:3300/0 conn(0x7f62e80b0500 msgr2=0x7f62e80b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62dffff700 1 --2- 172.21.15.47:0/1860581246 >> v2:172.21.15.18:3300/0 conn(0x7f62e80b0500 0x7f62e80b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62dffff700 1 -- 172.21.15.47:0/1860581246 >> v1:172.21.15.18:6789/0 conn(0x7f62e8145120 legacy=0x7f62e8061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62dffff700 1 -- 172.21.15.47:0/1860581246 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8145a10 con 0x7f62e8148e30 2020-07-18T03:26:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62dffff700 1 --2- 172.21.15.47:0/1860581246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e8148e30 0x7f62e81492e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f62e805d300 tx=0x7f62d0006e90).ready entity=mon.1 client_cookie=3aaf4efd2c194d46 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62e77fe700 1 -- 172.21.15.47:0/1860581246 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62d0009930 con 0x7f62e8148e30 2020-07-18T03:26:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62e77fe700 1 -- 172.21.15.47:0/1860581246 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0009a90 con 0x7f62e8148e30 2020-07-18T03:26:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62e77fe700 1 -- 172.21.15.47:0/1860581246 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62d0009e30 con 0x7f62e8148e30 2020-07-18T03:26:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 -- 172.21.15.47:0/1860581246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e8148e30 msgr2=0x7f62e81492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/1860581246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e8148e30 0x7f62e81492e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f62e805d300 tx=0x7f62d0006e90).stop 2020-07-18T03:26:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 -- 172.21.15.47:0/1860581246 shutdown_connections 2020-07-18T03:26:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/1860581246 >> v2:172.21.15.18:3300/0 conn(0x7f62e80b0500 0x7f62e80b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/1860581246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e8148e30 0x7f62e81492e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 -- 172.21.15.47:0/1860581246 >> 172.21.15.47:0/1860581246 conn(0x7f62e80b93c0 msgr2=0x7f62e80b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 -- 172.21.15.47:0/1860581246 shutdown_connections 2020-07-18T03:26:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.833+0000 7f62ee5a6700 1 -- 172.21.15.47:0/1860581246 wait complete. 2020-07-18T03:26:35.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.835+0000 7f62ee5a6700 1 Processor -- start 2020-07-18T03:26:35.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.835+0000 7f62ee5a6700 1 -- start start 2020-07-18T03:26:35.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.835+0000 7f62ee5a6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e80b0500 0x7f62e8141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.835+0000 7f62ee5a6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e80b3a60 con 0x7f62e8159650 2020-07-18T03:26:35.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.835+0000 7f62ee5a6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e8146770 con 0x7f62e8145120 2020-07-18T03:26:35.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.835+0000 7f62ee5a6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e81423d0 con 0x7f62e80b0500 2020-07-18T03:26:35.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62e80b0500 0x7f62e8141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62dffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f62e8145120 0x7f62e8158f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40948/0 (socket says 172.21.15.47:40948) 2020-07-18T03:26:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62dffff700 1 -- 172.21.15.47:0/3311866104 learned_addr learned my addr 172.21.15.47:0/3311866104 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e7fff700 1 -- 172.21.15.47:0/3311866104 >> v1:172.21.15.47:6789/0 conn(0x7f62e8145120 legacy=0x7f62e8158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e7fff700 1 -- 172.21.15.47:0/3311866104 >> v1:172.21.15.18:6789/0 conn(0x7f62e8159650 legacy=0x7f62e8154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974838661 0 0) 0x7f62e8146770 con 0x7f62e8145120 2020-07-18T03:26:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e7fff700 1 -- 172.21.15.47:0/3311866104 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e80b3c90 con 0x7f62e80b0500 2020-07-18T03:26:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e7fff700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b0500 0x7f62e8141e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f62e8141400 tx=0x7f62d8007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62d800e4a0 con 0x7f62e80b0500 2020-07-18T03:26:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e80b5140 con 0x7f62e80b0500 2020-07-18T03:26:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d800e600 con 0x7f62e80b0500 2020-07-18T03:26:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62d800e770 con 0x7f62e80b0500 2020-07-18T03:26:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.836+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e8156570 con 0x7f62e80b0500 2020-07-18T03:26:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.837+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e80b3e20 con 0x7f62e80b0500 2020-07-18T03:26:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.837+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f62d8010030 con 0x7f62e80b0500 2020-07-18T03:26:35.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.838+0000 7f62e4ff9700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f62c404fd90 0x7f62c4052250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.838+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f62d80676d0 con 0x7f62e80b0500 2020-07-18T03:26:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.838+0000 7f62dffff700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f62c404fd90 0x7f62c4052250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.838+0000 7f62dffff700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f62c404fd90 0x7f62c4052250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f62d00093a0 tx=0x7f62d0015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:35.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:35.841+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f62d8040120 con 0x7f62e80b0500 2020-07-18T03:26:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.058+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f62e8057450 con 0x7f62e80b0500 2020-07-18T03:26:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.062+0000 7f62e4ff9700 1 -- 172.21.15.47:0/3311866104 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f62d8043740 con 0x7f62e80b0500 2020-07-18T03:26:36.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f62c404fd90 msgr2=0x7f62c4052250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f62c404fd90 0x7f62c4052250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f62d00093a0 tx=0x7f62d0015000).stop 2020-07-18T03:26:36.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b0500 msgr2=0x7f62e8141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b0500 0x7f62e8141e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f62e8141400 tx=0x7f62d8007ac0).stop 2020-07-18T03:26:36.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 shutdown_connections 2020-07-18T03:26:36.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62e80b0500 0x7f62e8141e00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 --2- 172.21.15.47:0/3311866104 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f62c404fd90 0x7f62c4052250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 >> 172.21.15.47:0/3311866104 conn(0x7f62e80b93c0 msgr2=0x7f62e80b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 shutdown_connections 2020-07-18T03:26:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.064+0000 7f62ee5a6700 1 -- 172.21.15.47:0/3311866104 wait complete. 2020-07-18T03:26:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2020-07-18T03:26:36.075 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd 2020-07-18T03:26:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.190+0000 7f90cd719700 1 Processor -- start 2020-07-18T03:26:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.190+0000 7f90cd719700 1 -- start start 2020-07-18T03:26:36.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90cd719700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f90c805ffb0 0x7f90c80625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90cd719700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c8147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90cd719700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c80b5830 con 0x7f90c80aea60 2020-07-18T03:26:36.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90cd719700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c80b5a60 con 0x7f90c805ffb0 2020-07-18T03:26:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90cd719700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8060390 con 0x7f90c81471d0 2020-07-18T03:26:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c67fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f90c80aea60 0x7f90c8142e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40954/0 (socket says 172.21.15.47:40954) 2020-07-18T03:26:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c67fc700 1 -- 172.21.15.47:0/1454872394 learned_addr learned my addr 172.21.15.47:0/1454872394 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c6ffd700 1 --2- 172.21.15.47:0/1454872394 >> v2:172.21.15.18:3300/0 conn(0x7f90c805ffb0 0x7f90c80625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c77fe700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c8147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c5ffb700 1 -- 172.21.15.47:0/1454872394 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456987844 0 0) 0x7f90c80b5830 con 0x7f90c80aea60 2020-07-18T03:26:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c5ffb700 1 -- 172.21.15.47:0/1454872394 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b4003240 con 0x7f90c80aea60 2020-07-18T03:26:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c6ffd700 1 -- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 msgr2=0x7f90c8147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c6ffd700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c8147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.191+0000 7f90c6ffd700 1 -- 172.21.15.47:0/1454872394 >> v1:172.21.15.47:6789/0 conn(0x7f90c80aea60 legacy=0x7f90c8142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c5ffb700 1 -- 172.21.15.47:0/1454872394 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4048647876 0 0) 0x7f90b4003240 con 0x7f90c80aea60 2020-07-18T03:26:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c6ffd700 1 -- 172.21.15.47:0/1454872394 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8146da0 con 0x7f90c805ffb0 2020-07-18T03:26:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c77fe700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c8147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c6ffd700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c805ffb0 0x7f90c80625d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f90b8001870 tx=0x7f90bc006dd0).ready entity=mon.0 client_cookie=2acedc393a7d82af server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c5ffb700 1 -- 172.21.15.47:0/1454872394 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90b80016d0 con 0x7f90c805ffb0 2020-07-18T03:26:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c5ffb700 1 -- 172.21.15.47:0/1454872394 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00d570 con 0x7f90c805ffb0 2020-07-18T03:26:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90c5ffb700 1 -- 172.21.15.47:0/1454872394 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90bc00d6e0 con 0x7f90c805ffb0 2020-07-18T03:26:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 -- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c805ffb0 msgr2=0x7f90c80625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c805ffb0 0x7f90c80625d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f90b8001870 tx=0x7f90bc006dd0).stop 2020-07-18T03:26:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 -- 172.21.15.47:0/1454872394 shutdown_connections 2020-07-18T03:26:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c8147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 --2- 172.21.15.47:0/1454872394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c805ffb0 0x7f90c80625d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 -- 172.21.15.47:0/1454872394 >> 172.21.15.47:0/1454872394 conn(0x7f90c80b93c0 msgr2=0x7f90c80b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 -- 172.21.15.47:0/1454872394 shutdown_connections 2020-07-18T03:26:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.192+0000 7f90cd719700 1 -- 172.21.15.47:0/1454872394 wait complete. 2020-07-18T03:26:36.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 Processor -- start 2020-07-18T03:26:36.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 -- start start 2020-07-18T03:26:36.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c81365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f90c813aca0 0x7f90c813c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c8138ea0 con 0x7f90c80aea60 2020-07-18T03:26:36.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c81390d0 con 0x7f90c813aca0 2020-07-18T03:26:36.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90cd719700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8139370 con 0x7f90c81471d0 2020-07-18T03:26:36.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90c6ffd700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f90c80aea60 0x7f90c8143bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59726/0 (socket says 172.21.15.47:59726) 2020-07-18T03:26:36.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.194+0000 7f90c6ffd700 1 -- 172.21.15.47:0/1741473154 learned_addr learned my addr 172.21.15.47:0/1741473154 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:36.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c67fc700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c81365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c77fe700 1 --2- 172.21.15.47:0/1741473154 >> v2:172.21.15.18:3300/0 conn(0x7f90c813aca0 0x7f90c813c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582927818 0 0) 0x7f90c8138ea0 con 0x7f90c80aea60 2020-07-18T03:26:36.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a4003240 con 0x7f90c80aea60 2020-07-18T03:26:36.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c67fc700 1 -- 172.21.15.47:0/1741473154 >> v2:172.21.15.18:3300/0 conn(0x7f90c813aca0 msgr2=0x7f90c813c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c67fc700 1 --2- 172.21.15.47:0/1741473154 >> v2:172.21.15.18:3300/0 conn(0x7f90c813aca0 0x7f90c813c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c67fc700 1 -- 172.21.15.47:0/1741473154 >> v1:172.21.15.18:6789/0 conn(0x7f90c80aea60 legacy=0x7f90c8143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c67fc700 1 -- 172.21.15.47:0/1741473154 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c813a820 con 0x7f90c81471d0 2020-07-18T03:26:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90c67fc700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c81365c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f90b8001930 tx=0x7f90b80011f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90b8007170 con 0x7f90c81471d0 2020-07-18T03:26:36.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b80072d0 con 0x7f90c81471d0 2020-07-18T03:26:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c813cee0 con 0x7f90c81471d0 2020-07-18T03:26:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90b8004900 con 0x7f90c81471d0 2020-07-18T03:26:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.195+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c81399d0 con 0x7f90c81471d0 2020-07-18T03:26:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.196+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c81442e0 con 0x7f90c81471d0 2020-07-18T03:26:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.197+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f90b801b070 con 0x7f90c81471d0 2020-07-18T03:26:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.197+0000 7f90aeffd700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f90a4050310 0x7f90a40527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.197+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f90b8066f90 con 0x7f90c81471d0 2020-07-18T03:26:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.197+0000 7f90c6ffd700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f90a4050310 0x7f90a40527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.197+0000 7f90c6ffd700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f90a4050310 0x7f90a40527d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f90c813bfe0 tx=0x7f90bc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.201+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f90b803f9e0 con 0x7f90c81471d0 2020-07-18T03:26:36.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.418+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f90c813d1f0 con 0x7f90c81471d0 2020-07-18T03:26:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.422+0000 7f90aeffd700 1 -- 172.21.15.47:0/1741473154 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90b8043000 con 0x7f90c81471d0 2020-07-18T03:26:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f90a4050310 msgr2=0x7f90a40527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f90a4050310 0x7f90a40527d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f90c813bfe0 tx=0x7f90bc015000).stop 2020-07-18T03:26:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 msgr2=0x7f90c81365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c81365c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f90b8001930 tx=0x7f90b80011f0).stop 2020-07-18T03:26:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 shutdown_connections 2020-07-18T03:26:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f90a4050310 0x7f90a40527d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 --2- 172.21.15.47:0/1741473154 >> v2:172.21.15.18:3300/0 conn(0x7f90c813aca0 0x7f90c813c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.423+0000 7f90cd719700 1 --2- 172.21.15.47:0/1741473154 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f90c81471d0 0x7f90c81365c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.424+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 >> 172.21.15.47:0/1741473154 conn(0x7f90c80b93c0 msgr2=0x7f90c805f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.424+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 shutdown_connections 2020-07-18T03:26:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.424+0000 7f90cd719700 1 -- 172.21.15.47:0/1741473154 wait complete. 2020-07-18T03:26:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2020-07-18T03:26:36.435 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd-mirror 2020-07-18T03:26:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.549+0000 7f18a36e3700 1 Processor -- start 2020-07-18T03:26:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.549+0000 7f18a36e3700 1 -- start start 2020-07-18T03:26:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a36e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f189c0aea60 0x7f189c142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a36e3700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c143330 0x7f189c1471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a36e3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c0b5830 con 0x7f189c05ffb0 2020-07-18T03:26:36.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a36e3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c0b5a60 con 0x7f189c0aea60 2020-07-18T03:26:36.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a36e3700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c060390 con 0x7f189c143330 2020-07-18T03:26:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a0c7e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f189c0aea60 0x7f189c142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a147f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f189c05ffb0 0x7f189c0625d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40968/0 (socket says 172.21.15.47:40968) 2020-07-18T03:26:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.550+0000 7f18a147f700 1 -- 172.21.15.47:0/2590571468 learned_addr learned my addr 172.21.15.47:0/2590571468 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a0c7e700 1 -- 172.21.15.47:0/2590571468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c143330 msgr2=0x7f189c1471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a0c7e700 1 --2- 172.21.15.47:0/2590571468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c143330 0x7f189c1471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:26:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a0c7e700 1 -- 172.21.15.47:0/2590571468 >> v1:172.21.15.47:6789/0 conn(0x7f189c05ffb0 legacy=0x7f189c0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a0c7e700 1 -- 172.21.15.47:0/2590571468 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c146da0 con 0x7f189c0aea60 2020-07-18T03:26:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f188bfff700 1 -- 172.21.15.47:0/2590571468 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131621419 0 0) 0x7f189c0b5830 con 0x7f189c05ffb0 2020-07-18T03:26:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a0c7e700 1 --2- 172.21.15.47:0/2590571468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f189c0aea60 0x7f189c142e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f18980032b0 tx=0x7f18980091d0).ready entity=mon.1 client_cookie=cd8f4a70be86795c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f188bfff700 1 -- 172.21.15.47:0/2590571468 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189800ba50 con 0x7f189c0aea60 2020-07-18T03:26:36.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f188bfff700 1 -- 172.21.15.47:0/2590571468 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189800bbb0 con 0x7f189c0aea60 2020-07-18T03:26:36.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f188bfff700 1 -- 172.21.15.47:0/2590571468 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189800bd20 con 0x7f189c0aea60 2020-07-18T03:26:36.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 -- 172.21.15.47:0/2590571468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f189c0aea60 msgr2=0x7f189c142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 --2- 172.21.15.47:0/2590571468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f189c0aea60 0x7f189c142e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f18980032b0 tx=0x7f18980091d0).stop 2020-07-18T03:26:36.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 -- 172.21.15.47:0/2590571468 shutdown_connections 2020-07-18T03:26:36.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 --2- 172.21.15.47:0/2590571468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c143330 0x7f189c1471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:26:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 --2- 172.21.15.47:0/2590571468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f189c0aea60 0x7f189c142e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 -- 172.21.15.47:0/2590571468 >> 172.21.15.47:0/2590571468 conn(0x7f189c0b93c0 msgr2=0x7f189c0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 -- 172.21.15.47:0/2590571468 shutdown_connections 2020-07-18T03:26:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.551+0000 7f18a36e3700 1 -- 172.21.15.47:0/2590571468 wait complete. 2020-07-18T03:26:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.553+0000 7f18a36e3700 1 Processor -- start 2020-07-18T03:26:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.553+0000 7f18a36e3700 1 -- start start 2020-07-18T03:26:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.553+0000 7f18a36e3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f189c143330 0x7f189c155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.553+0000 7f18a36e3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c147e20 con 0x7f189c05ffb0 2020-07-18T03:26:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.553+0000 7f18a36e3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c155580 con 0x7f189c144890 2020-07-18T03:26:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.553+0000 7f18a36e3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c1564e0 con 0x7f189c143330 2020-07-18T03:26:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a1c80700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f189c144890 0x7f189c152710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40978/0 (socket says 172.21.15.47:40978) 2020-07-18T03:26:36.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a1c80700 1 -- 172.21.15.47:0/935748103 learned_addr learned my addr 172.21.15.47:0/935748103 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:36.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a0c7e700 1 --2- 172.21.15.47:0/935748103 >> v2:172.21.15.18:3300/0 conn(0x7f189c143330 0x7f189c155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489389502 0 0) 0x7f189c155580 con 0x7f189c144890 2020-07-18T03:26:36.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1874003240 con 0x7f189c144890 2020-07-18T03:26:36.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575027457 0 0) 0x7f189c147e20 con 0x7f189c05ffb0 2020-07-18T03:26:36.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c155580 con 0x7f189c05ffb0 2020-07-18T03:26:36.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3162380176 0 0) 0x7f1874003240 con 0x7f189c144890 2020-07-18T03:26:36.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f189c147e20 con 0x7f189c144890 2020-07-18T03:26:36.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f18940027e0 con 0x7f189c144890 2020-07-18T03:26:36.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a0c7e700 1 -- 172.21.15.47:0/935748103 >> v1:172.21.15.47:6789/0 conn(0x7f189c144890 legacy=0x7f189c152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a0c7e700 1 -- 172.21.15.47:0/935748103 >> v1:172.21.15.18:6789/0 conn(0x7f189c05ffb0 legacy=0x7f189c1472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a0c7e700 1 -- 172.21.15.47:0/935748103 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c155820 con 0x7f189c143330 2020-07-18T03:26:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18a0c7e700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c143330 0x7f189c155f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f189c148650 tx=0x7f1898016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.554+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18980205f0 con 0x7f189c143330 2020-07-18T03:26:36.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.555+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898020750 con 0x7f189c143330 2020-07-18T03:26:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.555+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c155ac0 con 0x7f189c143330 2020-07-18T03:26:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.555+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1898020ab0 con 0x7f189c143330 2020-07-18T03:26:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.555+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c158a40 con 0x7f189c143330 2020-07-18T03:26:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.555+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c155c50 con 0x7f189c143330 2020-07-18T03:26:36.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.556+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1898027030 con 0x7f189c143330 2020-07-18T03:26:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.556+0000 7f18897fa700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1874050760 0x7f1874052c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.556+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1898068d70 con 0x7f189c143330 2020-07-18T03:26:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.556+0000 7f18a147f700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1874050760 0x7f1874052c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.556+0000 7f18a147f700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1874050760 0x7f1874052c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f188c000fa0 tx=0x7f188c0078a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.560+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1898041840 con 0x7f189c143330 2020-07-18T03:26:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.771+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f189c057450 con 0x7f189c143330 2020-07-18T03:26:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.775+0000 7f18897fa700 1 -- 172.21.15.47:0/935748103 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18980250a0 con 0x7f189c143330 2020-07-18T03:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1874050760 msgr2=0x7f1874052c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1874050760 0x7f1874052c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f188c000fa0 tx=0x7f188c0078a0).stop 2020-07-18T03:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c143330 msgr2=0x7f189c155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c143330 0x7f189c155f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f189c148650 tx=0x7f1898016000).stop 2020-07-18T03:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 shutdown_connections 2020-07-18T03:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1874050760 0x7f1874052c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 --2- 172.21.15.47:0/935748103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c143330 0x7f189c155f60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 >> 172.21.15.47:0/935748103 conn(0x7f189c0b93c0 msgr2=0x7f189c0b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 shutdown_connections 2020-07-18T03:26:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.776+0000 7f18a36e3700 1 -- 172.21.15.47:0/935748103 wait complete. 2020-07-18T03:26:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2020-07-18T03:26:36.791 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rgw 2020-07-18T03:26:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.905+0000 7fcee56d2700 1 Processor -- start 2020-07-18T03:26:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.905+0000 7fcee56d2700 1 -- start start 2020-07-18T03:26:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcee56d2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0143300 0x7fcee01436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcee56d2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fcee0062b40 0x7fcee0062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcee56d2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcee0063490 0x7fcee00b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcee56d2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00b1640 con 0x7fcee0062b40 2020-07-18T03:26:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcee56d2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00b1820 con 0x7fcee0063490 2020-07-18T03:26:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcee56d2700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00b1ac0 con 0x7fcee0143300 2020-07-18T03:26:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcedf7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcee0063490 0x7fcee00b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcedf7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcee0063490 0x7fcee00b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59894/0 (socket says 172.21.15.47:59894) 2020-07-18T03:26:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.906+0000 7fcedf7fe700 1 -- 172.21.15.47:0/528995845 learned_addr learned my addr 172.21.15.47:0/528995845 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcede7fc700 1 --2- 172.21.15.47:0/528995845 >> v2:172.21.15.18:3300/0 conn(0x7fcee0062b40 0x7fcee0062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedeffd700 1 --2- 172.21.15.47:0/528995845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0143300 0x7fcee01436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedf7fe700 1 -- 172.21.15.47:0/528995845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0143300 msgr2=0x7fcee01436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedf7fe700 1 --2- 172.21.15.47:0/528995845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0143300 0x7fcee01436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedf7fe700 1 -- 172.21.15.47:0/528995845 >> v2:172.21.15.18:3300/0 conn(0x7fcee0062b40 msgr2=0x7fcee0062fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedf7fe700 1 --2- 172.21.15.47:0/528995845 >> v2:172.21.15.18:3300/0 conn(0x7fcee0062b40 0x7fcee0062fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedf7fe700 1 -- 172.21.15.47:0/528995845 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00b1d60 con 0x7fcee0063490 2020-07-18T03:26:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcedf7fe700 1 --2- 172.21.15.47:0/528995845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee0063490 0x7fcee00b4c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fced80032b0 tx=0x7fced8009270).ready entity=mon.1 client_cookie=b44d9abbd49fb47b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fceddffb700 1 -- 172.21.15.47:0/528995845 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced800bc00 con 0x7fcee0063490 2020-07-18T03:26:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fceddffb700 1 -- 172.21.15.47:0/528995845 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800bd60 con 0x7fcee0063490 2020-07-18T03:26:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fceddffb700 1 -- 172.21.15.47:0/528995845 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced8011400 con 0x7fcee0063490 2020-07-18T03:26:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 -- 172.21.15.47:0/528995845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee0063490 msgr2=0x7fcee00b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 --2- 172.21.15.47:0/528995845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee0063490 0x7fcee00b4c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fced80032b0 tx=0x7fced8009270).stop 2020-07-18T03:26:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 -- 172.21.15.47:0/528995845 shutdown_connections 2020-07-18T03:26:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 --2- 172.21.15.47:0/528995845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee0063490 0x7fcee00b4c70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 --2- 172.21.15.47:0/528995845 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0143300 0x7fcee01436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 --2- 172.21.15.47:0/528995845 >> v2:172.21.15.18:3300/0 conn(0x7fcee0062b40 0x7fcee0062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 -- 172.21.15.47:0/528995845 >> 172.21.15.47:0/528995845 conn(0x7fcee00ba4e0 msgr2=0x7fcee00ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 -- 172.21.15.47:0/528995845 shutdown_connections 2020-07-18T03:26:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.907+0000 7fcee56d2700 1 -- 172.21.15.47:0/528995845 wait complete. 2020-07-18T03:26:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcee56d2700 1 Processor -- start 2020-07-18T03:26:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcee56d2700 1 -- start start 2020-07-18T03:26:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcee56d2700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0066910 0x7fcee01517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcee56d2700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee013d550 con 0x7fcee0143300 2020-07-18T03:26:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcee56d2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee013d780 con 0x7fcee0062b40 2020-07-18T03:26:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcee56d2700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee013da20 con 0x7fcee0066910 2020-07-18T03:26:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcedeffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcee0062b40 0x7fcee01525d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40986/0 (socket says 172.21.15.47:40986) 2020-07-18T03:26:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.909+0000 7fcedeffd700 1 -- 172.21.15.47:0/167911047 learned_addr learned my addr 172.21.15.47:0/167911047 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcedf7fe700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0066910 0x7fcee01517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215839711 0 0) 0x7fcee013d780 con 0x7fcee0062b40 2020-07-18T03:26:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcebc003240 con 0x7fcee0062b40 2020-07-18T03:26:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472070967 0 0) 0x7fcee013d550 con 0x7fcee0143300 2020-07-18T03:26:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee013d780 con 0x7fcee0143300 2020-07-18T03:26:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3598582983 0 0) 0x7fcebc003240 con 0x7fcee0062b40 2020-07-18T03:26:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcee013d550 con 0x7fcee0062b40 2020-07-18T03:26:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fced4006e60 con 0x7fcee0062b40 2020-07-18T03:26:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0066910 msgr2=0x7fcee01517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcee0066910 0x7fcee01517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 >> v1:172.21.15.47:6789/0 conn(0x7fcee0062b40 legacy=0x7fcee01525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 >> v1:172.21.15.18:6789/0 conn(0x7fcee0143300 legacy=0x7fcee00b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcebc0044e0 0x7fcebc004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcebc004e10 0x7fcebc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee013da20 con 0x7fcebc004e10 2020-07-18T03:26:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00bc20 con 0x7fcebc003970 2020-07-18T03:26:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00bd90 con 0x7fcebc0044e0 2020-07-18T03:26:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcedeffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcedf7fe700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcebc004e10 0x7fcebc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.910+0000 7fcede7fc700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcebc0044e0 0x7fcebc004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcedeffd700 1 -- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcebc0044e0 msgr2=0x7fcebc004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcedeffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fcebc0044e0 0x7fcebc004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcedeffd700 1 -- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcebc004e10 msgr2=0x7fcebc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcedeffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcebc004e10 0x7fcebc00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcedeffd700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0152f60 con 0x7fcebc003970 2020-07-18T03:26:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee01517c0 con 0x7fcebc003970 2020-07-18T03:26:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0151c30 con 0x7fcebc003970 2020-07-18T03:26:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcedeffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fced400a000 tx=0x7fced40018a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced400ccc0 con 0x7fcebc003970 2020-07-18T03:26:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced400ce20 con 0x7fcebc003970 2020-07-18T03:26:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.911+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced4006210 con 0x7fcebc003970 2020-07-18T03:26:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.912+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee005b150 con 0x7fcebc003970 2020-07-18T03:26:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.912+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fced4006370 con 0x7fcebc003970 2020-07-18T03:26:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.912+0000 7fcec6ffd700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcebc058a30 0x7fcebc05aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.912+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fced4067c10 con 0x7fcebc003970 2020-07-18T03:26:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.912+0000 7fcede7fc700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcebc058a30 0x7fcebc05aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.912+0000 7fcede7fc700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcebc058a30 0x7fcebc05aef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fced000e000 tx=0x7fced0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:36.916+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fced406e080 con 0x7fcebc003970 2020-07-18T03:26:37.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.129+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fcee0057450 con 0x7fcebc003970 2020-07-18T03:26:37.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.133+0000 7fcec6ffd700 1 -- 172.21.15.47:0/167911047 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v14) v1 ==== 97+0+0 (secure 0 0 0) 0x7fced4040680 con 0x7fcebc003970 2020-07-18T03:26:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.134+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcebc058a30 msgr2=0x7fcebc05aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.134+0000 7fcee56d2700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fcebc058a30 0x7fcebc05aef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fced000e000 tx=0x7fced0008040).stop 2020-07-18T03:26:37.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.134+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcebc003970 msgr2=0x7fcebc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.134+0000 7fcee56d2700 1 --2- 172.21.15.47:0/167911047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcebc003970 0x7fcebc003d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fced400a000 tx=0x7fced40018a0).stop 2020-07-18T03:26:37.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.134+0000 7fcedf7fe700 1 -- 172.21.15.47:0/167911047 reap_dead start 2020-07-18T03:26:37.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.135+0000 7fcedf7fe700 1 -- 172.21.15.47:0/167911047 reap_dead start 2020-07-18T03:26:37.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.135+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 shutdown_connections 2020-07-18T03:26:37.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.135+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 >> 172.21.15.47:0/167911047 conn(0x7fcee00ba4e0 msgr2=0x7fcee00b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:37.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.135+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 shutdown_connections 2020-07-18T03:26:37.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.135+0000 7fcee56d2700 1 -- 172.21.15.47:0/167911047 wait complete. 2020-07-18T03:26:37.140 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2020-07-18T03:26:37.147 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:37.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.261+0000 7f0acdc08700 1 Processor -- start 2020-07-18T03:26:37.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.261+0000 7f0acdc08700 1 -- start start 2020-07-18T03:26:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0acdc08700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0ac80ad100 0x7f0ac80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0acdc08700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac81443f0 0x7f0ac8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0acdc08700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac8147330 con 0x7f0ac8147a00 2020-07-18T03:26:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0acdc08700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac80ae530 con 0x7f0ac80ad100 2020-07-18T03:26:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0acdc08700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80b57e0 con 0x7f0ac81443f0 2020-07-18T03:26:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0ac8147a00 0x7f0ac805f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41002/0 (socket says 172.21.15.47:41002) 2020-07-18T03:26:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac6ffd700 1 -- 172.21.15.47:0/248362060 learned_addr learned my addr 172.21.15.47:0/248362060 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac77fe700 1 --2- 172.21.15.47:0/248362060 >> v2:172.21.15.18:3300/0 conn(0x7f0ac80ad100 0x7f0ac80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac7fff700 1 --2- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac81443f0 0x7f0ac8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac67fc700 1 -- 172.21.15.47:0/248362060 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826098751 0 0) 0x7f0ac8147330 con 0x7f0ac8147a00 2020-07-18T03:26:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac67fc700 1 -- 172.21.15.47:0/248362060 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab4003240 con 0x7f0ac8147a00 2020-07-18T03:26:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac77fe700 1 -- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac81443f0 msgr2=0x7f0ac8147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac77fe700 1 --2- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac81443f0 0x7f0ac8147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac77fe700 1 -- 172.21.15.47:0/248362060 >> v1:172.21.15.47:6789/0 conn(0x7f0ac8147a00 legacy=0x7f0ac805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.262+0000 7f0ac77fe700 1 -- 172.21.15.47:0/248362060 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80b5a80 con 0x7f0ac80ad100 2020-07-18T03:26:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0ac77fe700 1 --2- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac80ad100 0x7f0ac80b4c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0ab8002f30 tx=0x7f0ab8006dd0).ready entity=mon.0 client_cookie=774e5a5594e19fd6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0ac67fc700 1 -- 172.21.15.47:0/248362060 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab80096a0 con 0x7f0ac80ad100 2020-07-18T03:26:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0ac67fc700 1 -- 172.21.15.47:0/248362060 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8009800 con 0x7f0ac80ad100 2020-07-18T03:26:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 -- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac80ad100 msgr2=0x7f0ac80b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 --2- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac80ad100 0x7f0ac80b4c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0ab8002f30 tx=0x7f0ab8006dd0).stop 2020-07-18T03:26:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0ac67fc700 1 -- 172.21.15.47:0/248362060 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab8009b60 con 0x7f0ac80ad100 2020-07-18T03:26:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 -- 172.21.15.47:0/248362060 shutdown_connections 2020-07-18T03:26:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 --2- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac81443f0 0x7f0ac8147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 --2- 172.21.15.47:0/248362060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac80ad100 0x7f0ac80b4c10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 -- 172.21.15.47:0/248362060 >> 172.21.15.47:0/248362060 conn(0x7f0ac80b9340 msgr2=0x7f0ac80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 -- 172.21.15.47:0/248362060 shutdown_connections 2020-07-18T03:26:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.263+0000 7f0acdc08700 1 -- 172.21.15.47:0/248362060 wait complete. 2020-07-18T03:26:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.264+0000 7f0acdc08700 1 Processor -- start 2020-07-18T03:26:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.264+0000 7f0acdc08700 1 -- start start 2020-07-18T03:26:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0acdc08700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0ac81443f0 0x7f0ac8141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0acdc08700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac8065f90 0x7f0ac8142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0acdc08700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac8140b40 con 0x7f0ac8147a00 2020-07-18T03:26:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0acdc08700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac8140960 con 0x7f0ac81443f0 2020-07-18T03:26:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0acdc08700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8140de0 con 0x7f0ac8065f90 2020-07-18T03:26:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0ac6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0ac8147a00 0x7f0ac8153ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41008/0 (socket says 172.21.15.47:41008) 2020-07-18T03:26:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0ac6ffd700 1 -- 172.21.15.47:0/3364836883 learned_addr learned my addr 172.21.15.47:0/3364836883 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0ac77fe700 1 --2- 172.21.15.47:0/3364836883 >> v2:172.21.15.18:3300/0 conn(0x7f0ac81443f0 0x7f0ac8141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0ac7fff700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac8065f90 0x7f0ac8142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022993925 0 0) 0x7f0ac8140b40 con 0x7f0ac8147a00 2020-07-18T03:26:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.265+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003240 con 0x7f0ac8147a00 2020-07-18T03:26:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0ac77fe700 1 -- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac8065f90 msgr2=0x7f0ac8142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0ac77fe700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac8065f90 0x7f0ac8142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0ac77fe700 1 -- 172.21.15.47:0/3364836883 >> v1:172.21.15.47:6789/0 conn(0x7f0ac8147a00 legacy=0x7f0ac8153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0ac77fe700 1 -- 172.21.15.47:0/3364836883 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8141080 con 0x7f0ac81443f0 2020-07-18T03:26:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0ac77fe700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac81443f0 0x7f0ac8141520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0ab8002f30 tx=0x7f0ab8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab801e460 con 0x7f0ac81443f0 2020-07-18T03:26:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab801e5c0 con 0x7f0ac81443f0 2020-07-18T03:26:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac8140160 con 0x7f0ac81443f0 2020-07-18T03:26:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab801e920 con 0x7f0ac81443f0 2020-07-18T03:26:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.266+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac8142560 con 0x7f0ac81443f0 2020-07-18T03:26:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.267+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8141210 con 0x7f0ac81443f0 2020-07-18T03:26:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.267+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0ab8025030 con 0x7f0ac81443f0 2020-07-18T03:26:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.268+0000 7f0aa77fe700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0aa00502e0 0x7f0aa00527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.268+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0ab8068f00 con 0x7f0ac81443f0 2020-07-18T03:26:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.268+0000 7f0ac6ffd700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0aa00502e0 0x7f0aa00527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.268+0000 7f0ac6ffd700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0aa00502e0 0x7f0aa00527a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f0abc000fc0 tx=0x7f0abc0078c0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.272+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0ab8041990 con 0x7f0ac81443f0 2020-07-18T03:26:37.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.429+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac8057450 con 0x7f0ac81443f0 2020-07-18T03:26:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.430+0000 7f0aa77fe700 1 -- 172.21.15.47:0/3364836883 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13417 (secure 0 0 0) 0x7f0ab8044f70 con 0x7f0ac81443f0 2020-07-18T03:26:37.431 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:37.432 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:26:34.572452+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0aa00502e0 msgr2=0x7f0aa00527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0aa00502e0 0x7f0aa00527a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f0abc000fc0 tx=0x7f0abc0078c0).stop 2020-07-18T03:26:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac81443f0 msgr2=0x7f0ac8141520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac81443f0 0x7f0ac8141520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0ab8002f30 tx=0x7f0ab8015000).stop 2020-07-18T03:26:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 shutdown_connections 2020-07-18T03:26:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ac8065f90 0x7f0ac8142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac81443f0 0x7f0ac8141520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 --2- 172.21.15.47:0/3364836883 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0aa00502e0 0x7f0aa00527a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 >> 172.21.15.47:0/3364836883 conn(0x7f0ac80b9340 msgr2=0x7f0ac8143680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 shutdown_connections 2020-07-18T03:26:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.432+0000 7f0acdc08700 1 -- 172.21.15.47:0/3364836883 wait complete. 2020-07-18T03:26:37.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-07-18T03:26:37.445 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.561+0000 7fd33f196700 1 Processor -- start 2020-07-18T03:26:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.561+0000 7fd33f196700 1 -- start start 2020-07-18T03:26:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33f196700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd3380ad330 0x7fd338143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33f196700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b3a10 0x7fd3380b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33f196700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd338147de0 con 0x7fd3380b59b0 2020-07-18T03:26:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33f196700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380ae000 con 0x7fd3380ad330 2020-07-18T03:26:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33f196700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338148010 con 0x7fd3380b3a10 2020-07-18T03:26:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd337fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd3380b59b0 0x7fd3380b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41016/0 (socket says 172.21.15.47:41016) 2020-07-18T03:26:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd337fff700 1 -- 172.21.15.47:0/2625444198 learned_addr learned my addr 172.21.15.47:0/2625444198 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33cf32700 1 --2- 172.21.15.47:0/2625444198 >> v2:172.21.15.18:3300/0 conn(0x7fd3380ad330 0x7fd338143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.562+0000 7fd33d733700 1 --2- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b3a10 0x7fd3380b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd3377fe700 1 -- 172.21.15.47:0/2625444198 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098324788 0 0) 0x7fd338147de0 con 0x7fd3380b59b0 2020-07-18T03:26:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd3377fe700 1 -- 172.21.15.47:0/2625444198 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd324003240 con 0x7fd3380b59b0 2020-07-18T03:26:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd3377fe700 1 -- 172.21.15.47:0/2625444198 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3266819519 0 0) 0x7fd324003240 con 0x7fd3380b59b0 2020-07-18T03:26:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33cf32700 1 -- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b3a10 msgr2=0x7fd3380b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33cf32700 1 --2- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b3a10 0x7fd3380b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33cf32700 1 -- 172.21.15.47:0/2625444198 >> v1:172.21.15.47:6789/0 conn(0x7fd3380b59b0 legacy=0x7fd3380b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33cf32700 1 -- 172.21.15.47:0/2625444198 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380ad710 con 0x7fd3380ad330 2020-07-18T03:26:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33cf32700 1 --2- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3380ad330 0x7fd338143640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd32c001870 tx=0x7fd328006dd0).ready entity=mon.0 client_cookie=19dd309ca9d0c091 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd3377fe700 1 -- 172.21.15.47:0/2625444198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd32c0016d0 con 0x7fd3380ad330 2020-07-18T03:26:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd3377fe700 1 -- 172.21.15.47:0/2625444198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800d570 con 0x7fd3380ad330 2020-07-18T03:26:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd3377fe700 1 -- 172.21.15.47:0/2625444198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd32800d6e0 con 0x7fd3380ad330 2020-07-18T03:26:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 -- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3380ad330 msgr2=0x7fd338143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 --2- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3380ad330 0x7fd338143640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd32c001870 tx=0x7fd328006dd0).stop 2020-07-18T03:26:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 -- 172.21.15.47:0/2625444198 shutdown_connections 2020-07-18T03:26:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 --2- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b3a10 0x7fd3380b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 --2- 172.21.15.47:0/2625444198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3380ad330 0x7fd338143640 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 -- 172.21.15.47:0/2625444198 >> 172.21.15.47:0/2625444198 conn(0x7fd33805f5a0 msgr2=0x7fd33805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 -- 172.21.15.47:0/2625444198 shutdown_connections 2020-07-18T03:26:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.563+0000 7fd33f196700 1 -- 172.21.15.47:0/2625444198 wait complete. 2020-07-18T03:26:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.565+0000 7fd33f196700 1 Processor -- start 2020-07-18T03:26:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.565+0000 7fd33f196700 1 -- start start 2020-07-18T03:26:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33f196700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd3380b3a10 0x7fd338158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33f196700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b59b0 0x7fd338141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33f196700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd338066510 0x7fd338155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33f196700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd338066c20 con 0x7fd3380b3a10 2020-07-18T03:26:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33f196700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3381550c0 con 0x7fd338066510 2020-07-18T03:26:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33f196700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338155360 con 0x7fd3380b59b0 2020-07-18T03:26:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd338066510 0x7fd338155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd338066510 0x7fd338155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59936/0 (socket says 172.21.15.47:59936) 2020-07-18T03:26:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 -- 172.21.15.47:0/97035029 learned_addr learned my addr 172.21.15.47:0/97035029 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd337fff700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b59b0 0x7fd338141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33cf32700 1 --2- 172.21.15.47:0/97035029 >> v2:172.21.15.18:3300/0 conn(0x7fd3380b3a10 0x7fd338158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 -- 172.21.15.47:0/97035029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b59b0 msgr2=0x7fd338141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b59b0 0x7fd338141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 -- 172.21.15.47:0/97035029 >> v2:172.21.15.18:3300/0 conn(0x7fd3380b3a10 msgr2=0x7fd338158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 --2- 172.21.15.47:0/97035029 >> v2:172.21.15.18:3300/0 conn(0x7fd3380b3a10 0x7fd338158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 -- 172.21.15.47:0/97035029 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3381422d0 con 0x7fd338066510 2020-07-18T03:26:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.566+0000 7fd33d733700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd338066510 0x7fd338155800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fd330006de0 tx=0x7fd3300091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.567+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33000c950 con 0x7fd338066510 2020-07-18T03:26:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.567+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33000cab0 con 0x7fd338066510 2020-07-18T03:26:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.567+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33000ce10 con 0x7fd338066510 2020-07-18T03:26:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.567+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd338155cf0 con 0x7fd338066510 2020-07-18T03:26:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.567+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33813bb90 con 0x7fd338066510 2020-07-18T03:26:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.568+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd338154f30 con 0x7fd338066510 2020-07-18T03:26:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.568+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd330022070 con 0x7fd338066510 2020-07-18T03:26:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.568+0000 7fd334ff9700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd31404ffd0 0x7fd314052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.568+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd330067f50 con 0x7fd338066510 2020-07-18T03:26:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.568+0000 7fd33cf32700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd31404ffd0 0x7fd314052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.568+0000 7fd33cf32700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd31404ffd0 0x7fd314052490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fd32801c000 tx=0x7fd328015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.571+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd330040a20 con 0x7fd338066510 2020-07-18T03:26:37.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.730+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd33805b120 con 0x7fd338066510 2020-07-18T03:26:37.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.730+0000 7fd334ff9700 1 -- 172.21.15.47:0/97035029 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v29) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd330016020 con 0x7fd338066510 2020-07-18T03:26:37.733 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd31404ffd0 msgr2=0x7fd314052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd31404ffd0 0x7fd314052490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fd32801c000 tx=0x7fd328015000).stop 2020-07-18T03:26:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd338066510 msgr2=0x7fd338155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd338066510 0x7fd338155800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fd330006de0 tx=0x7fd3300091d0).stop 2020-07-18T03:26:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 shutdown_connections 2020-07-18T03:26:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd31404ffd0 0x7fd314052490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd338066510 0x7fd338155800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 --2- 172.21.15.47:0/97035029 >> v2:172.21.15.18:3300/0 conn(0x7fd3380b3a10 0x7fd338158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 --2- 172.21.15.47:0/97035029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3380b59b0 0x7fd338141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 >> 172.21.15.47:0/97035029 conn(0x7fd33805f5a0 msgr2=0x7fd3380b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 shutdown_connections 2020-07-18T03:26:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.732+0000 7fd33f196700 1 -- 172.21.15.47:0/97035029 wait complete. 2020-07-18T03:26:37.744 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.860+0000 7f869d010700 1 Processor -- start 2020-07-18T03:26:37.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.860+0000 7f869d010700 1 -- start start 2020-07-18T03:26:37.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869d010700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8698141fb0 0x7f8698142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869d010700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8698061db0 con 0x7f8698144e70 2020-07-18T03:26:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869d010700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8698062050 con 0x7f86980b04b0 2020-07-18T03:26:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869d010700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8698145250 con 0x7f8698141fb0 2020-07-18T03:26:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869759e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f86980b04b0 0x7f86980b0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41032/0 (socket says 172.21.15.47:41032) 2020-07-18T03:26:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869759e700 1 -- 172.21.15.47:0/4000188183 learned_addr learned my addr 172.21.15.47:0/4000188183 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f869659c700 1 --2- 172.21.15.47:0/4000188183 >> v2:172.21.15.18:3300/0 conn(0x7f8698141fb0 0x7f8698142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883604720 0 0) 0x7f8698062050 con 0x7f86980b04b0 2020-07-18T03:26:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8678003240 con 0x7f86980b04b0 2020-07-18T03:26:37.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972161149 0 0) 0x7f8698061db0 con 0x7f8698144e70 2020-07-18T03:26:37.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8698062050 con 0x7f8698144e70 2020-07-18T03:26:37.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4160800777 0 0) 0x7f8678003240 con 0x7f86980b04b0 2020-07-18T03:26:37.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 >> v2:172.21.15.18:3300/0 conn(0x7f8698141fb0 msgr2=0x7f8698142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:37.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 --2- 172.21.15.47:0/4000188183 >> v2:172.21.15.18:3300/0 conn(0x7f8698141fb0 0x7f8698142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 >> v1:172.21.15.18:6789/0 conn(0x7f8698144e70 legacy=0x7f86980b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980622f0 con 0x7f86980b04b0 2020-07-18T03:26:37.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f86880016d0 con 0x7f86980b04b0 2020-07-18T03:26:37.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 >> v1:172.21.15.47:6789/0 conn(0x7f86980b04b0 legacy=0x7f86980b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8678003910 0x7f8678003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f86780044e0 0x7f8678004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8678004e50 0x7f867800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698145250 con 0x7f8678003910 2020-07-18T03:26:37.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867800bcb0 con 0x7f8678004e50 2020-07-18T03:26:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.861+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867800be20 con 0x7f86780044e0 2020-07-18T03:26:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f8696d9d700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8678003910 0x7f8678003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8678004e50 0x7f867800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869659c700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f86780044e0 0x7f8678004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 -- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f86780044e0 msgr2=0x7f8678004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:37.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f86780044e0 0x7f8678004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 -- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8678003910 msgr2=0x7f8678003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:37.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8678003910 0x7f8678003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 -- 172.21.15.47:0/4000188183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8678003240 con 0x7f8678004e50 2020-07-18T03:26:37.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 -- 172.21.15.47:0/4000188183 reap_dead start 2020-07-18T03:26:37.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869759e700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8678004e50 0x7f867800b730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f8678004450 tx=0x7f8698061f40).ready entity=mon.1 client_cookie=9ec739cad82c0417 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868800cd70 con 0x7f8678004e50 2020-07-18T03:26:37.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86880070d0 con 0x7f8678004e50 2020-07-18T03:26:37.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f8695d9b700 1 -- 172.21.15.47:0/4000188183 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8688007430 con 0x7f8678004e50 2020-07-18T03:26:37.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 -- 172.21.15.47:0/4000188183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8678004e50 msgr2=0x7f867800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8678004e50 0x7f867800b730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f8678004450 tx=0x7f8698061f40).stop 2020-07-18T03:26:37.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 -- 172.21.15.47:0/4000188183 shutdown_connections 2020-07-18T03:26:37.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 --2- 172.21.15.47:0/4000188183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8678004e50 0x7f867800b730 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 -- 172.21.15.47:0/4000188183 >> 172.21.15.47:0/4000188183 conn(0x7f869805f5a0 msgr2=0x7f869805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:37.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 -- 172.21.15.47:0/4000188183 shutdown_connections 2020-07-18T03:26:37.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.862+0000 7f869d010700 1 -- 172.21.15.47:0/4000188183 wait complete. 2020-07-18T03:26:37.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.864+0000 7f869d010700 1 Processor -- start 2020-07-18T03:26:37.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.864+0000 7f869d010700 1 -- start start 2020-07-18T03:26:37.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.864+0000 7f869d010700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8698063bf0 0x7f8698065850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869d010700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8698153010 0x7f8698146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869d010700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8698066d70 con 0x7f8698063fd0 2020-07-18T03:26:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869d010700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8698065e40 con 0x7f8698153010 2020-07-18T03:26:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869d010700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86980660e0 con 0x7f8698063bf0 2020-07-18T03:26:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8698063bf0 0x7f8698065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8698063bf0 0x7f8698065850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59952/0 (socket says 172.21.15.47:59952) 2020-07-18T03:26:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 -- 172.21.15.47:0/1266292884 learned_addr learned my addr 172.21.15.47:0/1266292884 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869759e700 1 --2- 172.21.15.47:0/1266292884 >> v2:172.21.15.18:3300/0 conn(0x7f8698153010 0x7f8698146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 -- 172.21.15.47:0/1266292884 >> v2:172.21.15.18:3300/0 conn(0x7f8698153010 msgr2=0x7f8698146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 --2- 172.21.15.47:0/1266292884 >> v2:172.21.15.18:3300/0 conn(0x7f8698153010 0x7f8698146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 -- 172.21.15.47:0/1266292884 >> v1:172.21.15.18:6789/0 conn(0x7f8698063fd0 legacy=0x7f869812da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 -- 172.21.15.47:0/1266292884 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698066380 con 0x7f8698063bf0 2020-07-18T03:26:37.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f8696d9d700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8698063bf0 0x7f8698065850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f869000d000 tx=0x7f8690007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8690006c70 con 0x7f8698063bf0 2020-07-18T03:26:37.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690006dd0 con 0x7f8698063bf0 2020-07-18T03:26:37.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698066820 con 0x7f8698063bf0 2020-07-18T03:26:37.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f869000c910 con 0x7f8698063bf0 2020-07-18T03:26:37.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.865+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869812e380 con 0x7f8698063bf0 2020-07-18T03:26:37.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.866+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8698142430 con 0x7f8698063bf0 2020-07-18T03:26:37.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.866+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f869001b030 con 0x7f8698063bf0 2020-07-18T03:26:37.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.867+0000 7f868effd700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f867404fee0 0x7f86740523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:37.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.867+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8690067910 con 0x7f8698063bf0 2020-07-18T03:26:37.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.867+0000 7f869659c700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f867404fee0 0x7f86740523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:37.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.867+0000 7f869659c700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f867404fee0 0x7f86740523a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8698064c00 tx=0x7f8680008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:37.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:37.870+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8690040380 con 0x7f8698063bf0 2020-07-18T03:26:38.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.027+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8698066510 con 0x7f8698063bf0 2020-07-18T03:26:38.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.028+0000 7f868effd700 1 -- 172.21.15.47:0/1266292884 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8690019070 con 0x7f8698063bf0 2020-07-18T03:26:38.030 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f867404fee0 msgr2=0x7f86740523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f867404fee0 0x7f86740523a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8698064c00 tx=0x7f8680008040).stop 2020-07-18T03:26:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8698063bf0 msgr2=0x7f8698065850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8698063bf0 0x7f8698065850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f869000d000 tx=0x7f8690007ef0).stop 2020-07-18T03:26:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 shutdown_connections 2020-07-18T03:26:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f867404fee0 0x7f86740523a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 --2- 172.21.15.47:0/1266292884 >> v2:172.21.15.18:3300/0 conn(0x7f8698153010 0x7f8698146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 --2- 172.21.15.47:0/1266292884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8698063bf0 0x7f8698065850 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 >> 172.21.15.47:0/1266292884 conn(0x7f869805f5a0 msgr2=0x7f86980b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 shutdown_connections 2020-07-18T03:26:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.029+0000 7f869d010700 1 -- 172.21.15.47:0/1266292884 wait complete. 2020-07-18T03:26:38.041 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-07-18T03:26:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.155+0000 7f83fd524700 1 Processor -- start 2020-07-18T03:26:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.155+0000 7f83fd524700 1 -- start start 2020-07-18T03:26:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83f80b5370 0x7f83f805fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83f80b41c0 0x7f83f80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80b3ac0 con 0x7f83f80b4620 2020-07-18T03:26:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f80b5a10 con 0x7f83f80b5370 2020-07-18T03:26:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80b0040 con 0x7f83f80b41c0 2020-07-18T03:26:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83f80b5370 0x7f83f805fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83f80b5370 0x7f83f805fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59960/0 (socket says 172.21.15.47:59960) 2020-07-18T03:26:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 -- 172.21.15.47:0/1711951052 learned_addr learned my addr 172.21.15.47:0/1711951052 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f67fc700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83f80b41c0 0x7f83f80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 -- 172.21.15.47:0/1711951052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83f80b41c0 msgr2=0x7f83f80ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83f80b41c0 0x7f83f80ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 -- 172.21.15.47:0/1711951052 >> v1:172.21.15.18:6789/0 conn(0x7f83f80b4620 legacy=0x7f83f805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 -- 172.21.15.47:0/1711951052 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80b3d20 con 0x7f83f80b5370 2020-07-18T03:26:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f67fc700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83f80b41c0 0x7f83f80ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f6ffd700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f80b5370 0x7f83f805fff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f83f8067280 tx=0x7f83e8006e90).ready entity=mon.1 client_cookie=5c7882b6e5bad359 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f5ffb700 1 -- 172.21.15.47:0/1711951052 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e80098e0 con 0x7f83f80b5370 2020-07-18T03:26:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f5ffb700 1 -- 172.21.15.47:0/1711951052 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8009a40 con 0x7f83f80b5370 2020-07-18T03:26:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83f5ffb700 1 -- 172.21.15.47:0/1711951052 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e8009de0 con 0x7f83f80b5370 2020-07-18T03:26:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- 172.21.15.47:0/1711951052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f80b5370 msgr2=0x7f83f805fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f80b5370 0x7f83f805fff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f83f8067280 tx=0x7f83e8006e90).stop 2020-07-18T03:26:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- 172.21.15.47:0/1711951052 shutdown_connections 2020-07-18T03:26:38.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f80b5370 0x7f83f805fff0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 --2- 172.21.15.47:0/1711951052 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83f80b41c0 0x7f83f80ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- 172.21.15.47:0/1711951052 >> 172.21.15.47:0/1711951052 conn(0x7f83f80b9340 msgr2=0x7f83f80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:38.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.156+0000 7f83fd524700 1 -- 172.21.15.47:0/1711951052 shutdown_connections 2020-07-18T03:26:38.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.157+0000 7f83fd524700 1 -- 172.21.15.47:0/1711951052 wait complete. 2020-07-18T03:26:38.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.158+0000 7f83fd524700 1 Processor -- start 2020-07-18T03:26:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.158+0000 7f83fd524700 1 -- start start 2020-07-18T03:26:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83fd524700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83f813c630 0x7f83f813b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83fd524700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80b2610 con 0x7f83f8153eb0 2020-07-18T03:26:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83fd524700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80b28b0 con 0x7f83f80b41c0 2020-07-18T03:26:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83fd524700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f813ee10 con 0x7f83f813c630 2020-07-18T03:26:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83f80b41c0 0x7f83f80b2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41054/0 (socket says 172.21.15.47:41054) 2020-07-18T03:26:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f6ffd700 1 -- 172.21.15.47:0/1492402673 learned_addr learned my addr 172.21.15.47:0/1492402673 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f67fc700 1 --2- 172.21.15.47:0/1492402673 >> v2:172.21.15.47:3300/0 conn(0x7f83f813c630 0x7f83f813b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f67fc700 1 -- 172.21.15.47:0/1492402673 >> v1:172.21.15.47:6789/0 conn(0x7f83f80b41c0 legacy=0x7f83f80b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f67fc700 1 -- 172.21.15.47:0/1492402673 >> v1:172.21.15.18:6789/0 conn(0x7f83f8153eb0 legacy=0x7f83f8149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f67fc700 1 -- 172.21.15.47:0/1492402673 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f814a590 con 0x7f83f813c630 2020-07-18T03:26:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448920011 0 0) 0x7f83f80b28b0 con 0x7f83f80b41c0 2020-07-18T03:26:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.159+0000 7f83f67fc700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f813c630 0x7f83f813b770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f83f81428e0 tx=0x7f83ec008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.160+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ec00e570 con 0x7f83f813c630 2020-07-18T03:26:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.160+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00c030 con 0x7f83f813c630 2020-07-18T03:26:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.160+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f813bc60 con 0x7f83f813c630 2020-07-18T03:26:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.160+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ec00e8c0 con 0x7f83f813c630 2020-07-18T03:26:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.160+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f813d020 con 0x7f83f813c630 2020-07-18T03:26:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.161+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f80b2a40 con 0x7f83f813c630 2020-07-18T03:26:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.161+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f83ec019030 con 0x7f83f813c630 2020-07-18T03:26:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.161+0000 7f83deffd700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83d404fe40 0x7f83d4052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.161+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f83ec067190 con 0x7f83f813c630 2020-07-18T03:26:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.161+0000 7f83f6ffd700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83d404fe40 0x7f83d4052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.161+0000 7f83f6ffd700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83d404fe40 0x7f83d4052300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f83e8006e90 tx=0x7f83e8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.164+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f83ec03fd10 con 0x7f83f813c630 2020-07-18T03:26:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.323+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f83f8057450 con 0x7f83f813c630 2020-07-18T03:26:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.323+0000 7f83deffd700 1 -- 172.21.15.47:0/1492402673 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+143 (secure 0 0 0) 0x7f83ec017070 con 0x7f83f813c630 2020-07-18T03:26:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:38.325 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2020-07-18T03:26:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2020-07-18T03:26:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2020-07-18T03:26:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:26:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2020-07-18T03:26:38.327 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2020-07-18T03:26:38.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:38.327 INFO:teuthology.orchestra.run.smithi047.stdout:] 2020-07-18T03:26:38.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83d404fe40 msgr2=0x7f83d4052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83d404fe40 0x7f83d4052300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f83e8006e90 tx=0x7f83e8015000).stop 2020-07-18T03:26:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f813c630 msgr2=0x7f83f813b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f813c630 0x7f83f813b770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f83f81428e0 tx=0x7f83ec008040).stop 2020-07-18T03:26:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 shutdown_connections 2020-07-18T03:26:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83d404fe40 0x7f83d4052300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 --2- 172.21.15.47:0/1492402673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f813c630 0x7f83f813b770 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 >> 172.21.15.47:0/1492402673 conn(0x7f83f80b9340 msgr2=0x7f83f80b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 shutdown_connections 2020-07-18T03:26:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.325+0000 7f83fd524700 1 -- 172.21.15.47:0/1492402673 wait complete. 2020-07-18T03:26:38.336 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-07-18T03:26:38.337 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-07-18T03:26:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.458+0000 7f4e3eb4d700 1 Processor -- start 2020-07-18T03:26:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3eb4d700 1 -- start start 2020-07-18T03:26:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3eb4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e38063e70 0x7f4e38066620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3eb4d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4e38066b10 0x7f4e380b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3eb4d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e38067240 con 0x7f4e38064250 2020-07-18T03:26:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3eb4d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e38067030 con 0x7f4e38066b10 2020-07-18T03:26:38.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3eb4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e380b7df0 con 0x7f4e38063e70 2020-07-18T03:26:38.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e3c8e9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e38063e70 0x7f4e38066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e37fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4e38064250 0x7f4e380b8290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41064/0 (socket says 172.21.15.47:41064) 2020-07-18T03:26:38.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.459+0000 7f4e37fff700 1 -- 172.21.15.47:0/1141824651 learned_addr learned my addr 172.21.15.47:0/1141824651 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:38.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3d0ea700 1 --2- 172.21.15.47:0/1141824651 >> v2:172.21.15.18:3300/0 conn(0x7f4e38066b10 0x7f4e380b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3c8e9700 1 -- 172.21.15.47:0/1141824651 >> v2:172.21.15.18:3300/0 conn(0x7f4e38066b10 msgr2=0x7f4e380b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:38.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3c8e9700 1 --2- 172.21.15.47:0/1141824651 >> v2:172.21.15.18:3300/0 conn(0x7f4e38066b10 0x7f4e380b89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3c8e9700 1 -- 172.21.15.47:0/1141824651 >> v1:172.21.15.47:6789/0 conn(0x7f4e38064250 legacy=0x7f4e380b8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e377fe700 1 -- 172.21.15.47:0/1141824651 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412934221 0 0) 0x7f4e38067240 con 0x7f4e38064250 2020-07-18T03:26:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3c8e9700 1 -- 172.21.15.47:0/1141824651 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380612f0 con 0x7f4e38063e70 2020-07-18T03:26:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3c8e9700 1 --2- 172.21.15.47:0/1141824651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e38063e70 0x7f4e38066620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f4e28002f30 tx=0x7f4e28006dd0).ready entity=mon.1 client_cookie=7e5675fbc35fd8ae server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e377fe700 1 -- 172.21.15.47:0/1141824651 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e280097f0 con 0x7f4e38063e70 2020-07-18T03:26:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e377fe700 1 -- 172.21.15.47:0/1141824651 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28009950 con 0x7f4e38063e70 2020-07-18T03:26:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e377fe700 1 -- 172.21.15.47:0/1141824651 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e28009cb0 con 0x7f4e38063e70 2020-07-18T03:26:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1141824651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e38063e70 msgr2=0x7f4e38066620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1141824651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e38063e70 0x7f4e38066620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f4e28002f30 tx=0x7f4e28006dd0).stop 2020-07-18T03:26:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1141824651 shutdown_connections 2020-07-18T03:26:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1141824651 >> v2:172.21.15.18:3300/0 conn(0x7f4e38066b10 0x7f4e380b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1141824651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e38063e70 0x7f4e38066620 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1141824651 >> 172.21.15.47:0/1141824651 conn(0x7f4e3805f5a0 msgr2=0x7f4e3805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1141824651 shutdown_connections 2020-07-18T03:26:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.460+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1141824651 wait complete. 2020-07-18T03:26:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 Processor -- start 2020-07-18T03:26:38.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 -- start start 2020-07-18T03:26:38.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e38158fd0 0x7f4e381454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e381474c0 0x7f4e38147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e381594f0 con 0x7f4e38064250 2020-07-18T03:26:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e38147e40 con 0x7f4e381474c0 2020-07-18T03:26:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.462+0000 7f4e3eb4d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38146d80 con 0x7f4e38158fd0 2020-07-18T03:26:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3d0ea700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e381474c0 0x7f4e38147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3c8e9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4e38064250 0x7f4e380b66a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41068/0 (socket says 172.21.15.47:41068) 2020-07-18T03:26:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3c8e9700 1 -- 172.21.15.47:0/1617126383 learned_addr learned my addr 172.21.15.47:0/1617126383 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e37fff700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e38158fd0 0x7f4e381454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3d0ea700 1 -- 172.21.15.47:0/1617126383 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e38158fd0 msgr2=0x7f4e381454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3d0ea700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e38158fd0 0x7f4e381454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3d0ea700 1 -- 172.21.15.47:0/1617126383 >> v1:172.21.15.47:6789/0 conn(0x7f4e38064250 legacy=0x7f4e380b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3d0ea700 1 -- 172.21.15.47:0/1617126383 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38147020 con 0x7f4e381474c0 2020-07-18T03:26:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802959336 0 0) 0x7f4e381594f0 con 0x7f4e38064250 2020-07-18T03:26:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3d0ea700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e381474c0 0x7f4e38147950 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f4e30007000 tx=0x7f4e30008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e30007820 con 0x7f4e381474c0 2020-07-18T03:26:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e30007980 con 0x7f4e381474c0 2020-07-18T03:26:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e38144f80 con 0x7f4e381474c0 2020-07-18T03:26:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e30007ce0 con 0x7f4e381474c0 2020-07-18T03:26:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.463+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e38141680 con 0x7f4e381474c0 2020-07-18T03:26:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.464+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e381471b0 con 0x7f4e381474c0 2020-07-18T03:26:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.464+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4e3001b460 con 0x7f4e381474c0 2020-07-18T03:26:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.464+0000 7f4e34ff9700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4e1004fef0 0x7f4e100523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.465+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4e30067d10 con 0x7f4e381474c0 2020-07-18T03:26:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.465+0000 7f4e3c8e9700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4e1004fef0 0x7f4e100523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.465+0000 7f4e3c8e9700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4e1004fef0 0x7f4e100523b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f4e28000b80 tx=0x7f4e28015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:38.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.468+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4e30040780 con 0x7f4e381474c0 2020-07-18T03:26:38.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:38.626+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4e3805b120 con 0x7f4e381474c0 2020-07-18T03:26:39.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.402+0000 7f4e34ff9700 1 -- 172.21.15.47:0/1617126383 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4e30043da0 con 0x7f4e381474c0 2020-07-18T03:26:39.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4e1004fef0 msgr2=0x7f4e100523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:39.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4e1004fef0 0x7f4e100523b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f4e28000b80 tx=0x7f4e28015000).stop 2020-07-18T03:26:39.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e381474c0 msgr2=0x7f4e38147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:39.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e381474c0 0x7f4e38147950 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f4e30007000 tx=0x7f4e30008040).stop 2020-07-18T03:26:39.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 shutdown_connections 2020-07-18T03:26:39.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e381474c0 0x7f4e38147950 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4e1004fef0 0x7f4e100523b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 --2- 172.21.15.47:0/1617126383 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4e38158fd0 0x7f4e381454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.405+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 >> 172.21.15.47:0/1617126383 conn(0x7f4e3805f5a0 msgr2=0x7f4e380614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:39.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.406+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 shutdown_connections 2020-07-18T03:26:39.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.406+0000 7f4e3eb4d700 1 -- 172.21.15.47:0/1617126383 wait complete. 2020-07-18T03:26:39.409 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2020-07-18T03:26:39.419 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-07-18T03:26:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.545+0000 7f42ac6c1700 1 Processor -- start 2020-07-18T03:26:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.545+0000 7f42ac6c1700 1 -- start start 2020-07-18T03:26:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.545+0000 7f42ac6c1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f42a40ad330 0x7f42a4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.545+0000 7f42ac6c1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f42a40b59b0 0x7f42a40b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.545+0000 7f42ac6c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42a40b6300 0x7f42a40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42ac6c1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a4147de0 con 0x7f42a40ad330 2020-07-18T03:26:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42ac6c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a40ae000 con 0x7f42a40b6300 2020-07-18T03:26:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42ac6c1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a4148010 con 0x7f42a40b59b0 2020-07-18T03:26:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42a40b6300 0x7f42a40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42a40b6300 0x7f42a40b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59988/0 (socket says 172.21.15.47:59988) 2020-07-18T03:26:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 -- 172.21.15.47:0/3207950296 learned_addr learned my addr 172.21.15.47:0/3207950296 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42a9c5c700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f42a40b59b0 0x7f42a40b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aa45d700 1 --2- 172.21.15.47:0/3207950296 >> v2:172.21.15.18:3300/0 conn(0x7f42a40ad330 0x7f42a4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 -- 172.21.15.47:0/3207950296 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f42a40b59b0 msgr2=0x7f42a40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f42a40b59b0 0x7f42a40b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 -- 172.21.15.47:0/3207950296 >> v2:172.21.15.18:3300/0 conn(0x7f42a40ad330 msgr2=0x7f42a4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 --2- 172.21.15.47:0/3207950296 >> v2:172.21.15.18:3300/0 conn(0x7f42a40ad330 0x7f42a4143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 -- 172.21.15.47:0/3207950296 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40ad710 con 0x7f42a40b6300 2020-07-18T03:26:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aa45d700 1 --2- 172.21.15.47:0/3207950296 >> v2:172.21.15.18:3300/0 conn(0x7f42a40ad330 0x7f42a4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42a9c5c700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f42a40b59b0 0x7f42a40b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42aac5e700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a40b6300 0x7f42a40b3a10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f429c0032b0 tx=0x7f429c009270).ready entity=mon.1 client_cookie=44a50e2fcdf9b53d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42a945b700 1 -- 172.21.15.47:0/3207950296 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f429c00bc00 con 0x7f42a40b6300 2020-07-18T03:26:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42a945b700 1 -- 172.21.15.47:0/3207950296 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c00bd60 con 0x7f42a40b6300 2020-07-18T03:26:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42a945b700 1 -- 172.21.15.47:0/3207950296 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f429c011400 con 0x7f42a40b6300 2020-07-18T03:26:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42ac6c1700 1 -- 172.21.15.47:0/3207950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a40b6300 msgr2=0x7f42a40b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a40b6300 0x7f42a40b3a10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f429c0032b0 tx=0x7f429c009270).stop 2020-07-18T03:26:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.546+0000 7f42ac6c1700 1 -- 172.21.15.47:0/3207950296 shutdown_connections 2020-07-18T03:26:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.547+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a40b6300 0x7f42a40b3a10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.547+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/3207950296 >> v2:172.21.15.18:3300/0 conn(0x7f42a40ad330 0x7f42a4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.547+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/3207950296 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f42a40b59b0 0x7f42a40b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.547+0000 7f42ac6c1700 1 -- 172.21.15.47:0/3207950296 >> 172.21.15.47:0/3207950296 conn(0x7f42a405f5a0 msgr2=0x7f42a405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.547+0000 7f42ac6c1700 1 -- 172.21.15.47:0/3207950296 shutdown_connections 2020-07-18T03:26:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.547+0000 7f42ac6c1700 1 -- 172.21.15.47:0/3207950296 wait complete. 2020-07-18T03:26:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.548+0000 7f42ac6c1700 1 Processor -- start 2020-07-18T03:26:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.548+0000 7f42ac6c1700 1 -- start start 2020-07-18T03:26:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42ac6c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42a413ebd0 0x7f42a40632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42ac6c1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40b0d60 con 0x7f42a40b59b0 2020-07-18T03:26:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42ac6c1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40b1000 con 0x7f42a40ad330 2020-07-18T03:26:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42ac6c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a40b12a0 con 0x7f42a413ebd0 2020-07-18T03:26:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aa45d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f42a40ad330 0x7f42a413f400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41082/0 (socket says 172.21.15.47:41082) 2020-07-18T03:26:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aa45d700 1 -- 172.21.15.47:0/893906609 learned_addr learned my addr 172.21.15.47:0/893906609 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aac5e700 1 --2- 172.21.15.47:0/893906609 >> v2:172.21.15.47:3300/0 conn(0x7f42a413ebd0 0x7f42a40632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aac5e700 1 -- 172.21.15.47:0/893906609 >> v1:172.21.15.47:6789/0 conn(0x7f42a40ad330 legacy=0x7f42a413f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aac5e700 1 -- 172.21.15.47:0/893906609 >> v1:172.21.15.18:6789/0 conn(0x7f42a40b59b0 legacy=0x7f42a413e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957710339 0 0) 0x7f42a40b1000 con 0x7f42a40ad330 2020-07-18T03:26:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aac5e700 1 -- 172.21.15.47:0/893906609 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40b2be0 con 0x7f42a413ebd0 2020-07-18T03:26:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42aac5e700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a413ebd0 0x7f42a40632d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f429c0032b0 tx=0x7f429c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f429c010a60 con 0x7f42a413ebd0 2020-07-18T03:26:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c010bc0 con 0x7f42a413ebd0 2020-07-18T03:26:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a4064260 con 0x7f42a413ebd0 2020-07-18T03:26:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f429c021480 con 0x7f42a413ebd0 2020-07-18T03:26:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.549+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a4064800 con 0x7f42a413ebd0 2020-07-18T03:26:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.550+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a40649f0 con 0x7f42a413ebd0 2020-07-18T03:26:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.551+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f429c00f070 con 0x7f42a413ebd0 2020-07-18T03:26:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.551+0000 7f42927fc700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f427c04fe10 0x7f427c0522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.551+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f429c069120 con 0x7f42a413ebd0 2020-07-18T03:26:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.551+0000 7f42aa45d700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f427c04fe10 0x7f427c0522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.551+0000 7f42aa45d700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f427c04fe10 0x7f427c0522d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f4294003070 tx=0x7f4294008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.554+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f429c041b30 con 0x7f42a413ebd0 2020-07-18T03:26:39.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:39.710+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f42a4064450 con 0x7f42a413ebd0 2020-07-18T03:26:40.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.409+0000 7f42927fc700 1 -- 172.21.15.47:0/893906609 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v31) v1 ==== 125+0+0 (secure 0 0 0) 0x7f429c015020 con 0x7f42a413ebd0 2020-07-18T03:26:40.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f427c04fe10 msgr2=0x7f427c0522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f427c04fe10 0x7f427c0522d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f4294003070 tx=0x7f4294008040).stop 2020-07-18T03:26:40.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a413ebd0 msgr2=0x7f42a40632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a413ebd0 0x7f42a40632d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f429c0032b0 tx=0x7f429c016000).stop 2020-07-18T03:26:40.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 shutdown_connections 2020-07-18T03:26:40.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42a413ebd0 0x7f42a40632d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 --2- 172.21.15.47:0/893906609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f427c04fe10 0x7f427c0522d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.412+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 >> 172.21.15.47:0/893906609 conn(0x7f42a405f5a0 msgr2=0x7f42a40b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:40.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.413+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 shutdown_connections 2020-07-18T03:26:40.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.413+0000 7f42ac6c1700 1 -- 172.21.15.47:0/893906609 wait complete. 2020-07-18T03:26:40.417 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2020-07-18T03:26:40.427 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-07-18T03:26:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.550+0000 7fc00195f700 1 Processor -- start 2020-07-18T03:26:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.550+0000 7fc00195f700 1 -- start start 2020-07-18T03:26:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fc00195f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbffc064250 0x7fbffc0b8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fc00195f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc066fa0 con 0x7fbffc063e70 2020-07-18T03:26:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fc00195f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc067240 con 0x7fbffc0b8780 2020-07-18T03:26:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fc00195f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0b7df0 con 0x7fbffc064250 2020-07-18T03:26:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fbffa7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbffc064250 0x7fbffc0b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fbffb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbffc0b8780 0x7fbffc1472f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41094/0 (socket says 172.21.15.47:41094) 2020-07-18T03:26:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.551+0000 7fbffb7fe700 1 -- 172.21.15.47:0/3395418069 learned_addr learned my addr 172.21.15.47:0/3395418069 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbffa7fc700 1 -- 172.21.15.47:0/3395418069 >> v1:172.21.15.47:6789/0 conn(0x7fbffc0b8780 legacy=0x7fbffc1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbffa7fc700 1 -- 172.21.15.47:0/3395418069 >> v1:172.21.15.18:6789/0 conn(0x7fbffc063e70 legacy=0x7fbffc066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbffa7fc700 1 -- 172.21.15.47:0/3395418069 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0612f0 con 0x7fbffc064250 2020-07-18T03:26:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbffa7fc700 1 --2- 172.21.15.47:0/3395418069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc064250 0x7fbffc0b8290 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fbffc05d130 tx=0x7fbfec006dd0).ready entity=mon.1 client_cookie=e61661f0769a1f2b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbff9ffb700 1 -- 172.21.15.47:0/3395418069 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfec0097b0 con 0x7fbffc064250 2020-07-18T03:26:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbff9ffb700 1 -- 172.21.15.47:0/3395418069 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec009910 con 0x7fbffc064250 2020-07-18T03:26:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fbff9ffb700 1 -- 172.21.15.47:0/3395418069 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfec009c70 con 0x7fbffc064250 2020-07-18T03:26:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 -- 172.21.15.47:0/3395418069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc064250 msgr2=0x7fbffc0b8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 --2- 172.21.15.47:0/3395418069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc064250 0x7fbffc0b8290 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fbffc05d130 tx=0x7fbfec006dd0).stop 2020-07-18T03:26:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 -- 172.21.15.47:0/3395418069 shutdown_connections 2020-07-18T03:26:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 --2- 172.21.15.47:0/3395418069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc064250 0x7fbffc0b8290 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 -- 172.21.15.47:0/3395418069 >> 172.21.15.47:0/3395418069 conn(0x7fbffc05f5a0 msgr2=0x7fbffc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 -- 172.21.15.47:0/3395418069 shutdown_connections 2020-07-18T03:26:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.552+0000 7fc00195f700 1 -- 172.21.15.47:0/3395418069 wait complete. 2020-07-18T03:26:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 Processor -- start 2020-07-18T03:26:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 -- start start 2020-07-18T03:26:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbffc063e70 0x7fbffc0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbffc0b8780 0x7fbffc0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbffc0b1ae0 0x7fbffc14a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0b1f60 con 0x7fbffc0b8780 2020-07-18T03:26:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc1463d0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fc00195f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc144810 con 0x7fbffc063e70 2020-07-18T03:26:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fbffb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbffc0b1ae0 0x7fbffc14a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fbffb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbffc0b1ae0 0x7fbffc14a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60010/0 (socket says 172.21.15.47:60010) 2020-07-18T03:26:40.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.554+0000 7fbffb7fe700 1 -- 172.21.15.47:0/3703003362 learned_addr learned my addr 172.21.15.47:0/3703003362 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:40.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffa7fc700 1 --2- 172.21.15.47:0/3703003362 >> v2:172.21.15.18:3300/0 conn(0x7fbffc0b8780 0x7fbffc0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffaffd700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbffc063e70 0x7fbffc0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffb7fe700 1 -- 172.21.15.47:0/3703003362 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbffc063e70 msgr2=0x7fbffc0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffb7fe700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbffc063e70 0x7fbffc0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffb7fe700 1 -- 172.21.15.47:0/3703003362 >> v2:172.21.15.18:3300/0 conn(0x7fbffc0b8780 msgr2=0x7fbffc0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffb7fe700 1 --2- 172.21.15.47:0/3703003362 >> v2:172.21.15.18:3300/0 conn(0x7fbffc0b8780 0x7fbffc0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffb7fe700 1 -- 172.21.15.47:0/3703003362 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc144ab0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffaffd700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbffc063e70 0x7fbffc0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbffb7fe700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc0b1ae0 0x7fbffc14a7b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fbffc0bb500 tx=0x7fbff4007e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbff4005b80 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4005ce0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc146050 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbff4007300 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.555+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc14ae60 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.556+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc0b9ab0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.556+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbff4010030 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.556+0000 7fbfe2ffd700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbfd8050090 0x7fbfd8052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.557+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbff4067cb0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.557+0000 7fbffaffd700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbfd8050090 0x7fbfd8052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.557+0000 7fbffaffd700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbfd8050090 0x7fbfd8052550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fbff0008000 tx=0x7fbff0009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.560+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbff40404c0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.768+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fbffc144c40 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.774+0000 7fbfe2ffd700 1 -- 172.21.15.47:0/3703003362 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v10) v1 ==== 187+0+0 (secure 0 0 0) 0x7fbff4043ae0 con 0x7fbffc0b1ae0 2020-07-18T03:26:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.778+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbfd8050090 msgr2=0x7fbfd8052550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.778+0000 7fc00195f700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbfd8050090 0x7fbfd8052550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fbff0008000 tx=0x7fbff0009040).stop 2020-07-18T03:26:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.778+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc0b1ae0 msgr2=0x7fbffc14a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.778+0000 7fc00195f700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc0b1ae0 0x7fbffc14a7b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fbffc0bb500 tx=0x7fbff4007e60).stop 2020-07-18T03:26:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.779+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 shutdown_connections 2020-07-18T03:26:40.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.779+0000 7fc00195f700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbfd8050090 0x7fbfd8052550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.779+0000 7fc00195f700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbffc0b1ae0 0x7fbffc14a7b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.779+0000 7fc00195f700 1 --2- 172.21.15.47:0/3703003362 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbffc063e70 0x7fbffc0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.779+0000 7fc00195f700 1 --2- 172.21.15.47:0/3703003362 >> v2:172.21.15.18:3300/0 conn(0x7fbffc0b8780 0x7fbffc0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.779+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 >> 172.21.15.47:0/3703003362 conn(0x7fbffc05f5a0 msgr2=0x7fbffc0b93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.780+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 shutdown_connections 2020-07-18T03:26:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.780+0000 7fc00195f700 1 -- 172.21.15.47:0/3703003362 wait complete. 2020-07-18T03:26:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 5 and data pool 6 2020-07-18T03:26:40.803 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.917+0000 7fdd837d6700 1 Processor -- start 2020-07-18T03:26:40.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.917+0000 7fdd837d6700 1 -- start start 2020-07-18T03:26:40.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd837d6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd7c0ad100 0x7fdd7c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd837d6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdd7c147a00 0x7fdd7c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd837d6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c05f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd837d6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c0ae530 con 0x7fdd7c147a00 2020-07-18T03:26:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd837d6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c147330 con 0x7fdd7c0ad100 2020-07-18T03:26:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd837d6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0b57e0 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd81572700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd7c0ad100 0x7fdd7c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd81572700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd7c0ad100 0x7fdd7c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60024/0 (socket says 172.21.15.47:60024) 2020-07-18T03:26:40.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.918+0000 7fdd81572700 1 -- 172.21.15.47:0/3806963491 learned_addr learned my addr 172.21.15.47:0/3806963491 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:40.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd80d71700 1 --2- 172.21.15.47:0/3806963491 >> v2:172.21.15.18:3300/0 conn(0x7fdd7c147a00 0x7fdd7c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81d73700 1 --2- 172.21.15.47:0/3806963491 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c05f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81572700 1 -- 172.21.15.47:0/3806963491 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 msgr2=0x7fdd7c05f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:40.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81572700 1 --2- 172.21.15.47:0/3806963491 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c05f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81572700 1 -- 172.21.15.47:0/3806963491 >> v2:172.21.15.18:3300/0 conn(0x7fdd7c147a00 msgr2=0x7fdd7c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:40.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81572700 1 --2- 172.21.15.47:0/3806963491 >> v2:172.21.15.18:3300/0 conn(0x7fdd7c147a00 0x7fdd7c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81572700 1 -- 172.21.15.47:0/3806963491 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0b5a80 con 0x7fdd7c0ad100 2020-07-18T03:26:40.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd80d71700 1 --2- 172.21.15.47:0/3806963491 >> v2:172.21.15.18:3300/0 conn(0x7fdd7c147a00 0x7fdd7c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd81572700 1 --2- 172.21.15.47:0/3806963491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd7c0ad100 0x7fdd7c0b4c10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fdd740032b0 tx=0x7fdd74009270).ready entity=mon.1 client_cookie=df4df787f2379adb server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd6bfff700 1 -- 172.21.15.47:0/3806963491 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd7400bc00 con 0x7fdd7c0ad100 2020-07-18T03:26:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd6bfff700 1 -- 172.21.15.47:0/3806963491 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7400bd60 con 0x7fdd7c0ad100 2020-07-18T03:26:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd6bfff700 1 -- 172.21.15.47:0/3806963491 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd74011400 con 0x7fdd7c0ad100 2020-07-18T03:26:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 -- 172.21.15.47:0/3806963491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd7c0ad100 msgr2=0x7fdd7c0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 --2- 172.21.15.47:0/3806963491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd7c0ad100 0x7fdd7c0b4c10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fdd740032b0 tx=0x7fdd74009270).stop 2020-07-18T03:26:40.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 -- 172.21.15.47:0/3806963491 shutdown_connections 2020-07-18T03:26:40.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 --2- 172.21.15.47:0/3806963491 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c05f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 --2- 172.21.15.47:0/3806963491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd7c0ad100 0x7fdd7c0b4c10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 --2- 172.21.15.47:0/3806963491 >> v2:172.21.15.18:3300/0 conn(0x7fdd7c147a00 0x7fdd7c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:40.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 -- 172.21.15.47:0/3806963491 >> 172.21.15.47:0/3806963491 conn(0x7fdd7c0b9340 msgr2=0x7fdd7c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:40.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 -- 172.21.15.47:0/3806963491 shutdown_connections 2020-07-18T03:26:40.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.919+0000 7fdd837d6700 1 -- 172.21.15.47:0/3806963491 wait complete. 2020-07-18T03:26:40.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.921+0000 7fdd837d6700 1 Processor -- start 2020-07-18T03:26:40.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.921+0000 7fdd837d6700 1 -- start start 2020-07-18T03:26:40.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.921+0000 7fdd837d6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c13b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.921+0000 7fdd837d6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c13c750 con 0x7fdd7c147a00 2020-07-18T03:26:40.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd837d6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c1588d0 con 0x7fdd7c13c2b0 2020-07-18T03:26:40.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd837d6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c13c9f0 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81d73700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdd7c13c2b0 0x7fdd7c158d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41120/0 (socket says 172.21.15.47:41120) 2020-07-18T03:26:40.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81d73700 1 -- 172.21.15.47:0/4107161747 learned_addr learned my addr 172.21.15.47:0/4107161747 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:40.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81572700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c13b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021843462 0 0) 0x7fdd7c1588d0 con 0x7fdd7c13c2b0 2020-07-18T03:26:40.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd54003240 con 0x7fdd7c13c2b0 2020-07-18T03:26:40.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27199113 0 0) 0x7fdd7c13c750 con 0x7fdd7c147a00 2020-07-18T03:26:40.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7c1588d0 con 0x7fdd7c147a00 2020-07-18T03:26:40.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81572700 1 -- 172.21.15.47:0/4107161747 >> v1:172.21.15.47:6789/0 conn(0x7fdd7c13c2b0 legacy=0x7fdd7c158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81572700 1 -- 172.21.15.47:0/4107161747 >> v1:172.21.15.18:6789/0 conn(0x7fdd7c147a00 legacy=0x7fdd7c13f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81572700 1 -- 172.21.15.47:0/4107161747 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0b3240 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd81572700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c13b930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fdd740032b0 tx=0x7fdd74016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd740107a0 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74010900 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.922+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c13be20 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.923+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd74010c60 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.923+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c1596c0 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.924+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c158a60 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.924+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fdd7400f070 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.924+0000 7fdd697fa700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdd54050450 0x7fdd54052910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.924+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdd74068970 con 0x7fdd7c05f5a0 2020-07-18T03:26:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.924+0000 7fdd80d71700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdd54050450 0x7fdd54052910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.924+0000 7fdd80d71700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdd54050450 0x7fdd54052910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fdd6c007000 tx=0x7fdd6c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:40.928+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fdd74041080 con 0x7fdd7c05f5a0 2020-07-18T03:26:41.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.093+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd7c057450 con 0x7fdd7c05f5a0 2020-07-18T03:26:41.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.095+0000 7fdd697fa700 1 -- 172.21.15.47:0/4107161747 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fdd74015020 con 0x7fdd7c05f5a0 2020-07-18T03:26:41.096 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:41.096 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:26:40.770311+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:26:38.665431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-07-18T03:26:39.748890+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:41.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdd54050450 msgr2=0x7fdd54052910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdd54050450 0x7fdd54052910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fdd6c007000 tx=0x7fdd6c008040).stop 2020-07-18T03:26:41.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 msgr2=0x7fdd7c13b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c13b930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fdd740032b0 tx=0x7fdd74016000).stop 2020-07-18T03:26:41.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 shutdown_connections 2020-07-18T03:26:41.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdd54050450 0x7fdd54052910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 --2- 172.21.15.47:0/4107161747 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdd7c05f5a0 0x7fdd7c13b930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 >> 172.21.15.47:0/4107161747 conn(0x7fdd7c0b9340 msgr2=0x7fdd7c05fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:41.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 shutdown_connections 2020-07-18T03:26:41.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.096+0000 7fdd837d6700 1 -- 172.21.15.47:0/4107161747 wait complete. 2020-07-18T03:26:41.108 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:41.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.223+0000 7f4f170da700 1 Processor -- start 2020-07-18T03:26:41.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.223+0000 7f4f170da700 1 -- start start 2020-07-18T03:26:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f170da700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4f100b5370 0x7f4f1005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f170da700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f101428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f170da700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f100b3ac0 con 0x7f4f100b41c0 2020-07-18T03:26:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f170da700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f100b5a10 con 0x7f4f100b5370 2020-07-18T03:26:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f170da700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100b0040 con 0x7f4f100ad9e0 2020-07-18T03:26:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4f100b5370 0x7f4f1005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f15677700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f101428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f0ffff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f4f100b41c0 0x7f4f100ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59890/0 (socket says 172.21.15.47:59890) 2020-07-18T03:26:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f0ffff700 1 -- 172.21.15.47:0/4150844258 learned_addr learned my addr 172.21.15.47:0/4150844258 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4f100b5370 0x7f4f1005fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:45716/0 (socket says 172.21.15.47:45716) 2020-07-18T03:26:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f15677700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f101428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:52638/0 (socket says 172.21.15.47:52638) 2020-07-18T03:26:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 -- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 msgr2=0x7f4f101428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 --2- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f101428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 -- 172.21.15.47:0/4150844258 >> v1:172.21.15.18:6789/0 conn(0x7f4f100b41c0 legacy=0x7f4f100ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 -- 172.21.15.47:0/4150844258 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100b3d20 con 0x7f4f100b5370 2020-07-18T03:26:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.224+0000 7f4f14e76700 1 --2- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f100b5370 0x7f4f1005fff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f4f100afb30 tx=0x7f4f04006dd0).ready entity=mon.0 client_cookie=84b64a4824b6a47c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f0f7fe700 1 -- 172.21.15.47:0/4150844258 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f040097b0 con 0x7f4f100b5370 2020-07-18T03:26:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f0f7fe700 1 -- 172.21.15.47:0/4150844258 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f04009910 con 0x7f4f100b5370 2020-07-18T03:26:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f0f7fe700 1 -- 172.21.15.47:0/4150844258 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f04009c70 con 0x7f4f100b5370 2020-07-18T03:26:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 -- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f100b5370 msgr2=0x7f4f1005fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 --2- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f100b5370 0x7f4f1005fff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f4f100afb30 tx=0x7f4f04006dd0).stop 2020-07-18T03:26:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 -- 172.21.15.47:0/4150844258 shutdown_connections 2020-07-18T03:26:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 --2- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f101428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 --2- 172.21.15.47:0/4150844258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f100b5370 0x7f4f1005fff0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 -- 172.21.15.47:0/4150844258 >> 172.21.15.47:0/4150844258 conn(0x7f4f100b9340 msgr2=0x7f4f100b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 -- 172.21.15.47:0/4150844258 shutdown_connections 2020-07-18T03:26:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.225+0000 7f4f170da700 1 -- 172.21.15.47:0/4150844258 wait complete. 2020-07-18T03:26:41.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.226+0000 7f4f170da700 1 Processor -- start 2020-07-18T03:26:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.226+0000 7f4f170da700 1 -- start start 2020-07-18T03:26:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f170da700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f1012ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f170da700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1013f510 con 0x7f4f100b41c0 2020-07-18T03:26:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f170da700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f10140900 con 0x7f4f10142b00 2020-07-18T03:26:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f170da700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f101401c0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f15677700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4f10142b00 0x7f4f10142f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41134/0 (socket says 172.21.15.47:41134) 2020-07-18T03:26:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f15677700 1 -- 172.21.15.47:0/1145585309 learned_addr learned my addr 172.21.15.47:0/1145585309 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f14e76700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f1012ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093444264 0 0) 0x7f4f10140900 con 0x7f4f10142b00 2020-07-18T03:26:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee8003240 con 0x7f4f10142b00 2020-07-18T03:26:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.227+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409099313 0 0) 0x7f4f1013f510 con 0x7f4f100b41c0 2020-07-18T03:26:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f10140900 con 0x7f4f100b41c0 2020-07-18T03:26:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f14e76700 1 -- 172.21.15.47:0/1145585309 >> v1:172.21.15.47:6789/0 conn(0x7f4f10142b00 legacy=0x7f4f10142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f14e76700 1 -- 172.21.15.47:0/1145585309 >> v1:172.21.15.18:6789/0 conn(0x7f4f100b41c0 legacy=0x7f4f1013cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f14e76700 1 -- 172.21.15.47:0/1145585309 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10140460 con 0x7f4f100ad9e0 2020-07-18T03:26:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f14e76700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f1012ed90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f4f100afb30 tx=0x7f4f04015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f040097b0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0401e460 con 0x7f4f100ad9e0 2020-07-18T03:26:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1012f280 con 0x7f4f100ad9e0 2020-07-18T03:26:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f0401e7c0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.228+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1013d4f0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.229+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f10057450 con 0x7f4f100ad9e0 2020-07-18T03:26:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.229+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4f04025030 con 0x7f4f100ad9e0 2020-07-18T03:26:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.230+0000 7f4f0cff9700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ee8050460 0x7f4ee8052920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.230+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4f04069350 con 0x7f4f100ad9e0 2020-07-18T03:26:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.230+0000 7f4f0ffff700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ee8050460 0x7f4ee8052920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.230+0000 7f4f0ffff700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ee8050460 0x7f4ee8052920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4efc001000 tx=0x7f4efc0078c0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.233+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4f040419c0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.396+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f101405f0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.398+0000 7f4f0cff9700 1 -- 172.21.15.47:0/1145585309 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f4f04044fe0 con 0x7f4f100ad9e0 2020-07-18T03:26:41.398 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:41.399 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:26:40.770311+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:26:38.665431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-07-18T03:26:39.748890+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:41.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.398+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ee8050460 msgr2=0x7f4ee8052920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ee8050460 0x7f4ee8052920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4efc001000 tx=0x7f4efc0078c0).stop 2020-07-18T03:26:41.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 msgr2=0x7f4f1012ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f1012ed90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f4f100afb30 tx=0x7f4f04015000).stop 2020-07-18T03:26:41.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 shutdown_connections 2020-07-18T03:26:41.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4ee8050460 0x7f4ee8052920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 --2- 172.21.15.47:0/1145585309 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4f100ad9e0 0x7f4f1012ed90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 >> 172.21.15.47:0/1145585309 conn(0x7f4f100b9340 msgr2=0x7f4f100b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:41.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 shutdown_connections 2020-07-18T03:26:41.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.399+0000 7f4f170da700 1 -- 172.21.15.47:0/1145585309 wait complete. 2020-07-18T03:26:41.410 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-07-18T03:26:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.527+0000 7fbc74c9f700 1 Processor -- start 2020-07-18T03:26:41.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.527+0000 7fbc74c9f700 1 -- start start 2020-07-18T03:26:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc700ad330 0x7fbc70143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc700b3a10 0x7fbc700b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc70147de0 con 0x7fbc700b59b0 2020-07-18T03:26:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc700ae000 con 0x7fbc700ad330 2020-07-18T03:26:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70148010 con 0x7fbc700b3a10 2020-07-18T03:26:41.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc700ad330 0x7fbc70143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc700ad330 0x7fbc70143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60050/0 (socket says 172.21.15.47:60050) 2020-07-18T03:26:41.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 -- 172.21.15.47:0/356025132 learned_addr learned my addr 172.21.15.47:0/356025132 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:41.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 -- 172.21.15.47:0/356025132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc700b3a10 msgr2=0x7fbc700b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:41.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 --2- 172.21.15.47:0/356025132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc700b3a10 0x7fbc700b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:26:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 -- 172.21.15.47:0/356025132 >> v1:172.21.15.18:6789/0 conn(0x7fbc700b59b0 legacy=0x7fbc700b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 -- 172.21.15.47:0/356025132 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc700ad710 con 0x7fbc700ad330 2020-07-18T03:26:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6e59c700 1 --2- 172.21.15.47:0/356025132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700ad330 0x7fbc70143640 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fbc70067970 tx=0x7fbc68006e90).ready entity=mon.1 client_cookie=4cb0309753cce78e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6d59a700 1 -- 172.21.15.47:0/356025132 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc680098e0 con 0x7fbc700ad330 2020-07-18T03:26:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6d59a700 1 -- 172.21.15.47:0/356025132 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68009a40 con 0x7fbc700ad330 2020-07-18T03:26:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc6d59a700 1 -- 172.21.15.47:0/356025132 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc68009de0 con 0x7fbc700ad330 2020-07-18T03:26:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- 172.21.15.47:0/356025132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700ad330 msgr2=0x7fbc70143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/356025132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700ad330 0x7fbc70143640 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fbc70067970 tx=0x7fbc68006e90).stop 2020-07-18T03:26:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- 172.21.15.47:0/356025132 shutdown_connections 2020-07-18T03:26:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/356025132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc700b3a10 0x7fbc700b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:26:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/356025132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700ad330 0x7fbc70143640 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- 172.21.15.47:0/356025132 >> 172.21.15.47:0/356025132 conn(0x7fbc7005f5a0 msgr2=0x7fbc7005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- 172.21.15.47:0/356025132 shutdown_connections 2020-07-18T03:26:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.528+0000 7fbc74c9f700 1 -- 172.21.15.47:0/356025132 wait complete. 2020-07-18T03:26:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.530+0000 7fbc74c9f700 1 Processor -- start 2020-07-18T03:26:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.530+0000 7fbc74c9f700 1 -- start start 2020-07-18T03:26:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc700b3a10 0x7fbc70142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbc701386a0 0x7fbc70138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7013bb50 con 0x7fbc700b59b0 2020-07-18T03:26:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7013b940 con 0x7fbc701386a0 2020-07-18T03:26:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7013bdf0 con 0x7fbc700b3a10 2020-07-18T03:26:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc700b3a10 0x7fbc70142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc700b3a10 0x7fbc70142880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60056/0 (socket says 172.21.15.47:60056) 2020-07-18T03:26:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 -- 172.21.15.47:0/2283158638 learned_addr learned my addr 172.21.15.47:0/2283158638 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6ed9d700 1 --2- 172.21.15.47:0/2283158638 >> v2:172.21.15.18:3300/0 conn(0x7fbc701386a0 0x7fbc70138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 -- 172.21.15.47:0/2283158638 >> v2:172.21.15.18:3300/0 conn(0x7fbc701386a0 msgr2=0x7fbc70138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 --2- 172.21.15.47:0/2283158638 >> v2:172.21.15.18:3300/0 conn(0x7fbc701386a0 0x7fbc70138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 -- 172.21.15.47:0/2283158638 >> v1:172.21.15.18:6789/0 conn(0x7fbc700b59b0 legacy=0x7fbc70137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 -- 172.21.15.47:0/2283158638 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70142d70 con 0x7fbc700b3a10 2020-07-18T03:26:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6ed9d700 1 --2- 172.21.15.47:0/2283158638 >> v2:172.21.15.18:3300/0 conn(0x7fbc701386a0 0x7fbc70138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc6e59c700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700b3a10 0x7fbc70142880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fbc70067970 tx=0x7fbc68015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc68009e80 con 0x7fbc700b3a10 2020-07-18T03:26:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6801d5d0 con 0x7fbc700b3a10 2020-07-18T03:26:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7013a960 con 0x7fbc700b3a10 2020-07-18T03:26:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc6801d930 con 0x7fbc700b3a10 2020-07-18T03:26:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.531+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc70066c20 con 0x7fbc700b3a10 2020-07-18T03:26:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.532+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7013bf30 con 0x7fbc700b3a10 2020-07-18T03:26:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.532+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbc68025030 con 0x7fbc700b3a10 2020-07-18T03:26:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.533+0000 7fbc667fc700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbc4c04ff80 0x7fbc4c052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.533+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbc680694b0 con 0x7fbc700b3a10 2020-07-18T03:26:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.533+0000 7fbc6dd9b700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbc4c04ff80 0x7fbc4c052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.533+0000 7fbc6dd9b700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbc4c04ff80 0x7fbc4c052440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fbc7013a890 tx=0x7fbc580069f0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.536+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbc68041c40 con 0x7fbc700b3a10 2020-07-18T03:26:41.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.747+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbc70062800 con 0x7fbc700b3a10 2020-07-18T03:26:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.773+0000 7fbc667fc700 1 -- 172.21.15.47:0/2283158638 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbc68045260 con 0x7fbc700b3a10 2020-07-18T03:26:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbc4c04ff80 msgr2=0x7fbc4c052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbc4c04ff80 0x7fbc4c052440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fbc7013a890 tx=0x7fbc580069f0).stop 2020-07-18T03:26:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700b3a10 msgr2=0x7fbc70142880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700b3a10 0x7fbc70142880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fbc70067970 tx=0x7fbc68015000).stop 2020-07-18T03:26:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 shutdown_connections 2020-07-18T03:26:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbc4c04ff80 0x7fbc4c052440 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/2283158638 >> v2:172.21.15.18:3300/0 conn(0x7fbc701386a0 0x7fbc70138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 --2- 172.21.15.47:0/2283158638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc700b3a10 0x7fbc70142880 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 >> 172.21.15.47:0/2283158638 conn(0x7fbc7005f5a0 msgr2=0x7fbc700af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 shutdown_connections 2020-07-18T03:26:41.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.775+0000 7fbc74c9f700 1 -- 172.21.15.47:0/2283158638 wait complete. 2020-07-18T03:26:41.787 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 Processor -- start 2020-07-18T03:26:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 -- start start 2020-07-18T03:26:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b41c0 0x7ff90c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b4620 0x7ff90c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c0b3ac0 con 0x7ff90c0b5370 2020-07-18T03:26:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c0b5a10 con 0x7ff90c0b4620 2020-07-18T03:26:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.901+0000 7ff912538700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0b0040 con 0x7ff90c0b41c0 2020-07-18T03:26:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff90c0b5370 0x7ff90c05fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41152/0 (socket says 172.21.15.47:41152) 2020-07-18T03:26:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90bfff700 1 -- 172.21.15.47:0/2021014586 learned_addr learned my addr 172.21.15.47:0/2021014586 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90b7fe700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b41c0 0x7ff90c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff910ad5700 1 --2- 172.21.15.47:0/2021014586 >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b4620 0x7ff90c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90affd700 1 -- 172.21.15.47:0/2021014586 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152356737 0 0) 0x7ff90c0b3ac0 con 0x7ff90c0b5370 2020-07-18T03:26:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90affd700 1 -- 172.21.15.47:0/2021014586 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f8003240 con 0x7ff90c0b5370 2020-07-18T03:26:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90affd700 1 -- 172.21.15.47:0/2021014586 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1291198477 0 0) 0x7ff8f8003240 con 0x7ff90c0b5370 2020-07-18T03:26:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff910ad5700 1 -- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b41c0 msgr2=0x7ff90c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff910ad5700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b41c0 0x7ff90c0ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff910ad5700 1 -- 172.21.15.47:0/2021014586 >> v1:172.21.15.47:6789/0 conn(0x7ff90c0b5370 legacy=0x7ff90c05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff910ad5700 1 -- 172.21.15.47:0/2021014586 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0adcb0 con 0x7ff90c0b4620 2020-07-18T03:26:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90b7fe700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b41c0 0x7ff90c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff910ad5700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff90c0b4620 0x7ff90c0ad7c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff8fc001870 tx=0x7ff9040091d0).ready entity=mon.0 client_cookie=a52f3f126cf1697a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.902+0000 7ff90affd700 1 -- 172.21.15.47:0/2021014586 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8fc0016d0 con 0x7ff90c0b4620 2020-07-18T03:26:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff90affd700 1 -- 172.21.15.47:0/2021014586 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90400b800 con 0x7ff90c0b4620 2020-07-18T03:26:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff90affd700 1 -- 172.21.15.47:0/2021014586 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90400b970 con 0x7ff90c0b4620 2020-07-18T03:26:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 -- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff90c0b4620 msgr2=0x7ff90c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff90c0b4620 0x7ff90c0ad7c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff8fc001870 tx=0x7ff9040091d0).stop 2020-07-18T03:26:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 -- 172.21.15.47:0/2021014586 shutdown_connections 2020-07-18T03:26:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff90c0b4620 0x7ff90c0ad7c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 --2- 172.21.15.47:0/2021014586 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b41c0 0x7ff90c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 -- 172.21.15.47:0/2021014586 >> 172.21.15.47:0/2021014586 conn(0x7ff90c0b9340 msgr2=0x7ff90c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 -- 172.21.15.47:0/2021014586 shutdown_connections 2020-07-18T03:26:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.903+0000 7ff912538700 1 -- 172.21.15.47:0/2021014586 wait complete. 2020-07-18T03:26:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.904+0000 7ff912538700 1 Processor -- start 2020-07-18T03:26:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.904+0000 7ff912538700 1 -- start start 2020-07-18T03:26:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff912538700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b41c0 0x7ff90c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff912538700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b5370 0x7ff90c13c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff912538700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c13cf00 con 0x7ff90c065ee0 2020-07-18T03:26:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff912538700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c1411c0 con 0x7ff90c0b41c0 2020-07-18T03:26:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff912538700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c13d1a0 con 0x7ff90c0b5370 2020-07-18T03:26:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff90bfff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b41c0 0x7ff90c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b5370 0x7ff90c13c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff910ad5700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ff90c065ee0 0x7ff90c140ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59926/0 (socket says 172.21.15.47:59926) 2020-07-18T03:26:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.905+0000 7ff910ad5700 1 -- 172.21.15.47:0/3342162024 learned_addr learned my addr 172.21.15.47:0/3342162024 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:41.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff90b7fe700 1 -- 172.21.15.47:0/3342162024 >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b41c0 msgr2=0x7ff90c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff90b7fe700 1 --2- 172.21.15.47:0/3342162024 >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b41c0 0x7ff90c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:41.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff90b7fe700 1 -- 172.21.15.47:0/3342162024 >> v1:172.21.15.18:6789/0 conn(0x7ff90c065ee0 legacy=0x7ff90c140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:41.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff90b7fe700 1 -- 172.21.15.47:0/3342162024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c13b680 con 0x7ff90c0b5370 2020-07-18T03:26:41.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff90b7fe700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b5370 0x7ff90c13c860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff90000c000 tx=0x7ff900008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90000cb90 con 0x7ff90c0b5370 2020-07-18T03:26:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000ccf0 con 0x7ff90c0b5370 2020-07-18T03:26:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c066390 con 0x7ff90c0b5370 2020-07-18T03:26:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9000125e0 con 0x7ff90c0b5370 2020-07-18T03:26:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.906+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c139ce0 con 0x7ff90c0b5370 2020-07-18T03:26:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.907+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c13cd50 con 0x7ff90c0b5370 2020-07-18T03:26:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.907+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff900012790 con 0x7ff90c0b5370 2020-07-18T03:26:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.907+0000 7ff8ebfff700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8e4050040 0x7ff8e4052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.907+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff900068250 con 0x7ff90c0b5370 2020-07-18T03:26:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.907+0000 7ff90bfff700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8e4050040 0x7ff8e4052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.908+0000 7ff90bfff700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8e4050040 0x7ff8e4052500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff8fc00d000 tx=0x7ff8fc000b80).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:41.911+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff900040940 con 0x7ff90c0b5370 2020-07-18T03:26:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.120+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff90c13b810 con 0x7ff90c0b5370 2020-07-18T03:26:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.121+0000 7ff8ebfff700 1 -- 172.21.15.47:0/3342162024 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7ff900016020 con 0x7ff90c0b5370 2020-07-18T03:26:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:42.123 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:26:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8e4050040 msgr2=0x7ff8e4052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8e4050040 0x7ff8e4052500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff8fc00d000 tx=0x7ff8fc000b80).stop 2020-07-18T03:26:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b5370 msgr2=0x7ff90c13c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b5370 0x7ff90c13c860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff90000c000 tx=0x7ff900008040).stop 2020-07-18T03:26:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 shutdown_connections 2020-07-18T03:26:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8e4050040 0x7ff8e4052500 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 --2- 172.21.15.47:0/3342162024 >> v2:172.21.15.18:3300/0 conn(0x7ff90c0b41c0 0x7ff90c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 --2- 172.21.15.47:0/3342162024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff90c0b5370 0x7ff90c13c860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 >> 172.21.15.47:0/3342162024 conn(0x7ff90c0b9340 msgr2=0x7ff90c14d590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 shutdown_connections 2020-07-18T03:26:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.122+0000 7ff912538700 1 -- 172.21.15.47:0/3342162024 wait complete. 2020-07-18T03:26:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:26:42.138 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:42.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 Processor -- start 2020-07-18T03:26:42.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- start start 2020-07-18T03:26:42.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83c8147e40 0x7f83c8143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c8147330 con 0x7f83c80ad100 2020-07-18T03:26:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c80b57e0 con 0x7f83c8147a00 2020-07-18T03:26:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c80ae530 con 0x7f83c8147e40 2020-07-18T03:26:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83c8147e40 0x7f83c8143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83c8147a00 0x7f83c81433e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41166/0 (socket says 172.21.15.47:41166) 2020-07-18T03:26:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c6ffd700 1 -- 172.21.15.47:0/4001740322 learned_addr learned my addr 172.21.15.47:0/4001740322 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c7fff700 1 -- 172.21.15.47:0/4001740322 >> v1:172.21.15.47:6789/0 conn(0x7f83c8147a00 legacy=0x7f83c81433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c7fff700 1 -- 172.21.15.47:0/4001740322 >> v1:172.21.15.18:6789/0 conn(0x7f83c80ad100 legacy=0x7f83c80b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c7fff700 1 -- 172.21.15.47:0/4001740322 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80b5a80 con 0x7f83c8147e40 2020-07-18T03:26:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c67fc700 1 -- 172.21.15.47:0/4001740322 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884555922 0 0) 0x7f83c80b57e0 con 0x7f83c8147a00 2020-07-18T03:26:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c7fff700 1 --2- 172.21.15.47:0/4001740322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c8147e40 0x7f83c8143af0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f83c00032b0 tx=0x7f83c00091d0).ready entity=mon.1 client_cookie=5582524367c5b052 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c67fc700 1 -- 172.21.15.47:0/4001740322 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c000ba50 con 0x7f83c8147e40 2020-07-18T03:26:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c67fc700 1 -- 172.21.15.47:0/4001740322 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c000bbb0 con 0x7f83c8147e40 2020-07-18T03:26:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83c67fc700 1 -- 172.21.15.47:0/4001740322 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c000bd20 con 0x7f83c8147e40 2020-07-18T03:26:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- 172.21.15.47:0/4001740322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c8147e40 msgr2=0x7f83c8143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/4001740322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c8147e40 0x7f83c8143af0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f83c00032b0 tx=0x7f83c00091d0).stop 2020-07-18T03:26:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- 172.21.15.47:0/4001740322 shutdown_connections 2020-07-18T03:26:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/4001740322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c8147e40 0x7f83c8143af0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- 172.21.15.47:0/4001740322 >> 172.21.15.47:0/4001740322 conn(0x7f83c80b9340 msgr2=0x7f83c80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- 172.21.15.47:0/4001740322 shutdown_connections 2020-07-18T03:26:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.253+0000 7f83cdfb1700 1 -- 172.21.15.47:0/4001740322 wait complete. 2020-07-18T03:26:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.255+0000 7f83cdfb1700 1 Processor -- start 2020-07-18T03:26:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.255+0000 7f83cdfb1700 1 -- start start 2020-07-18T03:26:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.255+0000 7f83cdfb1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83c80ad100 0x7f83c80b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83cdfb1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83c8155450 0x7f83c81524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83cdfb1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c8155dd0 con 0x7f83c8147a00 2020-07-18T03:26:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83cdfb1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c8156070 con 0x7f83c80ad100 2020-07-18T03:26:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83cdfb1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81485f0 con 0x7f83c8155450 2020-07-18T03:26:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83c80ad100 0x7f83c80b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83c80ad100 0x7f83c80b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60082/0 (socket says 172.21.15.47:60082) 2020-07-18T03:26:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 -- 172.21.15.47:0/646013422 learned_addr learned my addr 172.21.15.47:0/646013422 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c7fff700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83c8155450 0x7f83c81524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 -- 172.21.15.47:0/646013422 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83c8155450 msgr2=0x7f83c81524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83c8155450 0x7f83c81524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 -- 172.21.15.47:0/646013422 >> v1:172.21.15.18:6789/0 conn(0x7f83c8147a00 legacy=0x7f83c813a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887378546 0 0) 0x7f83c8155dd0 con 0x7f83c8147a00 2020-07-18T03:26:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 -- 172.21.15.47:0/646013422 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c81558c0 con 0x7f83c80ad100 2020-07-18T03:26:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83c77fe700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c80ad100 0x7f83c80b0a60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f83c805ca30 tx=0x7f83b8007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b8007240 con 0x7f83c80ad100 2020-07-18T03:26:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b80073a0 con 0x7f83c80ad100 2020-07-18T03:26:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8155b10 con 0x7f83c80ad100 2020-07-18T03:26:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b80066e0 con 0x7f83c80ad100 2020-07-18T03:26:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.256+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c8152b70 con 0x7f83c80ad100 2020-07-18T03:26:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.257+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c8148920 con 0x7f83c80ad100 2020-07-18T03:26:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.257+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f83b8010030 con 0x7f83c80ad100 2020-07-18T03:26:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.258+0000 7f83a77fe700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83a004ffb0 0x7f83a0052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.258+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f83b8067820 con 0x7f83c80ad100 2020-07-18T03:26:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.258+0000 7f83c6ffd700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83a004ffb0 0x7f83a0052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.258+0000 7f83c6ffd700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83a004ffb0 0x7f83a0052470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f83bc000ee0 tx=0x7f83bc007830).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.261+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f83b803ffb0 con 0x7f83c80ad100 2020-07-18T03:26:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.422+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83c8057450 con 0x7f83c80ad100 2020-07-18T03:26:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.423+0000 7f83a77fe700 1 -- 172.21.15.47:0/646013422 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f83b80435d0 con 0x7f83c80ad100 2020-07-18T03:26:42.425 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:42.426 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:26:41.771920+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:26:38.665431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-07-18T03:26:39.748890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83a004ffb0 msgr2=0x7f83a0052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83a004ffb0 0x7f83a0052470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f83bc000ee0 tx=0x7f83bc007830).stop 2020-07-18T03:26:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c80ad100 msgr2=0x7f83c80b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c80ad100 0x7f83c80b0a60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f83c805ca30 tx=0x7f83b8007b60).stop 2020-07-18T03:26:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 shutdown_connections 2020-07-18T03:26:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f83a004ffb0 0x7f83a0052470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f83c8155450 0x7f83c81524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 --2- 172.21.15.47:0/646013422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83c80ad100 0x7f83c80b0a60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.424+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 >> 172.21.15.47:0/646013422 conn(0x7f83c80b9340 msgr2=0x7f83c81469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.425+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 shutdown_connections 2020-07-18T03:26:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.425+0000 7f83cdfb1700 1 -- 172.21.15.47:0/646013422 wait complete. 2020-07-18T03:26:42.439 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:26:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.559+0000 7f4de254a700 1 Processor -- start 2020-07-18T03:26:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.559+0000 7f4de254a700 1 -- start start 2020-07-18T03:26:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4ddc080510 0x7f4ddc0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ddc153ad0 0x7f4ddc1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc080bb0 con 0x7f4ddc153f80 2020-07-18T03:26:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc157140 con 0x7f4ddc080510 2020-07-18T03:26:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc0808f0 con 0x7f4ddc153ad0 2020-07-18T03:26:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ddc153ad0 0x7f4ddc1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de12e8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4ddc153f80 0x7f4ddc063450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41178/0 (socket says 172.21.15.47:41178) 2020-07-18T03:26:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de12e8700 1 -- 172.21.15.47:0/2474484894 learned_addr learned my addr 172.21.15.47:0/2474484894 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de0ae7700 1 --2- 172.21.15.47:0/2474484894 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc080510 0x7f4ddc0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddbfff700 1 -- 172.21.15.47:0/2474484894 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc080510 msgr2=0x7f4ddc0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddbfff700 1 --2- 172.21.15.47:0/2474484894 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc080510 0x7f4ddc0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddbfff700 1 -- 172.21.15.47:0/2474484894 >> v1:172.21.15.47:6789/0 conn(0x7f4ddc153f80 legacy=0x7f4ddc063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddbfff700 1 -- 172.21.15.47:0/2474484894 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc080e50 con 0x7f4ddc153ad0 2020-07-18T03:26:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddb7fe700 1 -- 172.21.15.47:0/2474484894 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304788793 0 0) 0x7f4ddc080bb0 con 0x7f4ddc153f80 2020-07-18T03:26:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddbfff700 1 --2- 172.21.15.47:0/2474484894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc153ad0 0x7f4ddc1575e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f4dd0002f30 tx=0x7f4dd0006dd0).ready entity=mon.1 client_cookie=ec3b444c9d18a810 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddb7fe700 1 -- 172.21.15.47:0/2474484894 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dd00097f0 con 0x7f4ddc153ad0 2020-07-18T03:26:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddb7fe700 1 -- 172.21.15.47:0/2474484894 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0009950 con 0x7f4ddc153ad0 2020-07-18T03:26:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4ddb7fe700 1 -- 172.21.15.47:0/2474484894 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dd0009cb0 con 0x7f4ddc153ad0 2020-07-18T03:26:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 -- 172.21.15.47:0/2474484894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc153ad0 msgr2=0x7f4ddc1575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 --2- 172.21.15.47:0/2474484894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc153ad0 0x7f4ddc1575e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f4dd0002f30 tx=0x7f4dd0006dd0).stop 2020-07-18T03:26:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.560+0000 7f4de254a700 1 -- 172.21.15.47:0/2474484894 shutdown_connections 2020-07-18T03:26:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.561+0000 7f4de254a700 1 --2- 172.21.15.47:0/2474484894 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc080510 0x7f4ddc0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.561+0000 7f4de254a700 1 --2- 172.21.15.47:0/2474484894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc153ad0 0x7f4ddc1575e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.561+0000 7f4de254a700 1 -- 172.21.15.47:0/2474484894 >> 172.21.15.47:0/2474484894 conn(0x7f4ddc05c620 msgr2=0x7f4ddc0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.561+0000 7f4de254a700 1 -- 172.21.15.47:0/2474484894 shutdown_connections 2020-07-18T03:26:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.561+0000 7f4de254a700 1 -- 172.21.15.47:0/2474484894 wait complete. 2020-07-18T03:26:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.562+0000 7f4de254a700 1 Processor -- start 2020-07-18T03:26:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.562+0000 7f4de254a700 1 -- start start 2020-07-18T03:26:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de254a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4ddc080510 0x7f4ddc13ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de254a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4ddc153f80 0x7f4ddc1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de254a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ddc159880 0x7f4ddc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de254a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b4e50 con 0x7f4ddc153f80 2020-07-18T03:26:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de254a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b50f0 con 0x7f4ddc159880 2020-07-18T03:26:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de254a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b87f0 con 0x7f4ddc080510 2020-07-18T03:26:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ddc159880 0x7f4ddc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ddc159880 0x7f4ddc0b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60098/0 (socket says 172.21.15.47:60098) 2020-07-18T03:26:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 -- 172.21.15.47:0/710380457 learned_addr learned my addr 172.21.15.47:0/710380457 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de0ae7700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4ddc080510 0x7f4ddc13ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4ddbfff700 1 --2- 172.21.15.47:0/710380457 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc153f80 0x7f4ddc1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 -- 172.21.15.47:0/710380457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4ddc080510 msgr2=0x7f4ddc13ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4ddc080510 0x7f4ddc13ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 -- 172.21.15.47:0/710380457 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc153f80 msgr2=0x7f4ddc1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 --2- 172.21.15.47:0/710380457 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc153f80 0x7f4ddc1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.563+0000 7f4de12e8700 1 -- 172.21.15.47:0/710380457 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0b4ad0 con 0x7f4ddc159880 2020-07-18T03:26:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.564+0000 7f4de12e8700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc159880 0x7f4ddc0b8300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f4dcc000e90 tx=0x7f4dcc007960).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.564+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dcc0073f0 con 0x7f4ddc159880 2020-07-18T03:26:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.564+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc005030 con 0x7f4ddc159880 2020-07-18T03:26:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.564+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc0b17b0 con 0x7f4ddc159880 2020-07-18T03:26:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.564+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dcc005390 con 0x7f4ddc159880 2020-07-18T03:26:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.564+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc0b1c50 con 0x7f4ddc159880 2020-07-18T03:26:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.565+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc0690d0 con 0x7f4ddc159880 2020-07-18T03:26:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.565+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4dcc01b070 con 0x7f4ddc159880 2020-07-18T03:26:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.565+0000 7f4dbbfff700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4db4050040 0x7f4db4052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.565+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4dcc0671e0 con 0x7f4ddc159880 2020-07-18T03:26:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.565+0000 7f4de0ae7700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4db4050040 0x7f4db4052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.566+0000 7f4de0ae7700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4db4050040 0x7f4db4052500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f4dd400c000 tx=0x7f4dd4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.569+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4dcc03f970 con 0x7f4ddc159880 2020-07-18T03:26:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.782+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4ddc069f30 con 0x7f4ddc159880 2020-07-18T03:26:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.786+0000 7f4dbbfff700 1 -- 172.21.15.47:0/710380457 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4dcc042f90 con 0x7f4ddc159880 2020-07-18T03:26:42.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4db4050040 msgr2=0x7f4db4052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4db4050040 0x7f4db4052500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f4dd400c000 tx=0x7f4dd4008040).stop 2020-07-18T03:26:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc159880 msgr2=0x7f4ddc0b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc159880 0x7f4ddc0b8300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f4dcc000e90 tx=0x7f4dcc007960).stop 2020-07-18T03:26:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 shutdown_connections 2020-07-18T03:26:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4db4050040 0x7f4db4052500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ddc159880 0x7f4ddc0b8300 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 --2- 172.21.15.47:0/710380457 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4ddc080510 0x7f4ddc13ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 --2- 172.21.15.47:0/710380457 >> v2:172.21.15.18:3300/0 conn(0x7f4ddc153f80 0x7f4ddc1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 >> 172.21.15.47:0/710380457 conn(0x7f4ddc05c620 msgr2=0x7f4ddc0b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 shutdown_connections 2020-07-18T03:26:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.788+0000 7f4de254a700 1 -- 172.21.15.47:0/710380457 wait complete. 2020-07-18T03:26:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2020-07-18T03:26:42.801 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:26:42.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 Processor -- start 2020-07-18T03:26:42.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 -- start start 2020-07-18T03:26:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 0x7f9f440b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 0x7f9f44063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f44080bb0 con 0x7f9f44153ad0 2020-07-18T03:26:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f44157140 con 0x7f9f44080510 2020-07-18T03:26:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.923+0000 7f9f48c4b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440808f0 con 0x7f9f44157cf0 2020-07-18T03:26:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f42d9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 0x7f9f440b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 0x7f9f44063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f42d9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 0x7f9f440b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:45782/0 (socket says 172.21.15.47:45782) 2020-07-18T03:26:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 0x7f9f44063450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:52704/0 (socket says 172.21.15.47:52704) 2020-07-18T03:26:42.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f42d9d700 1 -- 172.21.15.47:0/1202916364 learned_addr learned my addr 172.21.15.47:0/1202916364 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4259c700 1 --1- 172.21.15.47:0/1202916364 >> v1:172.21.15.18:6789/0 conn(0x7f9f44153ad0 0x7f9f441575e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:59956/0 (socket says 172.21.15.47:59956) 2020-07-18T03:26:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 -- 172.21.15.47:0/1202916364 >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 msgr2=0x7f9f440b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 --2- 172.21.15.47:0/1202916364 >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 0x7f9f440b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 -- 172.21.15.47:0/1202916364 >> v1:172.21.15.18:6789/0 conn(0x7f9f44153ad0 legacy=0x7f9f441575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 -- 172.21.15.47:0/1202916364 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44080e50 con 0x7f9f44157cf0 2020-07-18T03:26:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f42d9d700 1 --2- 172.21.15.47:0/1202916364 >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 0x7f9f440b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f4359e700 1 --2- 172.21.15.47:0/1202916364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 0x7f9f44063450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f9f3c0032b0 tx=0x7f9f3c0091d0).ready entity=mon.2 client_cookie=34b98c8f3090c95d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f41d9b700 1 -- 172.21.15.47:0/1202916364 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3c00b9a0 con 0x7f9f44157cf0 2020-07-18T03:26:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.924+0000 7f9f41d9b700 1 -- 172.21.15.47:0/1202916364 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c00bb00 con 0x7f9f44157cf0 2020-07-18T03:26:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f41d9b700 1 -- 172.21.15.47:0/1202916364 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3c00bc70 con 0x7f9f44157cf0 2020-07-18T03:26:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 -- 172.21.15.47:0/1202916364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 msgr2=0x7f9f44063450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/1202916364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 0x7f9f44063450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f9f3c0032b0 tx=0x7f9f3c0091d0).stop 2020-07-18T03:26:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 -- 172.21.15.47:0/1202916364 shutdown_connections 2020-07-18T03:26:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/1202916364 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9f44157cf0 0x7f9f44063450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/1202916364 >> v2:172.21.15.18:3300/0 conn(0x7f9f44080510 0x7f9f440b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 -- 172.21.15.47:0/1202916364 >> 172.21.15.47:0/1202916364 conn(0x7f9f4405c620 msgr2=0x7f9f440b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 -- 172.21.15.47:0/1202916364 shutdown_connections 2020-07-18T03:26:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.925+0000 7f9f48c4b700 1 -- 172.21.15.47:0/1202916364 wait complete. 2020-07-18T03:26:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.926+0000 7f9f48c4b700 1 Processor -- start 2020-07-18T03:26:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.926+0000 7f9f48c4b700 1 -- start start 2020-07-18T03:26:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f48c4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f44080510 0x7f9f4413a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f48c4b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9f44153ad0 0x7f9f44152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f48c4b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f44124ba0 con 0x7f9f4415b690 2020-07-18T03:26:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f48c4b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440af840 con 0x7f9f44153ad0 2020-07-18T03:26:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f48c4b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f44124e40 con 0x7f9f44080510 2020-07-18T03:26:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f42d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f44080510 0x7f9f4413a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f4359e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f4415b690 0x7f9f44158160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41198/0 (socket says 172.21.15.47:41198) 2020-07-18T03:26:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f4359e700 1 -- 172.21.15.47:0/2706400494 learned_addr learned my addr 172.21.15.47:0/2706400494 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:42.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f4259c700 1 --2- 172.21.15.47:0/2706400494 >> v2:172.21.15.18:3300/0 conn(0x7f9f44153ad0 0x7f9f44152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f42d9d700 1 -- 172.21.15.47:0/2706400494 >> v2:172.21.15.18:3300/0 conn(0x7f9f44153ad0 msgr2=0x7f9f44152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.927+0000 7f9f42d9d700 1 --2- 172.21.15.47:0/2706400494 >> v2:172.21.15.18:3300/0 conn(0x7f9f44153ad0 0x7f9f44152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:42.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f42d9d700 1 -- 172.21.15.47:0/2706400494 >> v1:172.21.15.47:6789/0 conn(0x7f9f4415b690 legacy=0x7f9f44158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:42.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f42d9d700 1 -- 172.21.15.47:0/2706400494 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f441250e0 con 0x7f9f44080510 2020-07-18T03:26:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f42d9d700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f44080510 0x7f9f4413a510 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f9f3c0033f0 tx=0x7f9f34006e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3400c870 con 0x7f9f44080510 2020-07-18T03:26:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3400c9d0 con 0x7f9f44080510 2020-07-18T03:26:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f440ad690 con 0x7f9f44080510 2020-07-18T03:26:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3400cd30 con 0x7f9f44080510 2020-07-18T03:26:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.928+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f44124820 con 0x7f9f44080510 2020-07-18T03:26:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.929+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44125270 con 0x7f9f44080510 2020-07-18T03:26:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.929+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9f340123f0 con 0x7f9f44080510 2020-07-18T03:26:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.929+0000 7f9f2a7fc700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9f20050030 0x7f9f200524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.929+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9f34067d90 con 0x7f9f44080510 2020-07-18T03:26:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.929+0000 7f9f4259c700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9f20050030 0x7f9f200524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.930+0000 7f9f4259c700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9f20050030 0x7f9f200524f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f9f3800d000 tx=0x7f9f38007ad0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:42.933+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9f340404c0 con 0x7f9f44080510 2020-07-18T03:26:43.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.148+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9f44153890 con 0x7f9f44080510 2020-07-18T03:26:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.152+0000 7f9f2a7fc700 1 -- 172.21.15.47:0/2706400494 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9f34043ae0 con 0x7f9f44080510 2020-07-18T03:26:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9f20050030 msgr2=0x7f9f200524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9f20050030 0x7f9f200524f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f9f3800d000 tx=0x7f9f38007ad0).stop 2020-07-18T03:26:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f44080510 msgr2=0x7f9f4413a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f44080510 0x7f9f4413a510 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f9f3c0033f0 tx=0x7f9f34006e90).stop 2020-07-18T03:26:43.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 shutdown_connections 2020-07-18T03:26:43.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9f20050030 0x7f9f200524f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/2706400494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f44080510 0x7f9f4413a510 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 --2- 172.21.15.47:0/2706400494 >> v2:172.21.15.18:3300/0 conn(0x7f9f44153ad0 0x7f9f44152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 >> 172.21.15.47:0/2706400494 conn(0x7f9f4405c620 msgr2=0x7f9f44063450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:43.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 shutdown_connections 2020-07-18T03:26:43.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.154+0000 7f9f48c4b700 1 -- 172.21.15.47:0/2706400494 wait complete. 2020-07-18T03:26:43.158 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2020-07-18T03:26:43.175 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:26:43.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.295+0000 7f8ab5ab1700 1 Processor -- start 2020-07-18T03:26:43.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.295+0000 7f8ab5ab1700 1 -- start start 2020-07-18T03:26:43.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab5ab1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8ab0080510 0x7f8ab00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab5ab1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ab0157cf0 0x7f8ab0063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab5ab1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab0080bb0 con 0x7f8ab0153ad0 2020-07-18T03:26:43.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab5ab1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab0157140 con 0x7f8ab0080510 2020-07-18T03:26:43.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab5ab1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab00808f0 con 0x7f8ab0157cf0 2020-07-18T03:26:43.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ab0157cf0 0x7f8ab0063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ab0157cf0 0x7f8ab0063450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60120/0 (socket says 172.21.15.47:60120) 2020-07-18T03:26:43.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 -- 172.21.15.47:0/1112223675 learned_addr learned my addr 172.21.15.47:0/1112223675 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:43.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8aaffff700 1 --2- 172.21.15.47:0/1112223675 >> v2:172.21.15.18:3300/0 conn(0x7f8ab0080510 0x7f8ab00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 -- 172.21.15.47:0/1112223675 >> v2:172.21.15.18:3300/0 conn(0x7f8ab0080510 msgr2=0x7f8ab00b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:43.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1112223675 >> v2:172.21.15.18:3300/0 conn(0x7f8ab0080510 0x7f8ab00b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 -- 172.21.15.47:0/1112223675 >> v1:172.21.15.18:6789/0 conn(0x7f8ab0153ad0 legacy=0x7f8ab01575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:43.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 -- 172.21.15.47:0/1112223675 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0080e50 con 0x7f8ab0157cf0 2020-07-18T03:26:43.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.296+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1112223675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab0157cf0 0x7f8ab0063450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f8ab00b90e0 tx=0x7f8aa8009270).ready entity=mon.1 client_cookie=7a1c39903d9f5ac5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:43.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8aaf7fe700 1 -- 172.21.15.47:0/1112223675 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa800bbc0 con 0x7f8ab0157cf0 2020-07-18T03:26:43.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8aaf7fe700 1 -- 172.21.15.47:0/1112223675 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa800bd20 con 0x7f8ab0157cf0 2020-07-18T03:26:43.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8aaf7fe700 1 -- 172.21.15.47:0/1112223675 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa800be90 con 0x7f8ab0157cf0 2020-07-18T03:26:43.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1112223675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab0157cf0 msgr2=0x7f8ab0063450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 --2- 172.21.15.47:0/1112223675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab0157cf0 0x7f8ab0063450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f8ab00b90e0 tx=0x7f8aa8009270).stop 2020-07-18T03:26:43.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1112223675 shutdown_connections 2020-07-18T03:26:43.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 --2- 172.21.15.47:0/1112223675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab0157cf0 0x7f8ab0063450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 --2- 172.21.15.47:0/1112223675 >> v2:172.21.15.18:3300/0 conn(0x7f8ab0080510 0x7f8ab00b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1112223675 >> 172.21.15.47:0/1112223675 conn(0x7f8ab005c620 msgr2=0x7f8ab00b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1112223675 shutdown_connections 2020-07-18T03:26:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.297+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1112223675 wait complete. 2020-07-18T03:26:43.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.298+0000 7f8ab5ab1700 1 Processor -- start 2020-07-18T03:26:43.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.298+0000 7f8ab5ab1700 1 -- start start 2020-07-18T03:26:43.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8ab5ab1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 0x7f8ab00ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8ab5ab1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab00ae740 con 0x7f8ab0153ad0 2020-07-18T03:26:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8ab5ab1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab00ad5d0 con 0x7f8ab0080510 2020-07-18T03:26:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8ab5ab1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00ad870 con 0x7f8ab015a2f0 2020-07-18T03:26:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8aaffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ab0080510 0x7f8ab0155da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41210/0 (socket says 172.21.15.47:41210) 2020-07-18T03:26:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8aaffff700 1 -- 172.21.15.47:0/1027462995 learned_addr learned my addr 172.21.15.47:0/1027462995 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 0x7f8ab00ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535400305 0 0) 0x7f8ab00ad5d0 con 0x7f8ab0080510 2020-07-18T03:26:43.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.299+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a8c003240 con 0x7f8ab0080510 2020-07-18T03:26:43.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 4074661097 0 0) 0x7f8a8c003240 con 0x7f8ab0080510 2020-07-18T03:26:43.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ab00ad5d0 con 0x7f8ab0080510 2020-07-18T03:26:43.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8aa0006e60 con 0x7f8ab0080510 2020-07-18T03:26:43.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 msgr2=0x7f8ab00ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 0x7f8ab00ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 >> v1:172.21.15.47:6789/0 conn(0x7f8ab0080510 legacy=0x7f8ab0155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 >> v1:172.21.15.18:6789/0 conn(0x7f8ab0153ad0 legacy=0x7f8ab0158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 0x7f8a8c003800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a8c003cf0 0x7f8a8c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a8c0049d0 0x7f8a8c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00ad870 con 0x7f8a8c003cf0 2020-07-18T03:26:43.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0041f0 con 0x7f8a8c0049d0 2020-07-18T03:26:43.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c00b730 con 0x7f8ab015a2f0 2020-07-18T03:26:43.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a8c0049d0 0x7f8a8c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aa7fff700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a8c003cf0 0x7f8a8c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8aaffff700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 0x7f8a8c003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 -- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 msgr2=0x7f8a8c003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ab015a2f0 0x7f8a8c003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 -- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a8c003cf0 msgr2=0x7f8a8c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a8c003cf0 0x7f8a8c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab00adb10 con 0x7f8a8c0049d0 2020-07-18T03:26:43.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab00addb0 con 0x7f8a8c0049d0 2020-07-18T03:26:43.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab015ad00 con 0x7f8a8c0049d0 2020-07-18T03:26:43.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.300+0000 7f8ab484f700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a8c0049d0 0x7f8a8c004e50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f8aa801b000 tx=0x7f8aa8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:43.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.301+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa8010870 con 0x7f8a8c0049d0 2020-07-18T03:26:43.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.301+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa80109d0 con 0x7f8a8c0049d0 2020-07-18T03:26:43.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.301+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa8010d30 con 0x7f8a8c0049d0 2020-07-18T03:26:43.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.301+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab00643a0 con 0x7f8a8c0049d0 2020-07-18T03:26:43.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.302+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8aa800f070 con 0x7f8a8c0049d0 2020-07-18T03:26:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.302+0000 7f8aa77fe700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8a8c058260 0x7f8a8c05a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.302+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8aa80692c0 con 0x7f8a8c0049d0 2020-07-18T03:26:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.302+0000 7f8aaffff700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8a8c058260 0x7f8a8c05a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.302+0000 7f8aaffff700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8a8c058260 0x7f8a8c05a720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f8aa000f000 tx=0x7f8aa0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.305+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8aa8045480 con 0x7f8a8c0049d0 2020-07-18T03:26:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.522+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8ab0153890 con 0x7f8a8c0049d0 2020-07-18T03:26:43.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.526+0000 7f8aa77fe700 1 -- 172.21.15.47:0/1027462995 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8aa80150e0 con 0x7f8a8c0049d0 2020-07-18T03:26:43.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8a8c058260 msgr2=0x7f8a8c05a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8a8c058260 0x7f8a8c05a720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f8aa000f000 tx=0x7f8aa0008040).stop 2020-07-18T03:26:43.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a8c0049d0 msgr2=0x7f8a8c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 --2- 172.21.15.47:0/1027462995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a8c0049d0 0x7f8a8c004e50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f8aa801b000 tx=0x7f8aa8016000).stop 2020-07-18T03:26:43.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab484f700 1 -- 172.21.15.47:0/1027462995 reap_dead start 2020-07-18T03:26:43.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab484f700 1 -- 172.21.15.47:0/1027462995 reap_dead start 2020-07-18T03:26:43.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 shutdown_connections 2020-07-18T03:26:43.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 >> 172.21.15.47:0/1027462995 conn(0x7f8ab005c620 msgr2=0x7f8ab00b7720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:43.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 shutdown_connections 2020-07-18T03:26:43.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.527+0000 7f8ab5ab1700 1 -- 172.21.15.47:0/1027462995 wait complete. 2020-07-18T03:26:43.532 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.2 2020-07-18T03:26:43.540 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:26:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.660+0000 7f7c7091d700 1 Processor -- start 2020-07-18T03:26:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.660+0000 7f7c7091d700 1 -- start start 2020-07-18T03:26:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.660+0000 7f7c7091d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7c6c153ad0 0x7f7c6c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.660+0000 7f7c7091d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c080bb0 con 0x7f7c6c153f80 2020-07-18T03:26:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.660+0000 7f7c7091d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c157140 con 0x7f7c6c080510 2020-07-18T03:26:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.660+0000 7f7c7091d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0808f0 con 0x7f7c6c153ad0 2020-07-18T03:26:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c6ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7c6c080510 0x7f7c6c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41224/0 (socket says 172.21.15.47:41224) 2020-07-18T03:26:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/2647966395 learned_addr learned my addr 172.21.15.47:0/2647966395 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646162848 0 0) 0x7f7c6c157140 con 0x7f7c6c080510 2020-07-18T03:26:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/2647966395 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c153ad0 0x7f7c6c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c54003240 con 0x7f7c6c080510 2020-07-18T03:26:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4060220825 0 0) 0x7f7c54003240 con 0x7f7c6c080510 2020-07-18T03:26:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c153ad0 msgr2=0x7f7c6c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 --2- 172.21.15.47:0/2647966395 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c153ad0 0x7f7c6c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 >> v1:172.21.15.18:6789/0 conn(0x7f7c6c153f80 legacy=0x7f7c6c063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c080e50 con 0x7f7c6c080510 2020-07-18T03:26:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7c5c0016d0 con 0x7f7c6c080510 2020-07-18T03:26:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 >> v1:172.21.15.47:6789/0 conn(0x7f7c6c080510 legacy=0x7f7c6c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c54003730 0x7f7c54003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7c540044e0 0x7f7c54004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c54004e50 0x7f7c5400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0808f0 con 0x7f7c54004e50 2020-07-18T03:26:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c54004320 con 0x7f7c54003730 2020-07-18T03:26:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5404fc80 con 0x7f7c540044e0 2020-07-18T03:26:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c6ad9d700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c54003730 0x7f7c54003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c6b59e700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c54004e50 0x7f7c5400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.661+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7c540044e0 0x7f7c54004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7c540044e0 msgr2=0x7f7c54004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:43.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6ad9d700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7c540044e0 0x7f7c54004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c54004e50 msgr2=0x7f7c5400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:43.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6ad9d700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c54004e50 0x7f7c5400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/2647966395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54003240 con 0x7f7c54003730 2020-07-18T03:26:43.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7c540044e0 0x7f7c54004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:43.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6b59e700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c54004e50 0x7f7c5400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6b59e700 1 -- 172.21.15.47:0/2647966395 reap_dead start 2020-07-18T03:26:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c6ad9d700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c54003730 0x7f7c54003b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f7c540054b0 tx=0x7f7c5c007750).ready entity=mon.1 client_cookie=53db8dea971761e6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c5c0070d0 con 0x7f7c54003730 2020-07-18T03:26:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c007230 con 0x7f7c54003730 2020-07-18T03:26:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c69d9b700 1 -- 172.21.15.47:0/2647966395 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c5c0055f0 con 0x7f7c54003730 2020-07-18T03:26:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 -- 172.21.15.47:0/2647966395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c54003730 msgr2=0x7f7c54003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c54003730 0x7f7c54003b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f7c540054b0 tx=0x7f7c5c007750).stop 2020-07-18T03:26:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 -- 172.21.15.47:0/2647966395 shutdown_connections 2020-07-18T03:26:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 --2- 172.21.15.47:0/2647966395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c54003730 0x7f7c54003b50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 -- 172.21.15.47:0/2647966395 >> 172.21.15.47:0/2647966395 conn(0x7f7c6c05c620 msgr2=0x7f7c6c0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 -- 172.21.15.47:0/2647966395 shutdown_connections 2020-07-18T03:26:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.662+0000 7f7c7091d700 1 -- 172.21.15.47:0/2647966395 wait complete. 2020-07-18T03:26:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 Processor -- start 2020-07-18T03:26:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 -- start start 2020-07-18T03:26:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c6c0b5800 0x7f7c6c0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7c6c13ed30 0x7f7c6c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c0b11a0 con 0x7f7c6c1310a0 2020-07-18T03:26:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c130c00 con 0x7f7c6c13ed30 2020-07-18T03:26:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.664+0000 7f7c7091d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c13f190 con 0x7f7c6c0b5800 2020-07-18T03:26:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c6c0b5800 0x7f7c6c0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c6c0b5800 0x7f7c6c0b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60148/0 (socket says 172.21.15.47:60148) 2020-07-18T03:26:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/1645742065 learned_addr learned my addr 172.21.15.47:0/1645742065 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/1645742065 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c13ed30 0x7f7c6c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/1645742065 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c13ed30 msgr2=0x7f7c6c0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 --2- 172.21.15.47:0/1645742065 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c13ed30 0x7f7c6c0b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/1645742065 >> v1:172.21.15.18:6789/0 conn(0x7f7c6c1310a0 legacy=0x7f7c6c1314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 -- 172.21.15.47:0/1645742065 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c132740 con 0x7f7c6c0b5800 2020-07-18T03:26:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/1645742065 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c13ed30 0x7f7c6c0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c6ad9d700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c0b5800 0x7f7c6c0b5be0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f7c540054b0 tx=0x7f7c5c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c5c013850 con 0x7f7c6c0b5800 2020-07-18T03:26:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c0139b0 con 0x7f7c6c0b5800 2020-07-18T03:26:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c0b0df0 con 0x7f7c6c0b5800 2020-07-18T03:26:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c5c013d10 con 0x7f7c6c0b5800 2020-07-18T03:26:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.665+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c0b2e80 con 0x7f7c6c0b5800 2020-07-18T03:26:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.666+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c0b13a0 con 0x7f7c6c0b5800 2020-07-18T03:26:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.666+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7c5c01d070 con 0x7f7c6c0b5800 2020-07-18T03:26:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.667+0000 7f7c527fc700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c4804ffe0 0x7f7c480524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.667+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7c5c068db0 con 0x7f7c6c0b5800 2020-07-18T03:26:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.667+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c4804ffe0 0x7f7c480524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.667+0000 7f7c6a59c700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c4804ffe0 0x7f7c480524a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f7c60008000 tx=0x7f7c60008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.670+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7c5c041540 con 0x7f7c6c0b5800 2020-07-18T03:26:43.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.885+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7c6c061d20 con 0x7f7c6c0b5800 2020-07-18T03:26:43.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.889+0000 7f7c527fc700 1 -- 172.21.15.47:0/1645742065 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7c5c01a030 con 0x7f7c6c0b5800 2020-07-18T03:26:43.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c4804ffe0 msgr2=0x7f7c480524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c4804ffe0 0x7f7c480524a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f7c60008000 tx=0x7f7c60008040).stop 2020-07-18T03:26:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c0b5800 msgr2=0x7f7c6c0b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c0b5800 0x7f7c6c0b5be0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f7c540054b0 tx=0x7f7c5c015000).stop 2020-07-18T03:26:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 shutdown_connections 2020-07-18T03:26:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c4804ffe0 0x7f7c480524a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 --2- 172.21.15.47:0/1645742065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c0b5800 0x7f7c6c0b5be0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 --2- 172.21.15.47:0/1645742065 >> v2:172.21.15.18:3300/0 conn(0x7f7c6c13ed30 0x7f7c6c0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 >> 172.21.15.47:0/1645742065 conn(0x7f7c6c05c620 msgr2=0x7f7c6c154400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 shutdown_connections 2020-07-18T03:26:43.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:43.890+0000 7f7c7091d700 1 -- 172.21.15.47:0/1645742065 wait complete. 2020-07-18T03:26:43.895 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.3 2020-07-18T03:26:43.903 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.016+0000 7ff4434e4700 1 Processor -- start 2020-07-18T03:26:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.016+0000 7ff4434e4700 1 -- start start 2020-07-18T03:26:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff4434e4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c0ad100 0x7ff43c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff4434e4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff43c1443f0 0x7ff43c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff4434e4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c147330 con 0x7ff43c147a00 2020-07-18T03:26:44.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff4434e4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c0ae530 con 0x7ff43c1443f0 2020-07-18T03:26:44.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff4434e4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b57e0 con 0x7ff43c0ad100 2020-07-18T03:26:44.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff441a81700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff43c1443f0 0x7ff43c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff440a7f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff43c147a00 0x7ff43c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41248/0 (socket says 172.21.15.47:41248) 2020-07-18T03:26:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff440a7f700 1 -- 172.21.15.47:0/352000813 learned_addr learned my addr 172.21.15.47:0/352000813 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff441280700 1 --2- 172.21.15.47:0/352000813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c0ad100 0x7ff43c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff441a81700 1 -- 172.21.15.47:0/352000813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c0ad100 msgr2=0x7ff43c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff441a81700 1 --2- 172.21.15.47:0/352000813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c0ad100 0x7ff43c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff441a81700 1 -- 172.21.15.47:0/352000813 >> v1:172.21.15.47:6789/0 conn(0x7ff43c147a00 legacy=0x7ff43c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff42bfff700 1 -- 172.21.15.47:0/352000813 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973419531 0 0) 0x7ff43c147330 con 0x7ff43c147a00 2020-07-18T03:26:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.017+0000 7ff441a81700 1 -- 172.21.15.47:0/352000813 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0b5a80 con 0x7ff43c1443f0 2020-07-18T03:26:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff441a81700 1 --2- 172.21.15.47:0/352000813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c1443f0 0x7ff43c147e40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7ff4340032b0 tx=0x7ff4340091d0).ready entity=mon.1 client_cookie=bab81ac8aeef65d2 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff42bfff700 1 -- 172.21.15.47:0/352000813 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff43400ba50 con 0x7ff43c1443f0 2020-07-18T03:26:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff42bfff700 1 -- 172.21.15.47:0/352000813 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43400bbb0 con 0x7ff43c1443f0 2020-07-18T03:26:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff42bfff700 1 -- 172.21.15.47:0/352000813 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff43400bd20 con 0x7ff43c1443f0 2020-07-18T03:26:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 -- 172.21.15.47:0/352000813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c1443f0 msgr2=0x7ff43c147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 --2- 172.21.15.47:0/352000813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c1443f0 0x7ff43c147e40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7ff4340032b0 tx=0x7ff4340091d0).stop 2020-07-18T03:26:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 -- 172.21.15.47:0/352000813 shutdown_connections 2020-07-18T03:26:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 --2- 172.21.15.47:0/352000813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c1443f0 0x7ff43c147e40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 --2- 172.21.15.47:0/352000813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c0ad100 0x7ff43c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 -- 172.21.15.47:0/352000813 >> 172.21.15.47:0/352000813 conn(0x7ff43c0b9340 msgr2=0x7ff43c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 -- 172.21.15.47:0/352000813 shutdown_connections 2020-07-18T03:26:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.018+0000 7ff4434e4700 1 -- 172.21.15.47:0/352000813 wait complete. 2020-07-18T03:26:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.019+0000 7ff4434e4700 1 Processor -- start 2020-07-18T03:26:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.019+0000 7ff4434e4700 1 -- start start 2020-07-18T03:26:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff4434e4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff43c147a00 0x7ff43c142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff4434e4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c13d3a0 0x7ff43c13d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff4434e4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c159720 con 0x7ff43c0ad100 2020-07-18T03:26:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff4434e4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c142610 con 0x7ff43c147a00 2020-07-18T03:26:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff4434e4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c1591e0 con 0x7ff43c13d3a0 2020-07-18T03:26:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff440a7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff43c147a00 0x7ff43c142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff441280700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff43c0ad100 0x7ff43c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41250/0 (socket says 172.21.15.47:41250) 2020-07-18T03:26:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.020+0000 7ff441280700 1 -- 172.21.15.47:0/2595380015 learned_addr learned my addr 172.21.15.47:0/2595380015 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff440a7f700 1 -- 172.21.15.47:0/2595380015 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c13d3a0 msgr2=0x7ff43c13d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff441a81700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c13d3a0 0x7ff43c13d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256585264 0 0) 0x7ff43c159720 con 0x7ff43c0ad100 2020-07-18T03:26:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff440a7f700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c13d3a0 0x7ff43c13d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff440a7f700 1 -- 172.21.15.47:0/2595380015 >> v1:172.21.15.47:6789/0 conn(0x7ff43c0ad100 legacy=0x7ff43c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff440a7f700 1 -- 172.21.15.47:0/2595380015 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c159480 con 0x7ff43c147a00 2020-07-18T03:26:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff440a7f700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c147a00 0x7ff43c142120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7ff438000b80 tx=0x7ff4380079c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4380072e0 con 0x7ff43c147a00 2020-07-18T03:26:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438007440 con 0x7ff43c147a00 2020-07-18T03:26:44.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c155380 con 0x7ff43c147a00 2020-07-18T03:26:44.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4380057e0 con 0x7ff43c147a00 2020-07-18T03:26:44.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.021+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c155130 con 0x7ff43c147a00 2020-07-18T03:26:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.022+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c155510 con 0x7ff43c147a00 2020-07-18T03:26:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.022+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff438010070 con 0x7ff43c147a00 2020-07-18T03:26:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.022+0000 7ff4297fa700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff41404fef0 0x7ff4140523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.022+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff438067270 con 0x7ff43c147a00 2020-07-18T03:26:44.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.022+0000 7ff441280700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff41404fef0 0x7ff4140523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.023+0000 7ff441280700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff41404fef0 0x7ff4140523b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff42c006aa0 tx=0x7ff42c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:44.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.026+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff43803fa00 con 0x7ff43c147a00 2020-07-18T03:26:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.237+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff43c057450 con 0x7ff43c147a00 2020-07-18T03:26:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.237+0000 7ff4297fa700 1 -- 172.21.15.47:0/2595380015 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7ff438043020 con 0x7ff43c147a00 2020-07-18T03:26:44.239 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:44.240 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:26:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.238+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff41404fef0 msgr2=0x7ff4140523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.238+0000 7ff4434e4700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff41404fef0 0x7ff4140523b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff42c006aa0 tx=0x7ff42c008040).stop 2020-07-18T03:26:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.238+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c147a00 msgr2=0x7ff43c142120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.238+0000 7ff4434e4700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c147a00 0x7ff43c142120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7ff438000b80 tx=0x7ff4380079c0).stop 2020-07-18T03:26:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 shutdown_connections 2020-07-18T03:26:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff43c13d3a0 0x7ff43c13d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff41404fef0 0x7ff4140523b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 --2- 172.21.15.47:0/2595380015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff43c147a00 0x7ff43c142120 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 >> 172.21.15.47:0/2595380015 conn(0x7ff43c0b9340 msgr2=0x7ff43c060100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 shutdown_connections 2020-07-18T03:26:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.239+0000 7ff4434e4700 1 -- 172.21.15.47:0/2595380015 wait complete. 2020-07-18T03:26:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:26:44.454 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:26:40.770533+0000', 'modified': '2020-07-18T03:26:41.771684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5101}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5101': {'gid': 5101, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/3109073551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3109073551}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3109073551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:44.454 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:44.455 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:44.597 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5101,"want_state":"up:active","state":"up:active","rank_uptime":3.8160828229999999,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":11.044031512} 2020-07-18T03:26:44.598 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5101, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 3.816082823, "state": "up:active", "uptime": 11.044031512, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:44.598 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:44.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.712+0000 7f6feda21700 1 Processor -- start 2020-07-18T03:26:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 -- start start 2020-07-18T03:26:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fe80b2e10 0x7f6fe80b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6fe80b1560 0x7f6fe80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b2130 0x7f6fe80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe8144e30 con 0x7f6fe80b1560 2020-07-18T03:26:44.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe80afbd0 con 0x7f6fe80b2e10 2020-07-18T03:26:44.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6feda21700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe81491c0 con 0x7f6fe80b2130 2020-07-18T03:26:44.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6fe6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fe80b2e10 0x7f6fe80b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6fe6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fe80b2e10 0x7f6fe80b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60170/0 (socket says 172.21.15.47:60170) 2020-07-18T03:26:44.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.713+0000 7f6fe6ffd700 1 -- 172.21.15.47:0/1097264468 learned_addr learned my addr 172.21.15.47:0/1097264468 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:44.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe67fc700 1 --2- 172.21.15.47:0/1097264468 >> v2:172.21.15.18:3300/0 conn(0x7f6fe80b1560 0x7f6fe80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe77fe700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b2130 0x7f6fe80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe6ffd700 1 -- 172.21.15.47:0/1097264468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b2130 msgr2=0x7f6fe80b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:44.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe6ffd700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b2130 0x7f6fe80b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe6ffd700 1 -- 172.21.15.47:0/1097264468 >> v2:172.21.15.18:3300/0 conn(0x7f6fe80b1560 msgr2=0x7f6fe80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:44.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe6ffd700 1 --2- 172.21.15.47:0/1097264468 >> v2:172.21.15.18:3300/0 conn(0x7f6fe80b1560 0x7f6fe80b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe6ffd700 1 -- 172.21.15.47:0/1097264468 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8145050 con 0x7f6fe80b2e10 2020-07-18T03:26:44.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe77fe700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b2130 0x7f6fe80b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:44.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe6ffd700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b2e10 0x7f6fe80b31f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f6fe80ba340 tx=0x7f6fd0006e90).ready entity=mon.1 client_cookie=6b0d35b6344f5ea9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:44.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe5ffb700 1 -- 172.21.15.47:0/1097264468 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fd00098e0 con 0x7f6fe80b2e10 2020-07-18T03:26:44.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe5ffb700 1 -- 172.21.15.47:0/1097264468 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0009a40 con 0x7f6fe80b2e10 2020-07-18T03:26:44.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6fe5ffb700 1 -- 172.21.15.47:0/1097264468 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fd0009da0 con 0x7f6fe80b2e10 2020-07-18T03:26:44.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 -- 172.21.15.47:0/1097264468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b2e10 msgr2=0x7f6fe80b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b2e10 0x7f6fe80b31f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f6fe80ba340 tx=0x7f6fd0006e90).stop 2020-07-18T03:26:44.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 -- 172.21.15.47:0/1097264468 shutdown_connections 2020-07-18T03:26:44.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b2130 0x7f6fe80b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 --2- 172.21.15.47:0/1097264468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b2e10 0x7f6fe80b31f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 --2- 172.21.15.47:0/1097264468 >> v2:172.21.15.18:3300/0 conn(0x7f6fe80b1560 0x7f6fe80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 -- 172.21.15.47:0/1097264468 >> 172.21.15.47:0/1097264468 conn(0x7f6fe80b9340 msgr2=0x7f6fe80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:44.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 -- 172.21.15.47:0/1097264468 shutdown_connections 2020-07-18T03:26:44.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.714+0000 7f6feda21700 1 -- 172.21.15.47:0/1097264468 wait complete. 2020-07-18T03:26:44.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.716+0000 7f6feda21700 1 Processor -- start 2020-07-18T03:26:44.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.716+0000 7f6feda21700 1 -- start start 2020-07-18T03:26:44.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.716+0000 7f6feda21700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b1560 0x7f6fe8159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6feda21700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fe80b82a0 0x7f6fe80b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6feda21700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe81468d0 con 0x7f6fe80b2130 2020-07-18T03:26:44.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6feda21700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe81485b0 con 0x7f6fe80b82a0 2020-07-18T03:26:44.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6feda21700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8159960 con 0x7f6fe80b1560 2020-07-18T03:26:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fe80b82a0 0x7f6fe80b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fe80b82a0 0x7f6fe80b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60180/0 (socket says 172.21.15.47:60180) 2020-07-18T03:26:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 -- 172.21.15.47:0/259193560 learned_addr learned my addr 172.21.15.47:0/259193560 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe6ffd700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b1560 0x7f6fe8159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 -- 172.21.15.47:0/259193560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b1560 msgr2=0x7f6fe8159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b1560 0x7f6fe8159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 -- 172.21.15.47:0/259193560 >> v1:172.21.15.18:6789/0 conn(0x7f6fe80b2130 legacy=0x7f6fe8158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 -- 172.21.15.47:0/259193560 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8146b40 con 0x7f6fe80b82a0 2020-07-18T03:26:44.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fe77fe700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b82a0 0x7f6fe80b8700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6fe0007000 tx=0x7f6fe0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:44.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe0015070 con 0x7f6fe80b82a0 2020-07-18T03:26:44.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0007820 con 0x7f6fe80b82a0 2020-07-18T03:26:44.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe8146de0 con 0x7f6fe80b82a0 2020-07-18T03:26:44.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe0007b80 con 0x7f6fe80b82a0 2020-07-18T03:26:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.717+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe813f820 con 0x7f6fe80b82a0 2020-07-18T03:26:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.718+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe8057450 con 0x7f6fe80b82a0 2020-07-18T03:26:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.719+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6fe001a410 con 0x7f6fe80b82a0 2020-07-18T03:26:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.719+0000 7f6fdeffd700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6fc404ff80 0x7f6fc4052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.719+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6fe0067f20 con 0x7f6fe80b82a0 2020-07-18T03:26:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.719+0000 7f6fe6ffd700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6fc404ff80 0x7f6fc4052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.719+0000 7f6fe6ffd700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6fc404ff80 0x7f6fc4052440 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6fd001c000 tx=0x7f6fd0015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.722+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6fe0040650 con 0x7f6fe80b82a0 2020-07-18T03:26:44.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.879+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fe8066b60 con 0x7f6fe80b82a0 2020-07-18T03:26:44.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.880+0000 7f6fdeffd700 1 -- 172.21.15.47:0/259193560 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f6fe0043c70 con 0x7f6fe80b82a0 2020-07-18T03:26:44.882 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:44.883 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:26:41.771920+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:26:38.665431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-07-18T03:26:39.748890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.881+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6fc404ff80 msgr2=0x7f6fc4052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.881+0000 7f6feda21700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6fc404ff80 0x7f6fc4052440 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6fd001c000 tx=0x7f6fd0015000).stop 2020-07-18T03:26:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.881+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b82a0 msgr2=0x7f6fe80b8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.881+0000 7f6feda21700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b82a0 0x7f6fe80b8700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6fe0007000 tx=0x7f6fe0008040).stop 2020-07-18T03:26:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 shutdown_connections 2020-07-18T03:26:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6fc404ff80 0x7f6fc4052440 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe80b82a0 0x7f6fe80b8700 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 --2- 172.21.15.47:0/259193560 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fe80b1560 0x7f6fe8159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 >> 172.21.15.47:0/259193560 conn(0x7f6fe80b9340 msgr2=0x7f6fe81442c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:44.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 shutdown_connections 2020-07-18T03:26:44.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:44.882+0000 7f6feda21700 1 -- 172.21.15.47:0/259193560 wait complete. 2020-07-18T03:26:44.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:26:38.665431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:26:39.748890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:44.907 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.023+0000 7fa560d9f700 1 Processor -- start 2020-07-18T03:26:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa560d9f700 1 -- start start 2020-07-18T03:26:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa560d9f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa55c0619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa560d9f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa55c13efc0 0x7fa55c05f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa560d9f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c149120 con 0x7fa55c145690 2020-07-18T03:26:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa560d9f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c144d60 con 0x7fa55c13efc0 2020-07-18T03:26:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa560d9f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c144f50 con 0x7fa55c148d40 2020-07-18T03:26:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa55affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa55c145690 0x7fa55c13e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41272/0 (socket says 172.21.15.47:41272) 2020-07-18T03:26:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.024+0000 7fa55affd700 1 -- 172.21.15.47:0/3173851914 learned_addr learned my addr 172.21.15.47:0/3173851914 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55b7fe700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa55c0619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55bfff700 1 --2- 172.21.15.47:0/3173851914 >> v2:172.21.15.18:3300/0 conn(0x7fa55c13efc0 0x7fa55c05f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386100180 0 0) 0x7fa55c149120 con 0x7fa55c145690 2020-07-18T03:26:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa548003240 con 0x7fa55c145690 2020-07-18T03:26:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2417171587 0 0) 0x7fa548003240 con 0x7fa55c145690 2020-07-18T03:26:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 msgr2=0x7fa55c0619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa55c0619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 >> v2:172.21.15.18:3300/0 conn(0x7fa55c13efc0 msgr2=0x7fa55c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 --2- 172.21.15.47:0/3173851914 >> v2:172.21.15.18:3300/0 conn(0x7fa55c13efc0 0x7fa55c05f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c1451f0 con 0x7fa55c145690 2020-07-18T03:26:45.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fa5500016d0 con 0x7fa55c145690 2020-07-18T03:26:45.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 >> v1:172.21.15.47:6789/0 conn(0x7fa55c145690 legacy=0x7fa55c13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa548003730 0x7fa548003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5480044e0 0x7fa548004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa548004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c144f50 con 0x7fa548003730 2020-07-18T03:26:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa548005390 con 0x7fa5480044e0 2020-07-18T03:26:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480042f0 con 0x7fa55c148d40 2020-07-18T03:26:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55affd700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5480044e0 0x7fa548004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55b7fe700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa548003730 0x7fa548003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.025+0000 7fa55bfff700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa548004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55affd700 1 -- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 msgr2=0x7fa548004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:45.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55affd700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa548004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55affd700 1 -- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa548003730 msgr2=0x7fa548003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:45.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55affd700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa548003730 0x7fa548003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55affd700 1 -- 172.21.15.47:0/3173851914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548003240 con 0x7fa5480044e0 2020-07-18T03:26:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55bfff700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa55c148d40 0x7fa548004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55affd700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5480044e0 0x7fa548004960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fa5500013e0 tx=0x7fa55c0ae240).ready entity=mon.1 client_cookie=6424801fa01ed692 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5500070d0 con 0x7fa5480044e0 2020-07-18T03:26:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550007230 con 0x7fa5480044e0 2020-07-18T03:26:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55a7fc700 1 -- 172.21.15.47:0/3173851914 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5500055f0 con 0x7fa5480044e0 2020-07-18T03:26:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa560d9f700 1 -- 172.21.15.47:0/3173851914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5480044e0 msgr2=0x7fa548004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa560d9f700 1 --2- 172.21.15.47:0/3173851914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5480044e0 0x7fa548004960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fa5500013e0 tx=0x7fa55c0ae240).stop 2020-07-18T03:26:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa55bfff700 1 -- 172.21.15.47:0/3173851914 reap_dead start 2020-07-18T03:26:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa560d9f700 1 -- 172.21.15.47:0/3173851914 shutdown_connections 2020-07-18T03:26:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa560d9f700 1 -- 172.21.15.47:0/3173851914 >> 172.21.15.47:0/3173851914 conn(0x7fa55c0b91d0 msgr2=0x7fa55c0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa560d9f700 1 -- 172.21.15.47:0/3173851914 shutdown_connections 2020-07-18T03:26:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.026+0000 7fa560d9f700 1 -- 172.21.15.47:0/3173851914 wait complete. 2020-07-18T03:26:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa560d9f700 1 Processor -- start 2020-07-18T03:26:45.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa560d9f700 1 -- start start 2020-07-18T03:26:45.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa560d9f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa55c143970 0x7fa55c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa560d9f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c063ee0 con 0x7fa55c0af5d0 2020-07-18T03:26:45.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa560d9f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c157fb0 con 0x7fa55c143d50 2020-07-18T03:26:45.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa560d9f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c158250 con 0x7fa55c143970 2020-07-18T03:26:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa55affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa55c143d50 0x7fa55c13ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41286/0 (socket says 172.21.15.47:41286) 2020-07-18T03:26:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa55affd700 1 -- 172.21.15.47:0/1964878366 learned_addr learned my addr 172.21.15.47:0/1964878366 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.028+0000 7fa55b7fe700 1 --2- 172.21.15.47:0/1964878366 >> v2:172.21.15.18:3300/0 conn(0x7fa55c143970 0x7fa55c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907442400 0 0) 0x7fa55c157fb0 con 0x7fa55c143d50 2020-07-18T03:26:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa534003240 con 0x7fa55c143d50 2020-07-18T03:26:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2528988091 0 0) 0x7fa534003240 con 0x7fa55c143d50 2020-07-18T03:26:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa55b7fe700 1 -- 172.21.15.47:0/1964878366 >> v1:172.21.15.47:6789/0 conn(0x7fa55c143d50 legacy=0x7fa55c13ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa55b7fe700 1 -- 172.21.15.47:0/1964878366 >> v1:172.21.15.18:6789/0 conn(0x7fa55c0af5d0 legacy=0x7fa55c154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa55b7fe700 1 -- 172.21.15.47:0/1964878366 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c1584f0 con 0x7fa55c143970 2020-07-18T03:26:45.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa55b7fe700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa55c143970 0x7fa55c158990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fa550000f10 tx=0x7fa54c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5500070d0 con 0x7fa55c143970 2020-07-18T03:26:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c0137a0 con 0x7fa55c143970 2020-07-18T03:26:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c13b5a0 con 0x7fa55c143970 2020-07-18T03:26:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa54c013910 con 0x7fa55c143970 2020-07-18T03:26:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.029+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c064340 con 0x7fa55c143970 2020-07-18T03:26:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.030+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c158680 con 0x7fa55c143970 2020-07-18T03:26:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.030+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa54c013b10 con 0x7fa55c143970 2020-07-18T03:26:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.031+0000 7fa53b7fe700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa5340502f0 0x7fa5340527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.031+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa54c067db0 con 0x7fa55c143970 2020-07-18T03:26:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.031+0000 7fa55affd700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa5340502f0 0x7fa5340527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.031+0000 7fa55affd700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa5340502f0 0x7fa5340527b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa550008f90 tx=0x7fa550015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.035+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa54c0404c0 con 0x7fa55c143970 2020-07-18T03:26:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.192+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa55c0690d0 con 0x7fa55c143970 2020-07-18T03:26:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.193+0000 7fa53b7fe700 1 -- 172.21.15.47:0/1964878366 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v33) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa54c018070 con 0x7fa55c143970 2020-07-18T03:26:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa5340502f0 msgr2=0x7fa5340527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa5340502f0 0x7fa5340527b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa550008f90 tx=0x7fa550015000).stop 2020-07-18T03:26:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa55c143970 msgr2=0x7fa55c158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa55c143970 0x7fa55c158990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fa550000f10 tx=0x7fa54c008040).stop 2020-07-18T03:26:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 shutdown_connections 2020-07-18T03:26:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa55c143970 0x7fa55c158990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 --2- 172.21.15.47:0/1964878366 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa5340502f0 0x7fa5340527b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 >> 172.21.15.47:0/1964878366 conn(0x7fa55c0b91d0 msgr2=0x7fa55c0b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 shutdown_connections 2020-07-18T03:26:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.194+0000 7fa560d9f700 1 -- 172.21.15.47:0/1964878366 wait complete. 2020-07-18T03:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:45.206 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.321+0000 7fe3a8112700 1 Processor -- start 2020-07-18T03:26:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.321+0000 7fe3a8112700 1 -- start start 2020-07-18T03:26:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a8112700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a0144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a8112700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3a0147d50 0x7fe3a0144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a8112700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a014b520 con 0x7fe3a00ad020 2020-07-18T03:26:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a8112700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a01471b0 con 0x7fe3a0147d50 2020-07-18T03:26:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a8112700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0147450 con 0x7fe3a01478f0 2020-07-18T03:26:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a66af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3a0147d50 0x7fe3a0144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a5eae700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe3a00ad020 0x7fe3a00b9ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41292/0 (socket says 172.21.15.47:41292) 2020-07-18T03:26:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a66af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3a0147d50 0x7fe3a0144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60208/0 (socket says 172.21.15.47:60208) 2020-07-18T03:26:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a5eae700 1 -- 172.21.15.47:0/2542067506 learned_addr learned my addr 172.21.15.47:0/2542067506 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a56ad700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a0144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a66af700 1 -- 172.21.15.47:0/2542067506 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 msgr2=0x7fe3a0144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.322+0000 7fe3a66af700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a0144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a66af700 1 -- 172.21.15.47:0/2542067506 >> v1:172.21.15.47:6789/0 conn(0x7fe3a00ad020 legacy=0x7fe3a00b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a66af700 1 -- 172.21.15.47:0/2542067506 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00ba940 con 0x7fe3a0147d50 2020-07-18T03:26:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a4eac700 1 -- 172.21.15.47:0/2542067506 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517016370 0 0) 0x7fe3a014b520 con 0x7fe3a00ad020 2020-07-18T03:26:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a56ad700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a0144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:45.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a66af700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0147d50 0x7fe3a0144720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fe3980032b0 tx=0x7fe3980091d0).ready entity=mon.1 client_cookie=d7f2e14ab581eeae server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a4eac700 1 -- 172.21.15.47:0/2542067506 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800ba50 con 0x7fe3a0147d50 2020-07-18T03:26:45.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a4eac700 1 -- 172.21.15.47:0/2542067506 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800bbb0 con 0x7fe3a0147d50 2020-07-18T03:26:45.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a4eac700 1 -- 172.21.15.47:0/2542067506 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800bd20 con 0x7fe3a0147d50 2020-07-18T03:26:45.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 -- 172.21.15.47:0/2542067506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0147d50 msgr2=0x7fe3a0144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0147d50 0x7fe3a0144720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fe3980032b0 tx=0x7fe3980091d0).stop 2020-07-18T03:26:45.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 -- 172.21.15.47:0/2542067506 shutdown_connections 2020-07-18T03:26:45.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a0147d50 0x7fe3a0144720 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 --2- 172.21.15.47:0/2542067506 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a0144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 -- 172.21.15.47:0/2542067506 >> 172.21.15.47:0/2542067506 conn(0x7fe3a005f5a0 msgr2=0x7fe3a005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 -- 172.21.15.47:0/2542067506 shutdown_connections 2020-07-18T03:26:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.323+0000 7fe3a8112700 1 -- 172.21.15.47:0/2542067506 wait complete. 2020-07-18T03:26:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.324+0000 7fe3a8112700 1 Processor -- start 2020-07-18T03:26:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a8112700 1 -- start start 2020-07-18T03:26:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a8112700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a00b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a8112700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe3a00b7060 0x7fe3a00b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a8112700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a0158780 con 0x7fe3a00ad020 2020-07-18T03:26:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a8112700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a0155e80 con 0x7fe3a00b7060 2020-07-18T03:26:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a8112700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0158a20 con 0x7fe3a01478f0 2020-07-18T03:26:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a56ad700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a00b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a66af700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe3a00b7060 0x7fe3a00b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a5eae700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fe3a00ad020 0x7fe3a01459c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60064/0 (socket says 172.21.15.47:60064) 2020-07-18T03:26:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a56ad700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a00b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:52810/0 (socket says 172.21.15.47:52810) 2020-07-18T03:26:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.325+0000 7fe3a5eae700 1 -- 172.21.15.47:0/1386927831 learned_addr learned my addr 172.21.15.47:0/1386927831 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a66af700 1 -- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 msgr2=0x7fe3a00b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a66af700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a00b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a66af700 1 -- 172.21.15.47:0/1386927831 >> v1:172.21.15.18:6789/0 conn(0x7fe3a00ad020 legacy=0x7fe3a01459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a66af700 1 -- 172.21.15.47:0/1386927831 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00b7a00 con 0x7fe3a00b7060 2020-07-18T03:26:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a56ad700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a00b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870520789 0 0) 0x7fe3a0158780 con 0x7fe3a00ad020 2020-07-18T03:26:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a66af700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3a00b7060 0x7fe3a00b7510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe3980032b0 tx=0x7fe398016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3980108c0 con 0x7fe3a00b7060 2020-07-18T03:26:45.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398010a20 con 0x7fe3a00b7060 2020-07-18T03:26:45.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a0158400 con 0x7fe3a00b7060 2020-07-18T03:26:45.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.326+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe398010d80 con 0x7fe3a00b7060 2020-07-18T03:26:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.327+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0159160 con 0x7fe3a00b7060 2020-07-18T03:26:45.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.327+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a00ba3f0 con 0x7fe3a00b7060 2020-07-18T03:26:45.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.328+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe39801e070 con 0x7fe3a00b7060 2020-07-18T03:26:45.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.328+0000 7fe391ffb700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe37804fe90 0x7fe378052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.328+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe398068d40 con 0x7fe3a00b7060 2020-07-18T03:26:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.328+0000 7fe3a5eae700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe37804fe90 0x7fe378052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.328+0000 7fe3a5eae700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe37804fe90 0x7fe378052350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fe394007650 tx=0x7fe3940078e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.332+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe3980413b0 con 0x7fe3a00b7060 2020-07-18T03:26:45.389 INFO:tasks.ceph.mds.a.smithi047.stderr:2020-07-18T03:26:45.387+0000 7f3d1ccfa700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:26:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.489+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3a00674f0 con 0x7fe3a00b7060 2020-07-18T03:26:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.491+0000 7fe391ffb700 1 -- 172.21.15.47:0/1386927831 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe398015020 con 0x7fe3a00b7060 2020-07-18T03:26:45.492 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe37804fe90 msgr2=0x7fe378052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe37804fe90 0x7fe378052350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fe394007650 tx=0x7fe3940078e0).stop 2020-07-18T03:26:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3a00b7060 msgr2=0x7fe3a00b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3a00b7060 0x7fe3a00b7510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe3980032b0 tx=0x7fe398016000).stop 2020-07-18T03:26:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 shutdown_connections 2020-07-18T03:26:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe37804fe90 0x7fe378052350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3a00b7060 0x7fe3a00b7510 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 --2- 172.21.15.47:0/1386927831 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe3a01478f0 0x7fe3a00b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 >> 172.21.15.47:0/1386927831 conn(0x7fe3a005f5a0 msgr2=0x7fe3a014a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 shutdown_connections 2020-07-18T03:26:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.492+0000 7fe3a8112700 1 -- 172.21.15.47:0/1386927831 wait complete. 2020-07-18T03:26:45.504 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.620+0000 7fc1f5a7f700 1 Processor -- start 2020-07-18T03:26:45.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.620+0000 7fc1f5a7f700 1 -- start start 2020-07-18T03:26:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1f5a7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1f00b3a10 0x7fc1f00b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1f5a7f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f0147de0 con 0x7fc1f00b59b0 2020-07-18T03:26:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1f5a7f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00ae000 con 0x7fc1f00ad330 2020-07-18T03:26:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1f5a7f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f0148010 con 0x7fc1f00b3a10 2020-07-18T03:26:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1effff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1f00b3a10 0x7fc1f00b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1ef7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc1f00ad330 0x7fc1f0143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41306/0 (socket says 172.21.15.47:41306) 2020-07-18T03:26:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1effff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1f00b3a10 0x7fc1f00b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60222/0 (socket says 172.21.15.47:60222) 2020-07-18T03:26:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1ef7fe700 1 -- 172.21.15.47:0/3885265964 learned_addr learned my addr 172.21.15.47:0/3885265964 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1effff700 1 -- 172.21.15.47:0/3885265964 >> v1:172.21.15.47:6789/0 conn(0x7fc1f00ad330 legacy=0x7fc1f0143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1effff700 1 -- 172.21.15.47:0/3885265964 >> v1:172.21.15.18:6789/0 conn(0x7fc1f00b59b0 legacy=0x7fc1f00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1effff700 1 -- 172.21.15.47:0/3885265964 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00ad710 con 0x7fc1f00b3a10 2020-07-18T03:26:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1effff700 1 --2- 172.21.15.47:0/3885265964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f00b3a10 0x7fc1f00b3e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fc1e80032b0 tx=0x7fc1e80091d0).ready entity=mon.1 client_cookie=150c724b741810d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.621+0000 7fc1ee7fc700 1 -- 172.21.15.47:0/3885265964 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1e800bc40 con 0x7fc1f00b3a10 2020-07-18T03:26:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1ee7fc700 1 -- 172.21.15.47:0/3885265964 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e800bda0 con 0x7fc1f00b3a10 2020-07-18T03:26:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1ee7fc700 1 -- 172.21.15.47:0/3885265964 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1e8010660 con 0x7fc1f00b3a10 2020-07-18T03:26:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/3885265964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f00b3a10 msgr2=0x7fc1f00b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 --2- 172.21.15.47:0/3885265964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f00b3a10 0x7fc1f00b3e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fc1e80032b0 tx=0x7fc1e80091d0).stop 2020-07-18T03:26:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/3885265964 shutdown_connections 2020-07-18T03:26:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 --2- 172.21.15.47:0/3885265964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f00b3a10 0x7fc1f00b3e80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/3885265964 >> 172.21.15.47:0/3885265964 conn(0x7fc1f005f5a0 msgr2=0x7fc1f005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/3885265964 shutdown_connections 2020-07-18T03:26:45.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.622+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/3885265964 wait complete. 2020-07-18T03:26:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.623+0000 7fc1f5a7f700 1 Processor -- start 2020-07-18T03:26:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.623+0000 7fc1f5a7f700 1 -- start start 2020-07-18T03:26:45.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1f5a7f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 0x7fc1f013c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1f5a7f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f013b530 con 0x7fc1f00ad330 2020-07-18T03:26:45.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1f5a7f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f0141a20 con 0x7fc1f00b59b0 2020-07-18T03:26:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1f5a7f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0141cc0 con 0x7fc1f0143ea0 2020-07-18T03:26:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1eeffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc1f00b59b0 0x7fc1f013dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41316/0 (socket says 172.21.15.47:41316) 2020-07-18T03:26:45.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1eeffd700 1 -- 172.21.15.47:0/970514902 learned_addr learned my addr 172.21.15.47:0/970514902 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1effff700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 0x7fc1f013c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342442353 0 0) 0x7fc1f0141a20 con 0x7fc1f00b59b0 2020-07-18T03:26:45.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.624+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c8003240 con 0x7fc1f00b59b0 2020-07-18T03:26:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1560275173 0 0) 0x7fc1c8003240 con 0x7fc1f00b59b0 2020-07-18T03:26:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1f0141a20 con 0x7fc1f00b59b0 2020-07-18T03:26:45.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fc1e40019c0 con 0x7fc1f00b59b0 2020-07-18T03:26:45.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 msgr2=0x7fc1f013c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 0x7fc1f013c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 >> v1:172.21.15.47:6789/0 conn(0x7fc1f00b59b0 legacy=0x7fc1f013dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 >> v1:172.21.15.18:6789/0 conn(0x7fc1f00ad330 legacy=0x7fc1f00b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c8003800 0x7fc1c8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1c80044e0 0x7fc1c8004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 0x7fc1c8004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0141cc0 con 0x7fc1c80044e0 2020-07-18T03:26:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c8005300 con 0x7fc1c8003800 2020-07-18T03:26:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c800b730 con 0x7fc1f0143ea0 2020-07-18T03:26:45.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c8003800 0x7fc1c8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1effff700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 0x7fc1c8004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1eeffd700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1c80044e0 0x7fc1c8004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 -- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 msgr2=0x7fc1c8004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f0143ea0 0x7fc1c8004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 -- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1c80044e0 msgr2=0x7fc1c8004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1c80044e0 0x7fc1c8004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0141f60 con 0x7fc1c8003800 2020-07-18T03:26:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1eeffd700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1c80044e0 0x7fc1c8004920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f013b990 con 0x7fc1c8003800 2020-07-18T03:26:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f013c5b0 con 0x7fc1c8003800 2020-07-18T03:26:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1ef7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c8003800 0x7fc1c8003c20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fc1f00b93e0 tx=0x7fc1e0007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1e00072e0 con 0x7fc1c8003800 2020-07-18T03:26:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e0007440 con 0x7fc1c8003800 2020-07-18T03:26:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.625+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1e00055f0 con 0x7fc1c8003800 2020-07-18T03:26:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.626+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc1e0010070 con 0x7fc1c8003800 2020-07-18T03:26:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.626+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f013c8f0 con 0x7fc1c8003800 2020-07-18T03:26:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.627+0000 7fc1cf7fe700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1c8058340 0x7fc1c805a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.627+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc1e0066ea0 con 0x7fc1c8003800 2020-07-18T03:26:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.627+0000 7fc1eeffd700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1c8058340 0x7fc1c805a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.627+0000 7fc1eeffd700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1c8058340 0x7fc1c805a800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc1e400b000 tx=0x7fc1e4009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.630+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc1e003f630 con 0x7fc1c8003800 2020-07-18T03:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.789+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1f005b120 con 0x7fc1c8003800 2020-07-18T03:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.789+0000 7fc1cf7fe700 1 -- 172.21.15.47:0/970514902 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc1e0042c50 con 0x7fc1c8003800 2020-07-18T03:26:45.793 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1c8058340 msgr2=0x7fc1c805a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1c8058340 0x7fc1c805a800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc1e400b000 tx=0x7fc1e4009040).stop 2020-07-18T03:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c8003800 msgr2=0x7fc1c8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 --2- 172.21.15.47:0/970514902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c8003800 0x7fc1c8003c20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fc1f00b93e0 tx=0x7fc1e0007c00).stop 2020-07-18T03:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1effff700 1 -- 172.21.15.47:0/970514902 reap_dead start 2020-07-18T03:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1effff700 1 -- 172.21.15.47:0/970514902 reap_dead start 2020-07-18T03:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 shutdown_connections 2020-07-18T03:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 >> 172.21.15.47:0/970514902 conn(0x7fc1f005f5a0 msgr2=0x7fc1f00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 shutdown_connections 2020-07-18T03:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.791+0000 7fc1f5a7f700 1 -- 172.21.15.47:0/970514902 wait complete. 2020-07-18T03:26:45.804 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.921+0000 7f7cc3d31700 1 Processor -- start 2020-07-18T03:26:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.921+0000 7f7cc3d31700 1 -- start start 2020-07-18T03:26:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc3d31700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cbc142a20 0x7f7cbc142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc3d31700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc0af4b0 con 0x7f7cbc061c60 2020-07-18T03:26:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc3d31700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc0b6d80 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc3d31700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc0af710 con 0x7f7cbc142a20 2020-07-18T03:26:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc1acd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cbc142a20 0x7f7cbc142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc22ce700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7cbc0b3af0 0x7f7cbc0b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41326/0 (socket says 172.21.15.47:41326) 2020-07-18T03:26:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc22ce700 1 -- 172.21.15.47:0/1577817388 learned_addr learned my addr 172.21.15.47:0/1577817388 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc1acd700 1 -- 172.21.15.47:0/1577817388 >> v1:172.21.15.47:6789/0 conn(0x7f7cbc0b3af0 legacy=0x7f7cbc0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc1acd700 1 -- 172.21.15.47:0/1577817388 >> v1:172.21.15.18:6789/0 conn(0x7f7cbc061c60 legacy=0x7f7cbc0620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc0acb700 1 -- 172.21.15.47:0/1577817388 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940245436 0 0) 0x7f7cbc0b6d80 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc1acd700 1 -- 172.21.15.47:0/1577817388 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc0af9b0 con 0x7f7cbc142a20 2020-07-18T03:26:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc1acd700 1 --2- 172.21.15.47:0/1577817388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc142a20 0x7f7cbc142e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f7cac002f30 tx=0x7f7cac006dd0).ready entity=mon.1 client_cookie=b790eae4b819deae server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc0acb700 1 -- 172.21.15.47:0/1577817388 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cac0097f0 con 0x7f7cbc142a20 2020-07-18T03:26:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc0acb700 1 -- 172.21.15.47:0/1577817388 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac009950 con 0x7f7cbc142a20 2020-07-18T03:26:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.922+0000 7f7cc0acb700 1 -- 172.21.15.47:0/1577817388 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cac009cb0 con 0x7f7cbc142a20 2020-07-18T03:26:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 -- 172.21.15.47:0/1577817388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc142a20 msgr2=0x7f7cbc142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/1577817388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc142a20 0x7f7cbc142e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f7cac002f30 tx=0x7f7cac006dd0).stop 2020-07-18T03:26:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 -- 172.21.15.47:0/1577817388 shutdown_connections 2020-07-18T03:26:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/1577817388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc142a20 0x7f7cbc142e00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 -- 172.21.15.47:0/1577817388 >> 172.21.15.47:0/1577817388 conn(0x7f7cbc05f5a0 msgr2=0x7f7cbc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 -- 172.21.15.47:0/1577817388 shutdown_connections 2020-07-18T03:26:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.923+0000 7f7cc3d31700 1 -- 172.21.15.47:0/1577817388 wait complete. 2020-07-18T03:26:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.924+0000 7f7cc3d31700 1 Processor -- start 2020-07-18T03:26:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.924+0000 7f7cc3d31700 1 -- start start 2020-07-18T03:26:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc3d31700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cbc0b3af0 0x7f7cbc0b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc3d31700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7cbc154c40 0x7f7cbc062fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc3d31700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc146420 con 0x7f7cbc061c60 2020-07-18T03:26:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc3d31700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc146240 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc3d31700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1466c0 con 0x7f7cbc154c40 2020-07-18T03:26:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cbc0b3af0 0x7f7cbc0b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc1acd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7cbc061c60 0x7f7cbc0b2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41332/0 (socket says 172.21.15.47:41332) 2020-07-18T03:26:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7cbc0b3af0 0x7f7cbc0b9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60246/0 (socket says 172.21.15.47:60246) 2020-07-18T03:26:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc1acd700 1 -- 172.21.15.47:0/673139721 learned_addr learned my addr 172.21.15.47:0/673139721 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc22ce700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7cbc154c40 0x7f7cbc062fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 -- 172.21.15.47:0/673139721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7cbc154c40 msgr2=0x7f7cbc062fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7cbc154c40 0x7f7cbc062fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 -- 172.21.15.47:0/673139721 >> v1:172.21.15.47:6789/0 conn(0x7f7cbc061c60 legacy=0x7f7cbc0b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 -- 172.21.15.47:0/673139721 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc1550b0 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420723872 0 0) 0x7f7cbc146420 con 0x7f7cbc061c60 2020-07-18T03:26:45.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7cc12cc700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc0b3af0 0x7f7cbc0b9da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f7cbc05d1a0 tx=0x7f7cb8007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.925+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cb800f470 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.926+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb800f5d0 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.926+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc0634c0 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.926+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cb800f930 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.926+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc13dfc0 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.927+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc146800 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.927+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7cb8010070 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.928+0000 7f7ca9ffb700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c9404fe10 0x7f7c940522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.928+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7cb8067150 con 0x7f7cbc0b3af0 2020-07-18T03:26:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.928+0000 7f7cc1acd700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c9404fe10 0x7f7c940522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.928+0000 7f7cc1acd700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c9404fe10 0x7f7c940522d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7cac002ef0 tx=0x7f7cac015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:45.931+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7cb803f8e0 con 0x7f7cbc0b3af0 2020-07-18T03:26:46.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.086+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cbc0b00a0 con 0x7f7cbc0b3af0 2020-07-18T03:26:46.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.086+0000 7f7ca9ffb700 1 -- 172.21.15.47:0/673139721 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7cb8042f00 con 0x7f7cbc0b3af0 2020-07-18T03:26:46.089 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:46.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c9404fe10 msgr2=0x7f7c940522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c9404fe10 0x7f7c940522d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7cac002ef0 tx=0x7f7cac015000).stop 2020-07-18T03:26:46.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc0b3af0 msgr2=0x7f7cbc0b9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc0b3af0 0x7f7cbc0b9da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f7cbc05d1a0 tx=0x7f7cb8007b70).stop 2020-07-18T03:26:46.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 shutdown_connections 2020-07-18T03:26:46.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7cbc154c40 0x7f7cbc062fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7c9404fe10 0x7f7c940522d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 --2- 172.21.15.47:0/673139721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cbc0b3af0 0x7f7cbc0b9da0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 >> 172.21.15.47:0/673139721 conn(0x7f7cbc05f5a0 msgr2=0x7f7cbc05d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:46.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 shutdown_connections 2020-07-18T03:26:46.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.088+0000 7f7cc3d31700 1 -- 172.21.15.47:0/673139721 wait complete. 2020-07-18T03:26:46.101 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:26:46.101 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.215+0000 7f15b62f0700 1 Processor -- start 2020-07-18T03:26:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.215+0000 7f15b62f0700 1 -- start start 2020-07-18T03:26:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b62f0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f15b00ad100 0x7f15b00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b62f0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15b0147a00 0x7f15b0147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b62f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15b005f5a0 0x7f15b005fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b62f0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b00ae530 con 0x7f15b00ad100 2020-07-18T03:26:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b62f0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0147330 con 0x7f15b005f5a0 2020-07-18T03:26:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b62f0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00b57e0 con 0x7f15b0147a00 2020-07-18T03:26:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b488d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15b005f5a0 0x7f15b005fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b488d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15b005f5a0 0x7f15b005fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60256/0 (socket says 172.21.15.47:60256) 2020-07-18T03:26:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.216+0000 7f15b488d700 1 -- 172.21.15.47:0/1534915774 learned_addr learned my addr 172.21.15.47:0/1534915774 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15affff700 1 --2- 172.21.15.47:0/1534915774 >> v2:172.21.15.18:3300/0 conn(0x7f15b00ad100 0x7f15b00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15af7fe700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15b0147a00 0x7f15b0147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b488d700 1 -- 172.21.15.47:0/1534915774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15b0147a00 msgr2=0x7f15b0147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b488d700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15b0147a00 0x7f15b0147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b488d700 1 -- 172.21.15.47:0/1534915774 >> v2:172.21.15.18:3300/0 conn(0x7f15b00ad100 msgr2=0x7f15b00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b488d700 1 --2- 172.21.15.47:0/1534915774 >> v2:172.21.15.18:3300/0 conn(0x7f15b00ad100 0x7f15b00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b488d700 1 -- 172.21.15.47:0/1534915774 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00b5a80 con 0x7f15b005f5a0 2020-07-18T03:26:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15af7fe700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15b0147a00 0x7f15b0147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b488d700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b005f5a0 0x7f15b005fa10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f15a80032b0 tx=0x7f15a8009270).ready entity=mon.1 client_cookie=1b53c962c6463d4d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15aeffd700 1 -- 172.21.15.47:0/1534915774 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a800bc00 con 0x7f15b005f5a0 2020-07-18T03:26:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15aeffd700 1 -- 172.21.15.47:0/1534915774 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a800bd60 con 0x7f15b005f5a0 2020-07-18T03:26:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15aeffd700 1 -- 172.21.15.47:0/1534915774 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a8011400 con 0x7f15b005f5a0 2020-07-18T03:26:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 -- 172.21.15.47:0/1534915774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b005f5a0 msgr2=0x7f15b005fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b005f5a0 0x7f15b005fa10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f15a80032b0 tx=0x7f15a8009270).stop 2020-07-18T03:26:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 -- 172.21.15.47:0/1534915774 shutdown_connections 2020-07-18T03:26:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b005f5a0 0x7f15b005fa10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 --2- 172.21.15.47:0/1534915774 >> v2:172.21.15.18:3300/0 conn(0x7f15b00ad100 0x7f15b00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 --2- 172.21.15.47:0/1534915774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15b0147a00 0x7f15b0147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 -- 172.21.15.47:0/1534915774 >> 172.21.15.47:0/1534915774 conn(0x7f15b00b9340 msgr2=0x7f15b00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 -- 172.21.15.47:0/1534915774 shutdown_connections 2020-07-18T03:26:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.217+0000 7f15b62f0700 1 -- 172.21.15.47:0/1534915774 wait complete. 2020-07-18T03:26:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 Processor -- start 2020-07-18T03:26:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 -- start start 2020-07-18T03:26:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15b00ad100 0x7f15b0142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f15b0146e10 0x7f15b0065fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b0146740 con 0x7f15b0147a00 2020-07-18T03:26:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b01469e0 con 0x7f15b0146e10 2020-07-18T03:26:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.219+0000 7f15b62f0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0146430 con 0x7f15b00ad100 2020-07-18T03:26:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15b00ad100 0x7f15b0142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15af7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f15b0147a00 0x7f15b00648a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41348/0 (socket says 172.21.15.47:41348) 2020-07-18T03:26:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15af7fe700 1 -- 172.21.15.47:0/284841421 learned_addr learned my addr 172.21.15.47:0/284841421 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:46.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15b488d700 1 --2- 172.21.15.47:0/284841421 >> v2:172.21.15.18:3300/0 conn(0x7f15b0146e10 0x7f15b0065fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:46.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15affff700 1 -- 172.21.15.47:0/284841421 >> v2:172.21.15.18:3300/0 conn(0x7f15b0146e10 msgr2=0x7f15b0065fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15affff700 1 --2- 172.21.15.47:0/284841421 >> v2:172.21.15.18:3300/0 conn(0x7f15b0146e10 0x7f15b0065fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15affff700 1 -- 172.21.15.47:0/284841421 >> v1:172.21.15.47:6789/0 conn(0x7f15b0147a00 legacy=0x7f15b00648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15affff700 1 -- 172.21.15.47:0/284841421 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0064fb0 con 0x7f15b00ad100 2020-07-18T03:26:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114871644 0 0) 0x7f15b0146740 con 0x7f15b0147a00 2020-07-18T03:26:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15affff700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b00ad100 0x7f15b0142310 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f15a000a000 tx=0x7f15a0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a00118a0 con 0x7f15b00ad100 2020-07-18T03:26:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0011a00 con 0x7f15b00ad100 2020-07-18T03:26:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b0142800 con 0x7f15b00ad100 2020-07-18T03:26:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a0011d60 con 0x7f15b00ad100 2020-07-18T03:26:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.220+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b0066660 con 0x7f15b00ad100 2020-07-18T03:26:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.221+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b01465a0 con 0x7f15b00ad100 2020-07-18T03:26:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.221+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f15a001c460 con 0x7f15b00ad100 2020-07-18T03:26:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.222+0000 7f158ffff700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f158804fe60 0x7f1588052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.222+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f15a0068030 con 0x7f15b00ad100 2020-07-18T03:26:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.222+0000 7f15af7fe700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f158804fe60 0x7f1588052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.222+0000 7f15af7fe700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f158804fe60 0x7f1588052320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f15a4003070 tx=0x7f15a4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.225+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f15b01465a0 con 0x7f15b00ad100 2020-07-18T03:26:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.434+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15b0146570 con 0x7f15b00ad100 2020-07-18T03:26:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.435+0000 7f158ffff700 1 -- 172.21.15.47:0/284841421 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f15a0040760 con 0x7f15b00ad100 2020-07-18T03:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:26:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f158804fe60 msgr2=0x7f1588052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f158804fe60 0x7f1588052320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f15a4003070 tx=0x7f15a4008040).stop 2020-07-18T03:26:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b00ad100 msgr2=0x7f15b0142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b00ad100 0x7f15b0142310 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f15a000a000 tx=0x7f15a0008040).stop 2020-07-18T03:26:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 shutdown_connections 2020-07-18T03:26:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 --2- 172.21.15.47:0/284841421 >> v2:172.21.15.18:3300/0 conn(0x7f15b0146e10 0x7f15b0065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15b00ad100 0x7f15b0142310 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 --2- 172.21.15.47:0/284841421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f158804fe60 0x7f1588052320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 >> 172.21.15.47:0/284841421 conn(0x7f15b00b9340 msgr2=0x7f15b00b9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 shutdown_connections 2020-07-18T03:26:46.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:46.436+0000 7f15b62f0700 1 -- 172.21.15.47:0/284841421 wait complete. 2020-07-18T03:26:46.441 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:26:46.650 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:26:40.770533+0000', 'modified': '2020-07-18T03:26:41.771684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5101}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5101': {'gid': 5101, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/3109073551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3109073551}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3109073551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:46.651 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:46.652 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:46.794 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5101,"want_state":"up:active","state":"up:active","rank_uptime":6.0088828249999997,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":13.236828385000001} 2020-07-18T03:26:46.795 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5101, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 6.008882825, "state": "up:active", "uptime": 13.236828385, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:46.795 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:26:46.796 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:26:46.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:46.797 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:26:46.817 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:46.819 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:46.820 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42784sec preferred_lft 42784sec 2020-07-18T03:26:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:26:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:46.821 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:46.822 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:46.822 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:26:46.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:46.824 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:26:46.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:46.890 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:26:46.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:46.964 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:26:47.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.036 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:26:47.106 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:26:47.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.108 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:26:47.164 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:26:47.164 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:47.164 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:47.165 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:47.165 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:26:47.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.166 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:47.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.238 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:47.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.311 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:47.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.383 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:26:47.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.438 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:26:47.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.493 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:26:47.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.566 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-07-18T03:26:47.645 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:26:47.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.649 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-07-18T03:26:47.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.734 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:26:47.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.824 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:26:47.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.881 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:26:47.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:47.935 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:26:47.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:48.000 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:26:48.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:48.041 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:26:48.141 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:26:48.141 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:26:48.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:48.142 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:48.158 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:48.232 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:48.233 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:48.305 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:48.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:48.306 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:48.359 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:26:48.360 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:48.402 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:48.434 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:48.435 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:48.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:26:48.500+0000 7f1c485b11c0 -1 init, newargv = 0x55b45581d8c0 newargc=15 2020-07-18T03:26:48.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[33695]: starting ceph client 2020-07-18T03:26:48.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[33695]: starting fuse 2020-07-18T03:26:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:48.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:48.510 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:48.564 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:26:48.565 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:26:48.565 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:26:48.566 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:26:48.566 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:26:48.567 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:26:48.567 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:26:48.568 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:48.568 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:26:48.569 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:26:48.569 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:26:48.570 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:26:48.570 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:48.570 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:26:48.571 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:26:48.571 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:26:48.572 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:26:48.572 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:48.573 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:26:48.573 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:26:48.574 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:26:48.574 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:26:48.574 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:26:48.575 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:26:48.575 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:26:48.576 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:26:48.576 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:48.577 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:26:48.577 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:26:48.696 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.33695.asok 2020-07-18T03:26:48.876 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33695.asok 2020-07-18T03:26:48.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:48.877 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33695.asok status 2020-07-18T03:26:49.028 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:26:49.029 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:26:49.029 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:26:49.029 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:26:49.030 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:26:49.030 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:26:49.030 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:26:49.030 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "33695", 2020-07-18T03:26:49.030 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:26:49.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:49.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:26:49.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:26:49.031 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5362, 2020-07-18T03:26:49.031 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:26:49.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:26:49.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:26:49.032 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5362 2020-07-18T03:26:49.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:49.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:49.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:49.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:26:49.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4048855830 2020-07-18T03:26:49.033 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:49.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:49.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:49.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:49.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:26:49.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4048855830 2020-07-18T03:26:49.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:49.034 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5362 192.168.0.1:0/4048855830", 2020-07-18T03:26:49.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/4048855830", 2020-07-18T03:26:49.035 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:26:49.035 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 11, 2020-07-18T03:26:49.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 33, 2020-07-18T03:26:49.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:26:49.042 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:26:49.043 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:26:49.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:49.044 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:26:49.045 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:26:49.064 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:26:49.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:49.066 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:49.137 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:26:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.290+0000 7fac2fd4b700 1 Processor -- start 2020-07-18T03:26:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.290+0000 7fac2fd4b700 1 -- start start 2020-07-18T03:26:49.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2fd4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac280ad9e0 0x7fac281428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2fd4b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280b3ac0 con 0x7fac280b5370 2020-07-18T03:26:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2fd4b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280b3d20 con 0x7fac280b41c0 2020-07-18T03:26:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2fd4b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac280b5a10 con 0x7fac280ad9e0 2020-07-18T03:26:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2d2e6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac280b41c0 0x7fac280ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41370/0 (socket says 172.21.15.47:41370) 2020-07-18T03:26:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2d2e6700 1 -- 172.21.15.47:0/863738342 learned_addr learned my addr 172.21.15.47:0/863738342 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2e2e8700 1 --2- 172.21.15.47:0/863738342 >> v2:172.21.15.47:3300/0 conn(0x7fac280ad9e0 0x7fac281428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.291+0000 7fac2cae5700 1 -- 172.21.15.47:0/863738342 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676366233 0 0) 0x7fac280b3d20 con 0x7fac280b41c0 2020-07-18T03:26:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2cae5700 1 -- 172.21.15.47:0/863738342 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac10003240 con 0x7fac280b41c0 2020-07-18T03:26:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2e2e8700 1 -- 172.21.15.47:0/863738342 >> v1:172.21.15.47:6789/0 conn(0x7fac280b41c0 legacy=0x7fac280ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2e2e8700 1 -- 172.21.15.47:0/863738342 >> v1:172.21.15.18:6789/0 conn(0x7fac280b5370 legacy=0x7fac2805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2e2e8700 1 -- 172.21.15.47:0/863738342 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280b5750 con 0x7fac280ad9e0 2020-07-18T03:26:49.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2e2e8700 1 --2- 172.21.15.47:0/863738342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280ad9e0 0x7fac281428f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fac280afb30 tx=0x7fac200091d0).ready entity=mon.1 client_cookie=8af30e630aa12bb0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2cae5700 1 -- 172.21.15.47:0/863738342 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac2000bb80 con 0x7fac280ad9e0 2020-07-18T03:26:49.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2cae5700 1 -- 172.21.15.47:0/863738342 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000bce0 con 0x7fac280ad9e0 2020-07-18T03:26:49.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2cae5700 1 -- 172.21.15.47:0/863738342 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac20010600 con 0x7fac280ad9e0 2020-07-18T03:26:49.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 -- 172.21.15.47:0/863738342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280ad9e0 msgr2=0x7fac281428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/863738342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280ad9e0 0x7fac281428f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fac280afb30 tx=0x7fac200091d0).stop 2020-07-18T03:26:49.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 -- 172.21.15.47:0/863738342 shutdown_connections 2020-07-18T03:26:49.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/863738342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280ad9e0 0x7fac281428f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 -- 172.21.15.47:0/863738342 >> 172.21.15.47:0/863738342 conn(0x7fac280b9340 msgr2=0x7fac280b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:49.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 -- 172.21.15.47:0/863738342 shutdown_connections 2020-07-18T03:26:49.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.292+0000 7fac2fd4b700 1 -- 172.21.15.47:0/863738342 wait complete. 2020-07-18T03:26:49.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 Processor -- start 2020-07-18T03:26:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 -- start start 2020-07-18T03:26:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac280b41c0 0x7fac28130df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac280b5370 0x7fac280b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fac2812fbe0 0x7fac28130090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac281312e0 con 0x7fac2812fbe0 2020-07-18T03:26:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac28062e80 con 0x7fac280b41c0 2020-07-18T03:26:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2fd4b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac28063120 con 0x7fac280b5370 2020-07-18T03:26:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2dae7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac280b41c0 0x7fac28130df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2dae7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac280b41c0 0x7fac28130df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60286/0 (socket says 172.21.15.47:60286) 2020-07-18T03:26:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.294+0000 7fac2dae7700 1 -- 172.21.15.47:0/1332812732 learned_addr learned my addr 172.21.15.47:0/1332812732 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2e2e8700 1 --2- 172.21.15.47:0/1332812732 >> v2:172.21.15.18:3300/0 conn(0x7fac2812fbe0 0x7fac28130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2d2e6700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac280b5370 0x7fac280b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2dae7700 1 -- 172.21.15.47:0/1332812732 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac280b5370 msgr2=0x7fac280b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2dae7700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac280b5370 0x7fac280b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2dae7700 1 -- 172.21.15.47:0/1332812732 >> v2:172.21.15.18:3300/0 conn(0x7fac2812fbe0 msgr2=0x7fac28130090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2dae7700 1 --2- 172.21.15.47:0/1332812732 >> v2:172.21.15.18:3300/0 conn(0x7fac2812fbe0 0x7fac28130090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2dae7700 1 -- 172.21.15.47:0/1332812732 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280633c0 con 0x7fac280b41c0 2020-07-18T03:26:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2e2e8700 1 --2- 172.21.15.47:0/1332812732 >> v2:172.21.15.18:3300/0 conn(0x7fac2812fbe0 0x7fac28130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2d2e6700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac280b5370 0x7fac280b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2dae7700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280b41c0 0x7fac28130df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fac280af270 tx=0x7fac1c007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac1c00e4a0 con 0x7fac280b41c0 2020-07-18T03:26:49.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00e600 con 0x7fac280b41c0 2020-07-18T03:26:49.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac28130580 con 0x7fac280b41c0 2020-07-18T03:26:49.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac1c00e960 con 0x7fac280b41c0 2020-07-18T03:26:49.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.295+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac28130980 con 0x7fac280b41c0 2020-07-18T03:26:49.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.296+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fac1c019030 con 0x7fac280b41c0 2020-07-18T03:26:49.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.296+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac280ae930 con 0x7fac280b41c0 2020-07-18T03:26:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.297+0000 7fac19ffb700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac00050100 0x7fac000525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.297+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fac1c0671c0 con 0x7fac280b41c0 2020-07-18T03:26:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.297+0000 7fac2d2e6700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac00050100 0x7fac000525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.297+0000 7fac2d2e6700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac00050100 0x7fac000525c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fac2400b000 tx=0x7fac24009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.300+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fac1c03f8d0 con 0x7fac280b41c0 2020-07-18T03:26:49.334 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:26:49.337 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.458+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac28057450 con 0x7fac280b41c0 2020-07-18T03:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.459+0000 7fac19ffb700 1 -- 172.21.15.47:0/1332812732 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fac1c017090 con 0x7fac280b41c0 2020-07-18T03:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac00050100 msgr2=0x7fac000525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac00050100 0x7fac000525c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fac2400b000 tx=0x7fac24009040).stop 2020-07-18T03:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280b41c0 msgr2=0x7fac28130df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280b41c0 0x7fac28130df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fac280af270 tx=0x7fac1c007c30).stop 2020-07-18T03:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 shutdown_connections 2020-07-18T03:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac00050100 0x7fac000525c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/1332812732 >> v2:172.21.15.18:3300/0 conn(0x7fac2812fbe0 0x7fac28130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280b41c0 0x7fac28130df0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.460+0000 7fac2fd4b700 1 --2- 172.21.15.47:0/1332812732 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fac280b5370 0x7fac280b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.461+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 >> 172.21.15.47:0/1332812732 conn(0x7fac280b9340 msgr2=0x7fac280b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.461+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 shutdown_connections 2020-07-18T03:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.461+0000 7fac2fd4b700 1 -- 172.21.15.47:0/1332812732 wait complete. 2020-07-18T03:26:49.467 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:49.467 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:26:41.771920+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:26:38.665431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-07-18T03:26:39.748890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:26:49.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:26:38.665431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:26:39.748890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:26:49.474 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:26:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.590+0000 7efcf8130700 1 Processor -- start 2020-07-18T03:26:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.590+0000 7efcf8130700 1 -- start start 2020-07-18T03:26:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf8130700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efcf0142a20 0x7efcf0142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf8130700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efcf0061c60 0x7efcf00620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf8130700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efcf00b5dc0 0x7efcf0146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf8130700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf00af4b0 con 0x7efcf0061c60 2020-07-18T03:26:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf8130700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf00b6d80 con 0x7efcf00b5dc0 2020-07-18T03:26:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf8130700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00af710 con 0x7efcf0142a20 2020-07-18T03:26:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efcf00b5dc0 0x7efcf0146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efcf00b5dc0 0x7efcf0146ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60294/0 (socket says 172.21.15.47:60294) 2020-07-18T03:26:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 -- 172.21.15.47:0/4173244497 learned_addr learned my addr 172.21.15.47:0/4173244497 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf56cb700 1 --2- 172.21.15.47:0/4173244497 >> v2:172.21.15.18:3300/0 conn(0x7efcf0061c60 0x7efcf00620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf5ecc700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efcf0142a20 0x7efcf0142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 -- 172.21.15.47:0/4173244497 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efcf0142a20 msgr2=0x7efcf0142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efcf0142a20 0x7efcf0142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 -- 172.21.15.47:0/4173244497 >> v2:172.21.15.18:3300/0 conn(0x7efcf0061c60 msgr2=0x7efcf00620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 --2- 172.21.15.47:0/4173244497 >> v2:172.21.15.18:3300/0 conn(0x7efcf0061c60 0x7efcf00620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf66cd700 1 -- 172.21.15.47:0/4173244497 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf00af9b0 con 0x7efcf00b5dc0 2020-07-18T03:26:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.591+0000 7efcf5ecc700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efcf0142a20 0x7efcf0142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf66cd700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcf00b5dc0 0x7efcf0146ac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7efce80032b0 tx=0x7efce8009270).ready entity=mon.1 client_cookie=91cc58f7369a7d7e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf4eca700 1 -- 172.21.15.47:0/4173244497 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce8009e60 con 0x7efcf00b5dc0 2020-07-18T03:26:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf4eca700 1 -- 172.21.15.47:0/4173244497 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce800bc00 con 0x7efcf00b5dc0 2020-07-18T03:26:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf4eca700 1 -- 172.21.15.47:0/4173244497 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce800bd70 con 0x7efcf00b5dc0 2020-07-18T03:26:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 -- 172.21.15.47:0/4173244497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcf00b5dc0 msgr2=0x7efcf0146ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcf00b5dc0 0x7efcf0146ac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7efce80032b0 tx=0x7efce8009270).stop 2020-07-18T03:26:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 -- 172.21.15.47:0/4173244497 shutdown_connections 2020-07-18T03:26:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcf00b5dc0 0x7efcf0146ac0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 --2- 172.21.15.47:0/4173244497 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efcf0142a20 0x7efcf0142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 --2- 172.21.15.47:0/4173244497 >> v2:172.21.15.18:3300/0 conn(0x7efcf0061c60 0x7efcf00620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 -- 172.21.15.47:0/4173244497 >> 172.21.15.47:0/4173244497 conn(0x7efcf005f5a0 msgr2=0x7efcf005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 -- 172.21.15.47:0/4173244497 shutdown_connections 2020-07-18T03:26:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.592+0000 7efcf8130700 1 -- 172.21.15.47:0/4173244497 wait complete. 2020-07-18T03:26:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.593+0000 7efcf8130700 1 Processor -- start 2020-07-18T03:26:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.593+0000 7efcf8130700 1 -- start start 2020-07-18T03:26:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf8130700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efcf0142a20 0x7efcf0139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf8130700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf0142e00 con 0x7efcf0061c60 2020-07-18T03:26:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf8130700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf013a2e0 con 0x7efcf0063010 2020-07-18T03:26:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf8130700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf0143030 con 0x7efcf0142a20 2020-07-18T03:26:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf66cd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efcf0063010 0x7efcf0138eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41388/0 (socket says 172.21.15.47:41388) 2020-07-18T03:26:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf66cd700 1 -- 172.21.15.47:0/3194421474 learned_addr learned my addr 172.21.15.47:0/3194421474 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efcf56cb700 1 --2- 172.21.15.47:0/3194421474 >> v2:172.21.15.18:3300/0 conn(0x7efcf0142a20 0x7efcf0139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522717441 0 0) 0x7efcf013a2e0 con 0x7efcf0063010 2020-07-18T03:26:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.594+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc8003240 con 0x7efcf0063010 2020-07-18T03:26:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428248272 0 0) 0x7efcf0142e00 con 0x7efcf0061c60 2020-07-18T03:26:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf013a2e0 con 0x7efcf0061c60 2020-07-18T03:26:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2310819384 0 0) 0x7efcc8003240 con 0x7efcf0063010 2020-07-18T03:26:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efcf0142e00 con 0x7efcf0063010 2020-07-18T03:26:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7efce8009e60 con 0x7efcf0063010 2020-07-18T03:26:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efcf56cb700 1 -- 172.21.15.47:0/3194421474 >> v1:172.21.15.47:6789/0 conn(0x7efcf0063010 legacy=0x7efcf0138eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efcf56cb700 1 -- 172.21.15.47:0/3194421474 >> v1:172.21.15.18:6789/0 conn(0x7efcf0061c60 legacy=0x7efcf0144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efcf56cb700 1 -- 172.21.15.47:0/3194421474 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf01432d0 con 0x7efcf0142a20 2020-07-18T03:26:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efcf56cb700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efcf0142a20 0x7efcf0139df0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7efcec00c000 tx=0x7efcec006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcec00c610 con 0x7efcf0142a20 2020-07-18T03:26:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf0064080 con 0x7efcf0142a20 2020-07-18T03:26:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec00c770 con 0x7efcf0142a20 2020-07-18T03:26:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcec00cad0 con 0x7efcf0142a20 2020-07-18T03:26:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.595+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf0139780 con 0x7efcf0142a20 2020-07-18T03:26:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.596+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf0143490 con 0x7efcf0142a20 2020-07-18T03:26:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.597+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7efcec00cc80 con 0x7efcf0142a20 2020-07-18T03:26:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.597+0000 7efce1ffb700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efcc8050720 0x7efcc8052be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.597+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efcec067c00 con 0x7efcf0142a20 2020-07-18T03:26:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.597+0000 7efcf5ecc700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efcc8050720 0x7efcc8052be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.597+0000 7efcf5ecc700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efcc8050720 0x7efcc8052be0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7efce4007000 tx=0x7efce4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.601+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7efcec040310 con 0x7efcf0142a20 2020-07-18T03:26:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.762+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efcf005b120 con 0x7efcf0142a20 2020-07-18T03:26:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.762+0000 7efce1ffb700 1 -- 172.21.15.47:0/3194421474 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v33) v1 ==== 110+0+10 (secure 0 0 0) 0x7efcec043930 con 0x7efcf0142a20 2020-07-18T03:26:49.765 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:26:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efcc8050720 msgr2=0x7efcc8052be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efcc8050720 0x7efcc8052be0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7efce4007000 tx=0x7efce4008040).stop 2020-07-18T03:26:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efcf0142a20 msgr2=0x7efcf0139df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efcf0142a20 0x7efcf0139df0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7efcec00c000 tx=0x7efcec006dd0).stop 2020-07-18T03:26:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 shutdown_connections 2020-07-18T03:26:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efcc8050720 0x7efcc8052be0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 --2- 172.21.15.47:0/3194421474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efcf0142a20 0x7efcf0139df0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 >> 172.21.15.47:0/3194421474 conn(0x7efcf005f5a0 msgr2=0x7efcf013ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 shutdown_connections 2020-07-18T03:26:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.764+0000 7efcf8130700 1 -- 172.21.15.47:0/3194421474 wait complete. 2020-07-18T03:26:49.776 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:26:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.897+0000 7fb53f9f9700 1 Processor -- start 2020-07-18T03:26:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.897+0000 7fb53f9f9700 1 -- start start 2020-07-18T03:26:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53f9f9700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb538142a20 0x7fb538142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53f9f9700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb538061c60 0x7fb5380620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53f9f9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5380af4b0 con 0x7fb538146ac0 2020-07-18T03:26:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53f9f9700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380b6d80 con 0x7fb538142a20 2020-07-18T03:26:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53f9f9700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380af710 con 0x7fb538061c60 2020-07-18T03:26:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53df96700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb538146ac0 0x7fb538146f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41394/0 (socket says 172.21.15.47:41394) 2020-07-18T03:26:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53df96700 1 -- 172.21.15.47:0/1342124692 learned_addr learned my addr 172.21.15.47:0/1342124692 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53d795700 1 --2- 172.21.15.47:0/1342124692 >> v2:172.21.15.18:3300/0 conn(0x7fb538142a20 0x7fb538142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb53cf94700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb538061c60 0x7fb5380620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb527fff700 1 -- 172.21.15.47:0/1342124692 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357954873 0 0) 0x7fb5380af4b0 con 0x7fb538146ac0 2020-07-18T03:26:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.898+0000 7fb527fff700 1 -- 172.21.15.47:0/1342124692 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb520003240 con 0x7fb538146ac0 2020-07-18T03:26:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb527fff700 1 -- 172.21.15.47:0/1342124692 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 67947204 0 0) 0x7fb520003240 con 0x7fb538146ac0 2020-07-18T03:26:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53d795700 1 -- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb538061c60 msgr2=0x7fb5380620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53d795700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb538061c60 0x7fb5380620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53d795700 1 -- 172.21.15.47:0/1342124692 >> v1:172.21.15.47:6789/0 conn(0x7fb538146ac0 legacy=0x7fb538146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53d795700 1 -- 172.21.15.47:0/1342124692 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380af9b0 con 0x7fb538142a20 2020-07-18T03:26:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53cf94700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb538061c60 0x7fb5380620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53d795700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb538142a20 0x7fb538142e00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fb530001bf0 tx=0x7fb528006dd0).ready entity=mon.0 client_cookie=93a45cdfc90e2aa server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb527fff700 1 -- 172.21.15.47:0/1342124692 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb530001a50 con 0x7fb538142a20 2020-07-18T03:26:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb527fff700 1 -- 172.21.15.47:0/1342124692 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800d570 con 0x7fb538142a20 2020-07-18T03:26:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb527fff700 1 -- 172.21.15.47:0/1342124692 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52800d6e0 con 0x7fb538142a20 2020-07-18T03:26:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb538142a20 msgr2=0x7fb538142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb538142a20 0x7fb538142e00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fb530001bf0 tx=0x7fb528006dd0).stop 2020-07-18T03:26:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1342124692 shutdown_connections 2020-07-18T03:26:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb538142a20 0x7fb538142e00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1342124692 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb538061c60 0x7fb5380620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1342124692 >> 172.21.15.47:0/1342124692 conn(0x7fb53805f5a0 msgr2=0x7fb53805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1342124692 shutdown_connections 2020-07-18T03:26:49.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.899+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1342124692 wait complete. 2020-07-18T03:26:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.901+0000 7fb53f9f9700 1 Processor -- start 2020-07-18T03:26:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.901+0000 7fb53f9f9700 1 -- start start 2020-07-18T03:26:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb538146ac0 0x7fb538152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb53813cbc0 0x7fb538128970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb538062e80 con 0x7fb538061c60 2020-07-18T03:26:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380630b0 con 0x7fb538146ac0 2020-07-18T03:26:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb538063350 con 0x7fb53813cbc0 2020-07-18T03:26:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb538146ac0 0x7fb538152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb538146ac0 0x7fb538152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60314/0 (socket says 172.21.15.47:60314) 2020-07-18T03:26:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 -- 172.21.15.47:0/1602227297 learned_addr learned my addr 172.21.15.47:0/1602227297 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53df96700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb53813cbc0 0x7fb538128970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 -- 172.21.15.47:0/1602227297 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb53813cbc0 msgr2=0x7fb538128970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb53813cbc0 0x7fb538128970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 -- 172.21.15.47:0/1602227297 >> v1:172.21.15.18:6789/0 conn(0x7fb538061c60 legacy=0x7fb5380b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 -- 172.21.15.47:0/1602227297 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380635f0 con 0x7fb538146ac0 2020-07-18T03:26:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53cf94700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538146ac0 0x7fb538152de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fb53400c000 tx=0x7fb534006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb53400cbf0 con 0x7fb538146ac0 2020-07-18T03:26:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53400cd50 con 0x7fb538146ac0 2020-07-18T03:26:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5381532d0 con 0x7fb538146ac0 2020-07-18T03:26:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5340125e0 con 0x7fb538146ac0 2020-07-18T03:26:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.902+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5380b3490 con 0x7fb538146ac0 2020-07-18T03:26:49.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.903+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5380637b0 con 0x7fb538146ac0 2020-07-18T03:26:49.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.903+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb534012740 con 0x7fb538146ac0 2020-07-18T03:26:49.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.904+0000 7fb5257fa700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb51004ffb0 0x7fb510052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:49.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.904+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb534068320 con 0x7fb538146ac0 2020-07-18T03:26:49.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.904+0000 7fb53d795700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb51004ffb0 0x7fb510052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:49.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.904+0000 7fb53d795700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb51004ffb0 0x7fb510052470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fb538146460 tx=0x7fb528015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:49.907+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb534040a50 con 0x7fb538146ac0 2020-07-18T03:26:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.063+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb53805b120 con 0x7fb538146ac0 2020-07-18T03:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.063+0000 7fb5257fa700 1 -- 172.21.15.47:0/1602227297 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb534044070 con 0x7fb538146ac0 2020-07-18T03:26:50.066 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb51004ffb0 msgr2=0x7fb510052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb51004ffb0 0x7fb510052470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fb538146460 tx=0x7fb528015000).stop 2020-07-18T03:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538146ac0 msgr2=0x7fb538152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538146ac0 0x7fb538152de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fb53400c000 tx=0x7fb534006dd0).stop 2020-07-18T03:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 shutdown_connections 2020-07-18T03:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb51004ffb0 0x7fb510052470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb53813cbc0 0x7fb538128970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 --2- 172.21.15.47:0/1602227297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538146ac0 0x7fb538152de0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 >> 172.21.15.47:0/1602227297 conn(0x7fb53805f5a0 msgr2=0x7fb53814df10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 shutdown_connections 2020-07-18T03:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.065+0000 7fb53f9f9700 1 -- 172.21.15.47:0/1602227297 wait complete. 2020-07-18T03:26:50.077 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:26:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.206+0000 7f419ba0f700 1 Processor -- start 2020-07-18T03:26:50.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.206+0000 7f419ba0f700 1 -- start start 2020-07-18T03:26:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f419ba0f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f419405fce0 0x7f4194066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f419ba0f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f41940670c0 0x7f41941441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f419ba0f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41940600c0 0x7f41941446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f419ba0f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419405f5a0 con 0x7f41940670c0 2020-07-18T03:26:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f419ba0f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419405f840 con 0x7f41940600c0 2020-07-18T03:26:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f419ba0f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4194065360 con 0x7f419405fce0 2020-07-18T03:26:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f4199fac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41940600c0 0x7f41941446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f4199fac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41940600c0 0x7f41941446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60320/0 (socket says 172.21.15.47:60320) 2020-07-18T03:26:50.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.207+0000 7f4199fac700 1 -- 172.21.15.47:0/1252940117 learned_addr learned my addr 172.21.15.47:0/1252940117 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:50.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4198faa700 1 --2- 172.21.15.47:0/1252940117 >> v2:172.21.15.18:3300/0 conn(0x7f41940670c0 0x7f41941441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f41997ab700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f419405fce0 0x7f4194066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4199fac700 1 -- 172.21.15.47:0/1252940117 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f419405fce0 msgr2=0x7f4194066bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:50.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4199fac700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f419405fce0 0x7f4194066bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4199fac700 1 -- 172.21.15.47:0/1252940117 >> v2:172.21.15.18:3300/0 conn(0x7f41940670c0 msgr2=0x7f41941441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:50.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4199fac700 1 --2- 172.21.15.47:0/1252940117 >> v2:172.21.15.18:3300/0 conn(0x7f41940670c0 0x7f41941441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4199fac700 1 -- 172.21.15.47:0/1252940117 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419405c5e0 con 0x7f41940600c0 2020-07-18T03:26:50.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4198faa700 1 --2- 172.21.15.47:0/1252940117 >> v2:172.21.15.18:3300/0 conn(0x7f41940670c0 0x7f41941441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:50.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f41997ab700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f419405fce0 0x7f4194066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:26:50.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4199fac700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940600c0 0x7f41941446a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f418c002f30 tx=0x7f418c006e90).ready entity=mon.1 client_cookie=f4a77d3c6d83f614 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4183fff700 1 -- 172.21.15.47:0/1252940117 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f418c009920 con 0x7f41940600c0 2020-07-18T03:26:50.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4183fff700 1 -- 172.21.15.47:0/1252940117 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c009a80 con 0x7f41940600c0 2020-07-18T03:26:50.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f4183fff700 1 -- 172.21.15.47:0/1252940117 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f418c009de0 con 0x7f41940600c0 2020-07-18T03:26:50.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 -- 172.21.15.47:0/1252940117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940600c0 msgr2=0x7f41941446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940600c0 0x7f41941446a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f418c002f30 tx=0x7f418c006e90).stop 2020-07-18T03:26:50.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 -- 172.21.15.47:0/1252940117 shutdown_connections 2020-07-18T03:26:50.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940600c0 0x7f41941446a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 --2- 172.21.15.47:0/1252940117 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f419405fce0 0x7f4194066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 --2- 172.21.15.47:0/1252940117 >> v2:172.21.15.18:3300/0 conn(0x7f41940670c0 0x7f41941441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 -- 172.21.15.47:0/1252940117 >> 172.21.15.47:0/1252940117 conn(0x7f41940baf50 msgr2=0x7f419405b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 -- 172.21.15.47:0/1252940117 shutdown_connections 2020-07-18T03:26:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.208+0000 7f419ba0f700 1 -- 172.21.15.47:0/1252940117 wait complete. 2020-07-18T03:26:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.210+0000 7f419ba0f700 1 Processor -- start 2020-07-18T03:26:50.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.210+0000 7f419ba0f700 1 -- start start 2020-07-18T03:26:50.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f419ba0f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41940b5a20 0x7f4194133c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f419ba0f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41940b5eb0 0x7f4194134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f419ba0f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41940b6330 con 0x7f419405fce0 2020-07-18T03:26:50.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f419ba0f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41940b5710 con 0x7f41940b5a20 2020-07-18T03:26:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f419ba0f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41940b65d0 con 0x7f41940b5eb0 2020-07-18T03:26:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f41997ab700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f419405fce0 0x7f41941495f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41414/0 (socket says 172.21.15.47:41414) 2020-07-18T03:26:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f41997ab700 1 -- 172.21.15.47:0/2091645724 learned_addr learned my addr 172.21.15.47:0/2091645724 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:50.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4198faa700 1 --2- 172.21.15.47:0/2091645724 >> v2:172.21.15.47:3300/0 conn(0x7f41940b5a20 0x7f4194133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4199fac700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41940b5eb0 0x7f4194134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353339307 0 0) 0x7f41940b6330 con 0x7f419405fce0 2020-07-18T03:26:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f416c003240 con 0x7f419405fce0 2020-07-18T03:26:50.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4198faa700 1 -- 172.21.15.47:0/2091645724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41940b5eb0 msgr2=0x7f4194134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4198faa700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41940b5eb0 0x7f4194134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4198faa700 1 -- 172.21.15.47:0/2091645724 >> v1:172.21.15.47:6789/0 conn(0x7f419405fce0 legacy=0x7f41941495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4198faa700 1 -- 172.21.15.47:0/2091645724 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41940b6870 con 0x7f41940b5a20 2020-07-18T03:26:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4199fac700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41940b5eb0 0x7f4194134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:50.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f4198faa700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940b5a20 0x7f4194133c80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f4190007000 tx=0x7f4190008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.211+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4190007860 con 0x7f41940b5a20 2020-07-18T03:26:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.212+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41940b1430 con 0x7f41940b5a20 2020-07-18T03:26:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.212+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41900079c0 con 0x7f41940b5a20 2020-07-18T03:26:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.212+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4190007d20 con 0x7f41940b5a20 2020-07-18T03:26:50.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.212+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41940b4700 con 0x7f41940b5a20 2020-07-18T03:26:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.213+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41940b4540 con 0x7f41940b5a20 2020-07-18T03:26:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.213+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4190025430 con 0x7f41940b5a20 2020-07-18T03:26:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.214+0000 7f41817fa700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f416c050330 0x7f416c0527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.214+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4190068c50 con 0x7f41940b5a20 2020-07-18T03:26:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.214+0000 7f41997ab700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f416c050330 0x7f416c0527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.214+0000 7f41997ab700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f416c050330 0x7f416c0527f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f4184002f30 tx=0x7f4184008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.217+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4190041380 con 0x7f41940b5a20 2020-07-18T03:26:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.372+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f419405b120 con 0x7f41940b5a20 2020-07-18T03:26:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.372+0000 7f41817fa700 1 -- 172.21.15.47:0/2091645724 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+10 (secure 0 0 0) 0x7f41900449a0 con 0x7f41940b5a20 2020-07-18T03:26:50.374 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f416c050330 msgr2=0x7f416c0527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f416c050330 0x7f416c0527f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f4184002f30 tx=0x7f4184008040).stop 2020-07-18T03:26:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940b5a20 msgr2=0x7f4194133c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940b5a20 0x7f4194133c80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f4190007000 tx=0x7f4190008040).stop 2020-07-18T03:26:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 shutdown_connections 2020-07-18T03:26:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41940b5eb0 0x7f4194134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f416c050330 0x7f416c0527f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 --2- 172.21.15.47:0/2091645724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41940b5a20 0x7f4194133c80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.373+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 >> 172.21.15.47:0/2091645724 conn(0x7f41940baf50 msgr2=0x7f41940b85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.374+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 shutdown_connections 2020-07-18T03:26:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.374+0000 7f419ba0f700 1 -- 172.21.15.47:0/2091645724 wait complete. 2020-07-18T03:26:50.386 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:26:50.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.503+0000 7f59770ef700 1 Processor -- start 2020-07-18T03:26:50.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.503+0000 7f59770ef700 1 -- start start 2020-07-18T03:26:50.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 0x7f59700b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59700b6300 0x7f59700b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5970147de0 con 0x7f59700ad330 2020-07-18T03:26:50.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700ae000 con 0x7f59700b59b0 2020-07-18T03:26:50.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5970148010 con 0x7f59700b6300 2020-07-18T03:26:50.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59700b6300 0x7f59700b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59700b6300 0x7f59700b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60338/0 (socket says 172.21.15.47:60338) 2020-07-18T03:26:50.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 -- 172.21.15.47:0/4151668087 learned_addr learned my addr 172.21.15.47:0/4151668087 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:50.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f596ffff700 1 --2- 172.21.15.47:0/4151668087 >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 0x7f59700b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 -- 172.21.15.47:0/4151668087 >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 msgr2=0x7f59700b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:50.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 --2- 172.21.15.47:0/4151668087 >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 0x7f59700b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 -- 172.21.15.47:0/4151668087 >> v1:172.21.15.18:6789/0 conn(0x7f59700ad330 legacy=0x7f5970143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 -- 172.21.15.47:0/4151668087 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700ad710 con 0x7f59700b6300 2020-07-18T03:26:50.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f597568c700 1 --2- 172.21.15.47:0/4151668087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59700b6300 0x7f59700b3a10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f5960002f30 tx=0x7f5960006e90).ready entity=mon.1 client_cookie=de43fba2896e9a99 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f596f7fe700 1 -- 172.21.15.47:0/4151668087 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5960009850 con 0x7f59700b6300 2020-07-18T03:26:50.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f596f7fe700 1 -- 172.21.15.47:0/4151668087 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59600099b0 con 0x7f59700b6300 2020-07-18T03:26:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f596f7fe700 1 -- 172.21.15.47:0/4151668087 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5960009d10 con 0x7f59700b6300 2020-07-18T03:26:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 -- 172.21.15.47:0/4151668087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59700b6300 msgr2=0x7f59700b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 --2- 172.21.15.47:0/4151668087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59700b6300 0x7f59700b3a10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f5960002f30 tx=0x7f5960006e90).stop 2020-07-18T03:26:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 -- 172.21.15.47:0/4151668087 shutdown_connections 2020-07-18T03:26:50.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 --2- 172.21.15.47:0/4151668087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59700b6300 0x7f59700b3a10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 --2- 172.21.15.47:0/4151668087 >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 0x7f59700b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.504+0000 7f59770ef700 1 -- 172.21.15.47:0/4151668087 >> 172.21.15.47:0/4151668087 conn(0x7f597005f5a0 msgr2=0x7f597005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.505+0000 7f59770ef700 1 -- 172.21.15.47:0/4151668087 shutdown_connections 2020-07-18T03:26:50.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.505+0000 7f59770ef700 1 -- 172.21.15.47:0/4151668087 wait complete. 2020-07-18T03:26:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.506+0000 7f59770ef700 1 Processor -- start 2020-07-18T03:26:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.506+0000 7f59770ef700 1 -- start start 2020-07-18T03:26:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f59770ef700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 0x7f5970142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f59770ef700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5970144a30 con 0x7f59700ad330 2020-07-18T03:26:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f59770ef700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5970144cd0 con 0x7f5970146a40 2020-07-18T03:26:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f59770ef700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5970142b40 con 0x7f59700b59b0 2020-07-18T03:26:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f597568c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5970146a40 0x7f5970065220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41432/0 (socket says 172.21.15.47:41432) 2020-07-18T03:26:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f597568c700 1 -- 172.21.15.47:0/3081181815 learned_addr learned my addr 172.21.15.47:0/3081181815 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596ffff700 1 --2- 172.21.15.47:0/3081181815 >> v2:172.21.15.18:3300/0 conn(0x7f59700b59b0 0x7f5970142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669114562 0 0) 0x7f5970144cd0 con 0x7f5970146a40 2020-07-18T03:26:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5948003240 con 0x7f5970146a40 2020-07-18T03:26:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769547084 0 0) 0x7f5970144a30 con 0x7f59700ad330 2020-07-18T03:26:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596ffff700 1 -- 172.21.15.47:0/3081181815 >> v1:172.21.15.47:6789/0 conn(0x7f5970146a40 legacy=0x7f5970065220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596ffff700 1 -- 172.21.15.47:0/3081181815 >> v1:172.21.15.18:6789/0 conn(0x7f59700ad330 legacy=0x7f5970145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596ffff700 1 -- 172.21.15.47:0/3081181815 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970142c80 con 0x7f59700b59b0 2020-07-18T03:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.507+0000 7f596ffff700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59700b59b0 0x7f5970142650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f596400c000 tx=0x7f5964006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.508+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596400ca10 con 0x7f59700b59b0 2020-07-18T03:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.508+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596400cb70 con 0x7f59700b59b0 2020-07-18T03:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.508+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5970142f30 con 0x7f59700b59b0 2020-07-18T03:26:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.508+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59640123f0 con 0x7f59700b59b0 2020-07-18T03:26:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.508+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5970065b70 con 0x7f59700b59b0 2020-07-18T03:26:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.509+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5970144870 con 0x7f59700b59b0 2020-07-18T03:26:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.509+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f59640125f0 con 0x7f59700b59b0 2020-07-18T03:26:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.509+0000 7f596cff9700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f59480502c0 0x7f5948052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.509+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5964067e20 con 0x7f59700b59b0 2020-07-18T03:26:50.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.509+0000 7f5974e8b700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f59480502c0 0x7f5948052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.510+0000 7f5974e8b700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f59480502c0 0x7f5948052780 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f59680025b0 tx=0x7f5968007ac0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.513+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f59640404d0 con 0x7f59700b59b0 2020-07-18T03:26:50.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.673+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f597005ff50 con 0x7f59700b59b0 2020-07-18T03:26:50.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.673+0000 7f596cff9700 1 -- 172.21.15.47:0/3081181815 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5964043af0 con 0x7f59700b59b0 2020-07-18T03:26:50.676 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:26:50.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f59480502c0 msgr2=0x7f5948052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f59480502c0 0x7f5948052780 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f59680025b0 tx=0x7f5968007ac0).stop 2020-07-18T03:26:50.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59700b59b0 msgr2=0x7f5970142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59700b59b0 0x7f5970142650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f596400c000 tx=0x7f5964006dd0).stop 2020-07-18T03:26:50.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 shutdown_connections 2020-07-18T03:26:50.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f59480502c0 0x7f5948052780 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 --2- 172.21.15.47:0/3081181815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59700b59b0 0x7f5970142650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 >> 172.21.15.47:0/3081181815 conn(0x7f597005f5a0 msgr2=0x7f59700b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:50.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 shutdown_connections 2020-07-18T03:26:50.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.675+0000 7f59770ef700 1 -- 172.21.15.47:0/3081181815 wait complete. 2020-07-18T03:26:50.687 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:26:50.688 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.802+0000 7f869a8b3700 1 Processor -- start 2020-07-18T03:26:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.802+0000 7f869a8b3700 1 -- start start 2020-07-18T03:26:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.802+0000 7f869a8b3700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f869405f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.802+0000 7f869a8b3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940b7ae0 con 0x7f869414acf0 2020-07-18T03:26:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f869a8b3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869414a620 con 0x7f869414b150 2020-07-18T03:26:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f869a8b3700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940b7d10 con 0x7f8694062d60 2020-07-18T03:26:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8698e50700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f869414b150 0x7f8694146520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41438/0 (socket says 172.21.15.47:41438) 2020-07-18T03:26:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8698e50700 1 -- 172.21.15.47:0/2141112721 learned_addr learned my addr 172.21.15.47:0/2141112721 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8693fff700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f869405f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635007860 0 0) 0x7f869414a620 con 0x7f869414b150 2020-07-18T03:26:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8680003240 con 0x7f869414b150 2020-07-18T03:26:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393014274 0 0) 0x7f86940b7ae0 con 0x7f869414acf0 2020-07-18T03:26:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f869414a620 con 0x7f869414acf0 2020-07-18T03:26:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3852948249 0 0) 0x7f8680003240 con 0x7f869414b150 2020-07-18T03:26:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 msgr2=0x7f869405f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f869405f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 >> v1:172.21.15.18:6789/0 conn(0x7f869414acf0 legacy=0x7f86940b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869414a850 con 0x7f869414b150 2020-07-18T03:26:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f868c001a50 con 0x7f869414b150 2020-07-18T03:26:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8693fff700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f869405f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 >> v1:172.21.15.47:6789/0 conn(0x7f869414b150 legacy=0x7f8694146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f86800038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8680003db0 0x7f86800044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86800049d0 0x7f8680004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940b7d10 con 0x7f8680003db0 2020-07-18T03:26:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680005340 con 0x7f86800049d0 2020-07-18T03:26:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868000b760 con 0x7f8694062d60 2020-07-18T03:26:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.803+0000 7f8698e50700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86800049d0 0x7f8680004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f86937fe700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8680003db0 0x7f86800044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8693fff700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f86800038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 -- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 msgr2=0x7f86800038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8694062d60 0x7f86800038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 -- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8680003db0 msgr2=0x7f86800044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8680003db0 0x7f86800044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 -- 172.21.15.47:0/2141112721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680003240 con 0x7f86800049d0 2020-07-18T03:26:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86800049d0 0x7f8680004e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f868c001820 tx=0x7f86940b6c10).ready entity=mon.1 client_cookie=f0aa86e3d45572e8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868c005b80 con 0x7f86800049d0 2020-07-18T03:26:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c005ce0 con 0x7f86800049d0 2020-07-18T03:26:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8692ffd700 1 -- 172.21.15.47:0/2141112721 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868c0072c0 con 0x7f86800049d0 2020-07-18T03:26:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f869a8b3700 1 -- 172.21.15.47:0/2141112721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86800049d0 msgr2=0x7f8680004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f869a8b3700 1 --2- 172.21.15.47:0/2141112721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86800049d0 0x7f8680004e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f868c001820 tx=0x7f86940b6c10).stop 2020-07-18T03:26:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f8698e50700 1 -- 172.21.15.47:0/2141112721 reap_dead start 2020-07-18T03:26:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f869a8b3700 1 -- 172.21.15.47:0/2141112721 shutdown_connections 2020-07-18T03:26:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f869a8b3700 1 -- 172.21.15.47:0/2141112721 >> 172.21.15.47:0/2141112721 conn(0x7f869405cb10 msgr2=0x7f869406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f869a8b3700 1 -- 172.21.15.47:0/2141112721 shutdown_connections 2020-07-18T03:26:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.804+0000 7f869a8b3700 1 -- 172.21.15.47:0/2141112721 wait complete. 2020-07-18T03:26:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f869a8b3700 1 Processor -- start 2020-07-18T03:26:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f869a8b3700 1 -- start start 2020-07-18T03:26:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f869a8b3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8694141c30 0x7f86941420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f869a8b3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86941425b0 con 0x7f8694148cd0 2020-07-18T03:26:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f869a8b3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86941583c0 con 0x7f86941490b0 2020-07-18T03:26:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f869a8b3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8694158660 con 0x7f8694141c30 2020-07-18T03:26:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f8698e50700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8694141c30 0x7f86941420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f86937fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f86941490b0 0x7f8694144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41448/0 (socket says 172.21.15.47:41448) 2020-07-18T03:26:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.806+0000 7f86937fe700 1 -- 172.21.15.47:0/1779436684 learned_addr learned my addr 172.21.15.47:0/1779436684 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8698e50700 1 -- 172.21.15.47:0/1779436684 >> v1:172.21.15.47:6789/0 conn(0x7f86941490b0 legacy=0x7f8694144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8698e50700 1 -- 172.21.15.47:0/1779436684 >> v1:172.21.15.18:6789/0 conn(0x7f8694148cd0 legacy=0x7f8694158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8698e50700 1 -- 172.21.15.47:0/1779436684 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694144f80 con 0x7f8694141c30 2020-07-18T03:26:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8698e50700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8694141c30 0x7f86941420c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f8694143be0 tx=0x7f868c013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868c005b80 con 0x7f8694141c30 2020-07-18T03:26:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00b070 con 0x7f8694141c30 2020-07-18T03:26:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868c0125a0 con 0x7f8694141c30 2020-07-18T03:26:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8694145170 con 0x7f8694141c30 2020-07-18T03:26:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.807+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8694065d50 con 0x7f8694141c30 2020-07-18T03:26:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.808+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8694145300 con 0x7f8694141c30 2020-07-18T03:26:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.808+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f868c01c070 con 0x7f8694141c30 2020-07-18T03:26:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.808+0000 7f8673fff700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f866c04ff70 0x7f866c052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.808+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f868c0686b0 con 0x7f8694141c30 2020-07-18T03:26:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.809+0000 7f8693fff700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f866c04ff70 0x7f866c052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.809+0000 7f8693fff700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f866c04ff70 0x7f866c052430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8694065a20 tx=0x7f8684008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:50.812+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f868c040dc0 con 0x7f8694141c30 2020-07-18T03:26:51.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.022+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8694057450 con 0x7f8694141c30 2020-07-18T03:26:51.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.022+0000 7f8673fff700 1 -- 172.21.15.47:0/1779436684 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f868c0443e0 con 0x7f8694141c30 2020-07-18T03:26:51.025 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:51.025 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:26:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f866c04ff70 msgr2=0x7f866c052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f866c04ff70 0x7f866c052430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8694065a20 tx=0x7f8684008040).stop 2020-07-18T03:26:51.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8694141c30 msgr2=0x7f86941420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:51.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8694141c30 0x7f86941420c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f8694143be0 tx=0x7f868c013000).stop 2020-07-18T03:26:51.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 shutdown_connections 2020-07-18T03:26:51.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8694141c30 0x7f86941420c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 --2- 172.21.15.47:0/1779436684 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f866c04ff70 0x7f866c052430 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 >> 172.21.15.47:0/1779436684 conn(0x7f869405cb10 msgr2=0x7f86940b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 shutdown_connections 2020-07-18T03:26:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:51.024+0000 7f869a8b3700 1 -- 172.21.15.47:0/1779436684 wait complete. 2020-07-18T03:26:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:26:51.238 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:26:40.770533+0000', 'modified': '2020-07-18T03:26:41.771684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5101}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5101': {'gid': 5101, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/3109073551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3109073551}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3109073551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:26:51.239 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:26:51.239 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:26:51.376 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5101,"want_state":"up:active","state":"up:active","rank_uptime":10.595317912000001,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":17.823265478} 2020-07-18T03:26:51.377 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5101, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 10.595317912, "state": "up:active", "uptime": 17.823265478, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:26:51.377 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:26:51.378 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:26:51.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.379 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:26:51.394 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:51.394 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:51.395 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:51.395 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:51.395 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:51.395 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:51.395 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:51.396 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:51.396 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:26:51.396 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:51.396 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:26:51.396 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42780sec preferred_lft 42780sec 2020-07-18T03:26:51.397 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:26:51.397 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:51.397 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:51.397 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:51.397 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:26:51.398 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:51.398 INFO:teuthology.orchestra.run.smithi047.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:51.398 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0e:86:7a:35:9b:4f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:26:51.398 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:26:51.398 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:51.398 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50fb:8cff:fe53:9ead/64 scope link 2020-07-18T03:26:51.399 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:51.399 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:26:51.399 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0e:86:7a:35:9b:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:26:51.399 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::c86:7aff:fe35:9b4f/64 scope link 2020-07-18T03:26:51.399 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:51.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.402 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:26:51.464 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:26:51.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.466 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:26:51.521 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:26:51.522 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:26:51.522 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:26:51.522 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:26:51.522 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:26:51.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.523 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:26:51.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.595 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:26:51.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.670 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:26:51.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.743 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:26:51.797 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-07-18T03:26:51.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.798 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:26:51.851 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:26:51.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.853 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:26:51.907 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:26:51.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.908 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:26:51.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:51.982 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1' 2020-07-18T03:26:52.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.054 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:26:52.131 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:52.131 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:52.132 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:52.132 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.132 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:52.133 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.133 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:52.133 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 02:f9:46:d9:4d:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:26:52.134 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:26:52.134 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.134 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::f9:46ff:fed9:4d6a/64 scope link 2020-07-18T03:26:52.135 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.141 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:26:52.221 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:26:52.221 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:26:52.222 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:26:52.222 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.222 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:26:52.223 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.223 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:26:52.223 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 02:f9:46:d9:4d:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:26:52.223 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:26:52.224 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.224 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::f9:46ff:fed9:4d6a/64 scope link 2020-07-18T03:26:52.224 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:26:52.233 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-07-18T03:26:52.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.235 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1' 2020-07-18T03:26:52.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.313 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:26:52.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.402 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:26:52.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.466 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:26:52.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.522 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:26:52.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.589 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 up' 2020-07-18T03:26:52.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:26:52.629 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.1 master ceph-brx' 2020-07-18T03:26:52.730 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:26:52.730 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:26:52.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:52.731 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:52.746 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:52.829 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:52.831 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:52.901 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:52.902 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:52.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:52.904 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:52.958 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:26:52.958 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:26:52.959 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:53.000 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:26:53.032 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:26:53.032 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:26:53.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:26:53.096+0000 7fef2eefd1c0 -1 init, newargv = 0x555715b24c50 newargc=15 2020-07-18T03:26:53.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[34555]: starting ceph client 2020-07-18T03:26:53.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[34555]: starting fuse 2020-07-18T03:26:53.104 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:26:53.106 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:26:53.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:53.107 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:26:53.160 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:26:53.160 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:26:53.161 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:26:53.161 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:26:53.162 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:26:53.162 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:26:53.163 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:26:53.163 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:26:53.164 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:53.164 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:26:53.164 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:26:53.165 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:26:53.165 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:26:53.166 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:53.166 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:26:53.167 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:26:53.167 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:26:53.167 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:26:53.168 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:53.168 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:26:53.169 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:26:53.169 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:26:53.170 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:26:53.171 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:26:53.171 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:26:53.171 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:26:53.172 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:26:53.172 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:26:53.173 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:26:53.173 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:26:53.291 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.34555.asok 2020-07-18T03:26:53.471 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34555.asok 2020-07-18T03:26:53.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:53.472 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34555.asok status 2020-07-18T03:26:53.626 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:26:53.626 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:26:53.626 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:26:53.627 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:26:53.627 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:26:53.627 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:26:53.627 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:26:53.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "34555", 2020-07-18T03:26:53.628 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:26:53.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:53.628 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:26:53.628 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:26:53.628 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5416, 2020-07-18T03:26:53.629 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:26:53.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:26:53.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:26:53.629 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5416 2020-07-18T03:26:53.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:53.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:53.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:53.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:26:53.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3207407312 2020-07-18T03:26:53.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:26:53.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:53.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:26:53.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:26:53.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:26:53.632 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3207407312 2020-07-18T03:26:53.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:26:53.632 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5416 192.168.0.2:0/3207407312", 2020-07-18T03:26:53.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/3207407312", 2020-07-18T03:26:53.632 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:26:53.633 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 11, 2020-07-18T03:26:53.633 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 33, 2020-07-18T03:26:53.633 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:26:53.633 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:26:53.633 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:26:53.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:53.638 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:26:53.639 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:53.655 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:26:53.656 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:26:53.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:26:53.657 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:26:53.729 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2020-07-18T03:26:53.898 INFO:teuthology.orchestra.run.smithi047.stdout:{"mds_reconnect_timeout":"45.000000"} 2020-07-18T03:26:53.899 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2020-07-18T03:26:53.899 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2020-07-18T03:26:54.029 INFO:teuthology.orchestra.run.smithi047.stdout:{"ms_max_backoff":"15.000000"} 2020-07-18T03:26:54.029 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2020-07-18T03:26:54.030 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.144+0000 7f48609f8700 1 Processor -- start 2020-07-18T03:26:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.144+0000 7f48609f8700 1 -- start start 2020-07-18T03:26:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.145+0000 7f48609f8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f485c147a00 0x7f485c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.145+0000 7f48609f8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f485c1433e0 0x7f485c143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.145+0000 7f48609f8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c147330 con 0x7f485c0ad100 2020-07-18T03:26:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.145+0000 7f48609f8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c0ae530 con 0x7f485c1433e0 2020-07-18T03:26:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.145+0000 7f48609f8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0b57e0 con 0x7f485c147a00 2020-07-18T03:26:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.145+0000 7f485ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f485c1433e0 0x7f485c143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f485c0ad100 0x7f485c0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41470/0 (socket says 172.21.15.47:41470) 2020-07-18T03:26:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485a59c700 1 -- 172.21.15.47:0/1948018793 learned_addr learned my addr 172.21.15.47:0/1948018793 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f4859d9b700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f485c147a00 0x7f485c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485ad9d700 1 -- 172.21.15.47:0/1948018793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f485c147a00 msgr2=0x7f485c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485ad9d700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f485c147a00 0x7f485c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485ad9d700 1 -- 172.21.15.47:0/1948018793 >> v1:172.21.15.47:6789/0 conn(0x7f485c0ad100 legacy=0x7f485c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485959a700 1 -- 172.21.15.47:0/1948018793 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875715883 0 0) 0x7f485c147330 con 0x7f485c0ad100 2020-07-18T03:26:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485ad9d700 1 -- 172.21.15.47:0/1948018793 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c0b5a80 con 0x7f485c1433e0 2020-07-18T03:26:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f4859d9b700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f485c147a00 0x7f485c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485ad9d700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f485c1433e0 0x7f485c143850 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f4854002f30 tx=0x7f4854006dd0).ready entity=mon.1 client_cookie=a973a3a7a82ebfec server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485959a700 1 -- 172.21.15.47:0/1948018793 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48540097f0 con 0x7f485c1433e0 2020-07-18T03:26:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485959a700 1 -- 172.21.15.47:0/1948018793 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854009950 con 0x7f485c1433e0 2020-07-18T03:26:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f485959a700 1 -- 172.21.15.47:0/1948018793 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4854009cb0 con 0x7f485c1433e0 2020-07-18T03:26:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 -- 172.21.15.47:0/1948018793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f485c1433e0 msgr2=0x7f485c143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f485c1433e0 0x7f485c143850 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f4854002f30 tx=0x7f4854006dd0).stop 2020-07-18T03:26:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 -- 172.21.15.47:0/1948018793 shutdown_connections 2020-07-18T03:26:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f485c1433e0 0x7f485c143850 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 --2- 172.21.15.47:0/1948018793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f485c147a00 0x7f485c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 -- 172.21.15.47:0/1948018793 >> 172.21.15.47:0/1948018793 conn(0x7f485c0b9340 msgr2=0x7f485c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 -- 172.21.15.47:0/1948018793 shutdown_connections 2020-07-18T03:26:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.146+0000 7f48609f8700 1 -- 172.21.15.47:0/1948018793 wait complete. 2020-07-18T03:26:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.148+0000 7f48609f8700 1 Processor -- start 2020-07-18T03:26:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.148+0000 7f48609f8700 1 -- start start 2020-07-18T03:26:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.148+0000 7f48609f8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f485c153fc0 0x7f485c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.148+0000 7f48609f8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c156fa0 con 0x7f485c0ad100 2020-07-18T03:26:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.148+0000 7f48609f8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c157240 con 0x7f485c147a00 2020-07-18T03:26:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.148+0000 7f48609f8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c156d80 con 0x7f485c153fc0 2020-07-18T03:26:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f4859d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f485c147a00 0x7f485c1528a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41476/0 (socket says 172.21.15.47:41476) 2020-07-18T03:26:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f4859d9b700 1 -- 172.21.15.47:0/957618701 learned_addr learned my addr 172.21.15.47:0/957618701 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f485ad9d700 1 --2- 172.21.15.47:0/957618701 >> v2:172.21.15.18:3300/0 conn(0x7f485c153fc0 0x7f485c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285153531 0 0) 0x7f485c157240 con 0x7f485c147a00 2020-07-18T03:26:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4838003240 con 0x7f485c147a00 2020-07-18T03:26:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1591975644 0 0) 0x7f4838003240 con 0x7f485c147a00 2020-07-18T03:26:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f485c157240 con 0x7f485c147a00 2020-07-18T03:26:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f484c0096f0 con 0x7f485c147a00 2020-07-18T03:26:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 >> v2:172.21.15.18:3300/0 conn(0x7f485c153fc0 msgr2=0x7f485c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 --2- 172.21.15.47:0/957618701 >> v2:172.21.15.18:3300/0 conn(0x7f485c153fc0 0x7f485c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 >> v1:172.21.15.47:6789/0 conn(0x7f485c147a00 legacy=0x7f485c1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 >> v1:172.21.15.18:6789/0 conn(0x7f485c0ad100 legacy=0x7f485c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4838003800 0x7f4838003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48380044e0 0x7f4838004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4838004e10 0x7f483800b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c156d80 con 0x7f48380044e0 2020-07-18T03:26:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838005380 con 0x7f4838003800 2020-07-18T03:26:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483800bc20 con 0x7f4838004e10 2020-07-18T03:26:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f485a59c700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4838003800 0x7f4838003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f4859d9b700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48380044e0 0x7f4838004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.149+0000 7f485ad9d700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4838004e10 0x7f483800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485a59c700 1 -- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4838004e10 msgr2=0x7f483800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485a59c700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4838004e10 0x7f483800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485a59c700 1 -- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48380044e0 msgr2=0x7f4838004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485a59c700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48380044e0 0x7f4838004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485a59c700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c1574e0 con 0x7f4838003800 2020-07-18T03:26:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485ad9d700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4838004e10 0x7f483800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485ad9d700 1 -- 172.21.15.47:0/957618701 reap_dead start 2020-07-18T03:26:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c062110 con 0x7f4838003800 2020-07-18T03:26:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c062300 con 0x7f4838003800 2020-07-18T03:26:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f485a59c700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4838003800 0x7f4838003c20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f485c0b3370 tx=0x7f4844007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48440072e0 con 0x7f4838003800 2020-07-18T03:26:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4844007440 con 0x7f4838003800 2020-07-18T03:26:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.150+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4844005780 con 0x7f4838003800 2020-07-18T03:26:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.151+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c112600 con 0x7f4838003800 2020-07-18T03:26:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.151+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f484401a060 con 0x7f4838003800 2020-07-18T03:26:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.151+0000 7f48527fc700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4838053e10 0x7f48380562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.151+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4844067300 con 0x7f4838003800 2020-07-18T03:26:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.151+0000 7f4859d9b700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4838053e10 0x7f48380562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.152+0000 7f4859d9b700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4838053e10 0x7f48380562d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f484c00f000 tx=0x7f484c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.155+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f484403fa90 con 0x7f4838003800 2020-07-18T03:26:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.362+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f485c057450 con 0x7f4838003800 2020-07-18T03:26:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.363+0000 7f48527fc700 1 -- 172.21.15.47:0/957618701 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f4844018070 con 0x7f4838003800 2020-07-18T03:26:54.365 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:54.365 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:26:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4838053e10 msgr2=0x7f48380562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4838053e10 0x7f48380562d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f484c00f000 tx=0x7f484c008040).stop 2020-07-18T03:26:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4838003800 msgr2=0x7f4838003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4838003800 0x7f4838003c20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f485c0b3370 tx=0x7f4844007a60).stop 2020-07-18T03:26:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 shutdown_connections 2020-07-18T03:26:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4838053e10 0x7f48380562d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 --2- 172.21.15.47:0/957618701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4838003800 0x7f4838003c20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 >> 172.21.15.47:0/957618701 conn(0x7f485c0b9340 msgr2=0x7f485c0b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 shutdown_connections 2020-07-18T03:26:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.364+0000 7f48609f8700 1 -- 172.21.15.47:0/957618701 wait complete. 2020-07-18T03:26:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:26:54.377 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_defer_session_stale false 2020-07-18T03:26:54.509 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_defer_session_stale = 'false' (not observed, change may require restart) "} 2020-07-18T03:26:54.509 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_defer_session_stale = 'false' (not observed, change may require restart) " } 2020-07-18T03:26:54.510 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:26:54.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.626+0000 7f4a3e68e700 1 Processor -- start 2020-07-18T03:26:54.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.626+0000 7f4a3e68e700 1 -- start start 2020-07-18T03:26:54.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3e68e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3e68e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3e68e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a3805f5a0 0x7f4a3805fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3e68e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380ae530 con 0x7f4a380ad100 2020-07-18T03:26:54.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3e68e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a38147330 con 0x7f4a3805f5a0 2020-07-18T03:26:54.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3e68e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b57e0 con 0x7f4a38147a00 2020-07-18T03:26:54.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a3805f5a0 0x7f4a3805fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a3805f5a0 0x7f4a3805fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60404/0 (socket says 172.21.15.47:60404) 2020-07-18T03:26:54.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 -- 172.21.15.47:0/2221630462 learned_addr learned my addr 172.21.15.47:0/2221630462 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:26:54.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a37fff700 1 --2- 172.21.15.47:0/2221630462 >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a377fe700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 -- 172.21.15.47:0/2221630462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 msgr2=0x7f4a38147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:54.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 -- 172.21.15.47:0/2221630462 >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 msgr2=0x7f4a380b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:26:54.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 --2- 172.21.15.47:0/2221630462 >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 -- 172.21.15.47:0/2221630462 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380b5a80 con 0x7f4a3805f5a0 2020-07-18T03:26:54.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a37fff700 1 --2- 172.21.15.47:0/2221630462 >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:54.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a377fe700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:26:54.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.627+0000 7f4a3cc2b700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3805f5a0 0x7f4a3805fa10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f4a300032b0 tx=0x7f4a30009270).ready entity=mon.1 client_cookie=6aa4216dd08e67c8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:54.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a36ffd700 1 -- 172.21.15.47:0/2221630462 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3000bc00 con 0x7f4a3805f5a0 2020-07-18T03:26:54.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a36ffd700 1 -- 172.21.15.47:0/2221630462 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000bd60 con 0x7f4a3805f5a0 2020-07-18T03:26:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a36ffd700 1 -- 172.21.15.47:0/2221630462 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a30011400 con 0x7f4a3805f5a0 2020-07-18T03:26:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 -- 172.21.15.47:0/2221630462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3805f5a0 msgr2=0x7f4a3805fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3805f5a0 0x7f4a3805fa10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f4a300032b0 tx=0x7f4a30009270).stop 2020-07-18T03:26:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 -- 172.21.15.47:0/2221630462 shutdown_connections 2020-07-18T03:26:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3805f5a0 0x7f4a3805fa10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/2221630462 >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/2221630462 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 -- 172.21.15.47:0/2221630462 >> 172.21.15.47:0/2221630462 conn(0x7f4a380b9340 msgr2=0x7f4a380b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 -- 172.21.15.47:0/2221630462 shutdown_connections 2020-07-18T03:26:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.628+0000 7f4a3e68e700 1 -- 172.21.15.47:0/2221630462 wait complete. 2020-07-18T03:26:54.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.629+0000 7f4a3e68e700 1 Processor -- start 2020-07-18T03:26:54.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.629+0000 7f4a3e68e700 1 -- start start 2020-07-18T03:26:54.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3e68e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3e68e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3e68e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3813c940 con 0x7f4a38149200 2020-07-18T03:26:54.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3e68e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3813cbe0 con 0x7f4a380ad100 2020-07-18T03:26:54.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3e68e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b2e20 con 0x7f4a38147a00 2020-07-18T03:26:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3cc2b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4a38149200 0x7f4a3813bae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41498/0 (socket says 172.21.15.47:41498) 2020-07-18T03:26:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a3cc2b700 1 -- 172.21.15.47:0/3706883150 learned_addr learned my addr 172.21.15.47:0/3706883150 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:26:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a37fff700 1 --2- 172.21.15.47:0/3706883150 >> v2:172.21.15.18:3300/0 conn(0x7f4a380ad100 0x7f4a380b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a377fe700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420977975 0 0) 0x7f4a3813c940 con 0x7f4a38149200 2020-07-18T03:26:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a10003240 con 0x7f4a38149200 2020-07-18T03:26:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a37fff700 1 -- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 msgr2=0x7f4a38149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a37fff700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.630+0000 7f4a37fff700 1 -- 172.21.15.47:0/3706883150 >> v1:172.21.15.47:6789/0 conn(0x7f4a38149200 legacy=0x7f4a3813bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a37fff700 1 -- 172.21.15.47:0/3706883150 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3813e810 con 0x7f4a380ad100 2020-07-18T03:26:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a377fe700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:26:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a37fff700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a380ad100 0x7f4a380b2930 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f4a380b8520 tx=0x7f4a28008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a280118a0 con 0x7f4a380ad100 2020-07-18T03:26:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3814a010 con 0x7f4a380ad100 2020-07-18T03:26:54.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28011a00 con 0x7f4a380ad100 2020-07-18T03:26:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a28011d60 con 0x7f4a380ad100 2020-07-18T03:26:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.631+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3813c3b0 con 0x7f4a380ad100 2020-07-18T03:26:54.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.632+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a38147e80 con 0x7f4a380ad100 2020-07-18T03:26:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.632+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4a2801c460 con 0x7f4a380ad100 2020-07-18T03:26:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.632+0000 7f4a17fff700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4a100503b0 0x7f4a10052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:26:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.633+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a28068100 con 0x7f4a380ad100 2020-07-18T03:26:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.633+0000 7f4a377fe700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4a100503b0 0x7f4a10052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:26:54.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.633+0000 7f4a377fe700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4a100503b0 0x7f4a10052870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f4a2c00c000 tx=0x7f4a2c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:26:54.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.637+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4a28040710 con 0x7f4a380ad100 2020-07-18T03:26:54.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.844+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a380b7150 con 0x7f4a380ad100 2020-07-18T03:26:54.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a17fff700 1 -- 172.21.15.47:0/3706883150 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f4a28043d30 con 0x7f4a380ad100 2020-07-18T03:26:54.847 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:26:54.848 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:26:54.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4a100503b0 msgr2=0x7f4a10052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4a100503b0 0x7f4a10052870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f4a2c00c000 tx=0x7f4a2c008040).stop 2020-07-18T03:26:54.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a380ad100 msgr2=0x7f4a380b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:26:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a380ad100 0x7f4a380b2930 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f4a380b8520 tx=0x7f4a28008040).stop 2020-07-18T03:26:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 shutdown_connections 2020-07-18T03:26:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4a100503b0 0x7f4a10052870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a380ad100 0x7f4a380b2930 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 --2- 172.21.15.47:0/3706883150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4a38147a00 0x7f4a38149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:26:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 >> 172.21.15.47:0/3706883150 conn(0x7f4a380b9340 msgr2=0x7f4a380b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:26:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 shutdown_connections 2020-07-18T03:26:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:26:54.846+0000 7f4a3e68e700 1 -- 172.21.15.47:0/3706883150 wait complete. 2020-07-18T03:26:54.852 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:26:54.858 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:26:54.859 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:26:54.860 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:26:54.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[33695]: fuse finished with error 0 and tester_r 0 2020-07-18T03:26:54.901 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:27:01.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:01.005 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:27:01.024 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:27:01.025 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:27:01.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:01.026 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:27:01.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:01.110 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:27:01.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:01.174 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:27:01.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:01.220 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:27:01.235 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:27:01.235 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:27:01.236 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:27:01.236 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:01.236 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:27:01.236 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:01.236 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:27:01.237 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.237 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:27:01.237 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.237 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:27:01.237 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42770sec preferred_lft 42770sec 2020-07-18T03:27:01.238 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:27:01.238 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:01.238 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:27:01.238 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.238 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:27:01.239 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.239 INFO:teuthology.orchestra.run.smithi047.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:27:01.239 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether d6:fd:2f:14:3e:39 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.239 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:27:01.240 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:01.240 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50fb:8cff:fe53:9ead/64 scope link 2020-07-18T03:27:01.240 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:01.240 INFO:teuthology.orchestra.run.smithi047.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:27:01.240 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether d6:fd:2f:14:3e:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:27:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::d4fd:2fff:fe14:3e39/64 scope link 2020-07-18T03:27:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:01.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:01.242 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:27:01.318 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.318 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:27:01.318 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.318 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.319 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.319 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.319 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.319 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.319 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.320 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.320 INFO:teuthology.orchestra.run.smithi047.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.320 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether d6:fd:2f:14:3e:39 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:01.320 INFO:teuthology.orchestra.run.smithi047.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:27:01.320 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether d6:fd:2f:14:3e:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:27:01.321 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:27:01.322 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:27:01.322 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:27:01.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[34555]: fuse finished with error 0 and tester_r 0 2020-07-18T03:27:01.409 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:27:19.418 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:27:19.423 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:27:31.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:31.934 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:27:31.954 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:27:31.955 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:27:31.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:31.956 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:27:32.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.038 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.1' 2020-07-18T03:27:32.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.116 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:27:32.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.166 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:27:32.181 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:27:32.182 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:27:32.182 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:27:32.182 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:32.182 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:27:32.183 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:32.183 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:27:32.183 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.183 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:27:32.183 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.184 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:27:32.184 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42739sec preferred_lft 42739sec 2020-07-18T03:27:32.184 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:27:32.184 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:32.184 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:27:32.185 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.185 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:27:32.185 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.185 INFO:teuthology.orchestra.run.smithi047.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state DOWN group default qlen 1000 2020-07-18T03:27:32.185 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.186 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:27:32.186 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:32.186 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50fb:8cff:fe53:9ead/64 scope link 2020-07-18T03:27:32.186 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:27:32.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.187 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:27:32.254 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:32.254 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:27:32.255 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:32.255 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.255 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:27:32.255 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.255 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:32.256 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.256 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:32.256 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.256 INFO:teuthology.orchestra.run.smithi047.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:27:32.256 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:27:32.257 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:27:32.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.258 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:27:32.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.330 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:27:32.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.419 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:27:32.435 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:27:32.435 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:27:32.435 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:27:32.436 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:27:32.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.437 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:27:32.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.519 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:27:32.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:27:32.599 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:27:32.684 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:27:32.685 INFO:teuthology.orchestra.run.smithi047:> import cephfs as libcephfs 2020-07-18T03:27:32.685 INFO:teuthology.orchestra.run.smithi047:> cephfs = libcephfs.LibCephFS(conffile='"'"''"'"') 2020-07-18T03:27:32.686 INFO:teuthology.orchestra.run.smithi047:> cephfs.mount() 2020-07-18T03:27:32.686 INFO:teuthology.orchestra.run.smithi047:> client_id = cephfs.get_instance_id() 2020-07-18T03:27:32.686 INFO:teuthology.orchestra.run.smithi047:> cephfs.abort_conn() 2020-07-18T03:27:32.687 INFO:teuthology.orchestra.run.smithi047:> print(client_id) 2020-07-18T03:27:32.687 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:27:32.834 INFO:teuthology.orchestra.run.smithi047.stdout:5211 2020-07-18T03:27:32.993 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:27:33.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.146+0000 7f7a6efba700 1 Processor -- start 2020-07-18T03:27:33.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.146+0000 7f7a6efba700 1 -- start start 2020-07-18T03:27:33.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6efba700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a680b5370 0x7f7a6805fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:27:33.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6efba700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a680b4620 0x7f7a681428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:27:33.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6efba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b3ac0 con 0x7f7a680b41c0 2020-07-18T03:27:33.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6efba700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680b5a10 con 0x7f7a680b4620 2020-07-18T03:27:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6efba700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b0040 con 0x7f7a680b5370 2020-07-18T03:27:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6d557700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a680b4620 0x7f7a681428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:27:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a67fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a680b41c0 0x7f7a680ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41526/0 (socket says 172.21.15.47:41526) 2020-07-18T03:27:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a67fff700 1 -- 172.21.15.47:0/3421567115 learned_addr learned my addr 172.21.15.47:0/3421567115 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:27:33.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6cd56700 1 --2- 172.21.15.47:0/3421567115 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a680b5370 0x7f7a6805fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:27:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6d557700 1 -- 172.21.15.47:0/3421567115 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a680b5370 msgr2=0x7f7a6805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:27:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6d557700 1 --2- 172.21.15.47:0/3421567115 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a680b5370 0x7f7a6805fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6d557700 1 -- 172.21.15.47:0/3421567115 >> v1:172.21.15.47:6789/0 conn(0x7f7a680b41c0 legacy=0x7f7a680ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:27:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a677fe700 1 -- 172.21.15.47:0/3421567115 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405549143 0 0) 0x7f7a680b3ac0 con 0x7f7a680b41c0 2020-07-18T03:27:33.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6d557700 1 -- 172.21.15.47:0/3421567115 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680b3d20 con 0x7f7a680b4620 2020-07-18T03:27:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6d557700 1 --2- 172.21.15.47:0/3421567115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a680b4620 0x7f7a681428f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f7a600032b0 tx=0x7f7a600091d0).ready entity=mon.1 client_cookie=e48fff0d41271d57 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:27:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a677fe700 1 -- 172.21.15.47:0/3421567115 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6000b9a0 con 0x7f7a680b4620 2020-07-18T03:27:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a677fe700 1 -- 172.21.15.47:0/3421567115 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6000bb00 con 0x7f7a680b4620 2020-07-18T03:27:33.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a677fe700 1 -- 172.21.15.47:0/3421567115 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6000bc70 con 0x7f7a680b4620 2020-07-18T03:27:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.147+0000 7f7a6efba700 1 -- 172.21.15.47:0/3421567115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a680b4620 msgr2=0x7f7a681428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:27:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 --2- 172.21.15.47:0/3421567115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a680b4620 0x7f7a681428f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f7a600032b0 tx=0x7f7a600091d0).stop 2020-07-18T03:27:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 -- 172.21.15.47:0/3421567115 shutdown_connections 2020-07-18T03:27:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 --2- 172.21.15.47:0/3421567115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a680b4620 0x7f7a681428f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 --2- 172.21.15.47:0/3421567115 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a680b5370 0x7f7a6805fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 -- 172.21.15.47:0/3421567115 >> 172.21.15.47:0/3421567115 conn(0x7f7a680b9340 msgr2=0x7f7a680b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:27:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 -- 172.21.15.47:0/3421567115 shutdown_connections 2020-07-18T03:27:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.148+0000 7f7a6efba700 1 -- 172.21.15.47:0/3421567115 wait complete. 2020-07-18T03:27:33.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.149+0000 7f7a6efba700 1 Processor -- start 2020-07-18T03:27:33.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.149+0000 7f7a6efba700 1 -- start start 2020-07-18T03:27:33.165 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6efba700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7a680b41c0 0x7f7a680b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:27:33.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6efba700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a68155e20 0x7f7a681562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:27:33.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6efba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6814a7d0 con 0x7f7a680b5370 2020-07-18T03:27:33.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6efba700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a68149df0 con 0x7f7a680b41c0 2020-07-18T03:27:33.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6efba700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6814a090 con 0x7f7a68155e20 2020-07-18T03:27:33.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a67fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a680b5370 0x7f7a68143e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41534/0 (socket says 172.21.15.47:41534) 2020-07-18T03:27:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a67fff700 1 -- 172.21.15.47:0/2307877786 learned_addr learned my addr 172.21.15.47:0/2307877786 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:27:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6cd56700 1 --2- 172.21.15.47:0/2307877786 >> v2:172.21.15.18:3300/0 conn(0x7f7a680b41c0 0x7f7a680b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:27:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a6d557700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a68155e20 0x7f7a681562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:27:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222633941 0 0) 0x7f7a6814a7d0 con 0x7f7a680b5370 2020-07-18T03:27:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.150+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a40003240 con 0x7f7a680b5370 2020-07-18T03:27:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1607260862 0 0) 0x7f7a40003240 con 0x7f7a680b5370 2020-07-18T03:27:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6cd56700 1 -- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a68155e20 msgr2=0x7f7a681562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:27:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6cd56700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a68155e20 0x7f7a681562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6cd56700 1 -- 172.21.15.47:0/2307877786 >> v1:172.21.15.47:6789/0 conn(0x7f7a680b5370 legacy=0x7f7a68143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:27:33.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6cd56700 1 -- 172.21.15.47:0/2307877786 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6814a330 con 0x7f7a680b41c0 2020-07-18T03:27:33.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6cd56700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680b41c0 0x7f7a680b1b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f7a5c001a00 tx=0x7f7a58006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:27:33.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c001860 con 0x7f7a680b41c0 2020-07-18T03:27:33.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5800be50 con 0x7f7a680b41c0 2020-07-18T03:27:33.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5800c990 con 0x7f7a680b41c0 2020-07-18T03:27:33.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68141ff0 con 0x7f7a680b41c0 2020-07-18T03:27:33.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.151+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a681567c0 con 0x7f7a680b41c0 2020-07-18T03:27:33.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.152+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a680ba190 con 0x7f7a680b41c0 2020-07-18T03:27:33.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.152+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7a5800cb40 con 0x7f7a680b41c0 2020-07-18T03:27:33.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.153+0000 7f7a64ff9700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a40050330 0x7f7a400527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:27:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.153+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a58067ab0 con 0x7f7a680b41c0 2020-07-18T03:27:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.153+0000 7f7a67fff700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a40050330 0x7f7a400527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:27:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.153+0000 7f7a67fff700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a40050330 0x7f7a400527f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f7a5c0019c0 tx=0x7f7a5c0076a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:27:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.157+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7a580401e0 con 0x7f7a680b41c0 2020-07-18T03:27:33.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.364+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a680b0aa0 con 0x7f7a680b41c0 2020-07-18T03:27:33.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.365+0000 7f7a64ff9700 1 -- 172.21.15.47:0/2307877786 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f7a58043800 con 0x7f7a680b41c0 2020-07-18T03:27:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:27:33.368 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:27:33.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a40050330 msgr2=0x7f7a400527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:27:33.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a40050330 0x7f7a400527f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f7a5c0019c0 tx=0x7f7a5c0076a0).stop 2020-07-18T03:27:33.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680b41c0 msgr2=0x7f7a680b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:27:33.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680b41c0 0x7f7a680b1b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f7a5c001a00 tx=0x7f7a58006dd0).stop 2020-07-18T03:27:33.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 shutdown_connections 2020-07-18T03:27:33.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7a68155e20 0x7f7a681562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680b41c0 0x7f7a680b1b10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 --2- 172.21.15.47:0/2307877786 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7a40050330 0x7f7a400527f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:27:33.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.366+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 >> 172.21.15.47:0/2307877786 conn(0x7f7a680b9340 msgr2=0x7f7a6814d570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:27:33.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.367+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 shutdown_connections 2020-07-18T03:27:33.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:27:33.367+0000 7f7a6efba700 1 -- 172.21.15.47:0/2307877786 wait complete. 2020-07-18T03:27:33.371 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:27:33.379 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2020-07-18T03:27:33.514 INFO:teuthology.orchestra.run.smithi047.stdout:[{"id":5211,"entity":{"name":{"type":"client","num":5211},"addr":{"type":"any","addr":"172.21.15.47:0","nonce":1657157254}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":0.67498581999999996,"requests_in_flight":0,"completed_requests":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99844858761906752,"halflife":300},"inst":"client.5211 172.21.15.47:0/1657157254","completed_requests":[],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"admin","hostname":"smithi047","pid":"35209","root":"/"}},{"id":5416,"entity":{"name":{"type":"client","num":5416},"addr":{"type":"any","addr":"192.168.0.2:0","nonce":3207407312}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":1,"uptime":40.406153091,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.91087363127098608,"halflife":300},"inst":"client.5416 192.168.0.2:0/3207407312","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"1","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"34555","root":"/"}}] 2020-07-18T03:27:33.515 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "admin", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "pid": "35209", "root": "/" }, "completed_requests": [], "entity": { "addr": { "addr": "172.21.15.47:0", "nonce": 1657157254, "type": "any" }, "name": { "num": 5211, "type": "client" } }, "id": 5211, "inst": "client.5211 172.21.15.47:0/1657157254", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9984485876190675 }, "state": "open", "uptime": 0.67498582, "used_inos": [] }, { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "1", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "34555", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.2:0", "nonce": 3207407312, "type": "any" }, "name": { "num": 5416, "type": "client" } }, "id": 5416, "inst": "client.5416 192.168.0.2:0/3207407312", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9108736312709861 }, "state": "open", "uptime": 40.406153091, "used_inos": [] } ] 2020-07-18T03:27:49.503 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:27:49.508 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:28:19.577 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:28:19.583 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:28:49.656 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:28:49.662 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:29:03.517 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:03.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.648+0000 7f0ae4f9e700 1 Processor -- start 2020-07-18T03:29:03.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.648+0000 7f0ae4f9e700 1 -- start start 2020-07-18T03:29:03.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.648+0000 7f0ae4f9e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ae0142f10 0x7f0ae00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:03.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.648+0000 7f0ae4f9e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae00afbd0 con 0x7f0ae00b3f10 2020-07-18T03:29:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.648+0000 7f0ae4f9e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae00afe70 con 0x7f0ae00ade90 2020-07-18T03:29:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.648+0000 7f0ae4f9e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00ad9d0 con 0x7f0ae0142f10 2020-07-18T03:29:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adf7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0ae00ade90 0x7f0ae00b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41540/0 (socket says 172.21.15.47:41540) 2020-07-18T03:29:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adf7fe700 1 -- 172.21.15.47:0/3183844987 learned_addr learned my addr 172.21.15.47:0/3183844987 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0ad6ffd700 1 --2- 172.21.15.47:0/3183844987 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ae0142f10 0x7f0ae00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123934134 0 0) 0x7f0ae00afe70 con 0x7f0ae00ade90 2020-07-18T03:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac0003240 con 0x7f0ae00ade90 2020-07-18T03:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361477911 0 0) 0x7f0ae00afbd0 con 0x7f0ae00b3f10 2020-07-18T03:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae00afe70 con 0x7f0ae00b3f10 2020-07-18T03:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0ad6ffd700 1 -- 172.21.15.47:0/3183844987 >> v1:172.21.15.47:6789/0 conn(0x7f0ae00ade90 legacy=0x7f0ae00b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0ad6ffd700 1 -- 172.21.15.47:0/3183844987 >> v1:172.21.15.18:6789/0 conn(0x7f0ae00b3f10 legacy=0x7f0ae014dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0ad6ffd700 1 -- 172.21.15.47:0/3183844987 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0143370 con 0x7f0ae0142f10 2020-07-18T03:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0ad6ffd700 1 --2- 172.21.15.47:0/3183844987 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ae0142f10 0x7f0ae00b3a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f0ac8002f30 tx=0x7f0ac8006dd0).ready entity=mon.2 client_cookie=710987e90d4f408b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.649+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac80096e0 con 0x7f0ae0142f10 2020-07-18T03:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac8009840 con 0x7f0ae0142f10 2020-07-18T03:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/3183844987 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ae0142f10 msgr2=0x7f0ae00b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/3183844987 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ae0142f10 0x7f0ae00b3a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f0ac8002f30 tx=0x7f0ac8006dd0).stop 2020-07-18T03:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0adeffd700 1 -- 172.21.15.47:0/3183844987 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac8009ba0 con 0x7f0ae0142f10 2020-07-18T03:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/3183844987 shutdown_connections 2020-07-18T03:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/3183844987 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0ae0142f10 0x7f0ae00b3a20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/3183844987 >> 172.21.15.47:0/3183844987 conn(0x7f0ae00b9e50 msgr2=0x7f0ae00b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/3183844987 shutdown_connections 2020-07-18T03:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.650+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/3183844987 wait complete. 2020-07-18T03:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.651+0000 7f0ae4f9e700 1 Processor -- start 2020-07-18T03:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.651+0000 7f0ae4f9e700 1 -- start start 2020-07-18T03:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0ae4f9e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0ae00ade90 0x7f0ae0063b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0ae4f9e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0ae013eee0 0x7f0ae013f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0ae4f9e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae013ea40 con 0x7f0ae00b3f10 2020-07-18T03:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0ae4f9e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae00625d0 con 0x7f0ae00ade90 2020-07-18T03:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0ae4f9e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae00627e0 con 0x7f0ae013eee0 2020-07-18T03:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0ae013eee0 0x7f0ae013f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0ae013eee0 0x7f0ae013f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60462/0 (socket says 172.21.15.47:60462) 2020-07-18T03:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 -- 172.21.15.47:0/1953174678 learned_addr learned my addr 172.21.15.47:0/1953174678 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adf7fe700 1 --2- 172.21.15.47:0/1953174678 >> v2:172.21.15.18:3300/0 conn(0x7f0ae00ade90 0x7f0ae0063b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 -- 172.21.15.47:0/1953174678 >> v2:172.21.15.18:3300/0 conn(0x7f0ae00ade90 msgr2=0x7f0ae0063b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 --2- 172.21.15.47:0/1953174678 >> v2:172.21.15.18:3300/0 conn(0x7f0ae00ade90 0x7f0ae0063b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 -- 172.21.15.47:0/1953174678 >> v1:172.21.15.18:6789/0 conn(0x7f0ae00b3f10 legacy=0x7f0ae00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adffff700 1 -- 172.21.15.47:0/1953174678 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae00b2650 con 0x7f0ae013eee0 2020-07-18T03:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.652+0000 7f0adf7fe700 1 --2- 172.21.15.47:0/1953174678 >> v2:172.21.15.18:3300/0 conn(0x7f0ae00ade90 0x7f0ae0063b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.653+0000 7f0adffff700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ae013eee0 0x7f0ae013f350 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f0ad8001450 tx=0x7f0ad8007d40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.653+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ad800a7b0 con 0x7f0ae013eee0 2020-07-18T03:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.653+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad800a910 con 0x7f0ae013eee0 2020-07-18T03:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.653+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae0140020 con 0x7f0ae013eee0 2020-07-18T03:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.653+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ad800aa80 con 0x7f0ae013eee0 2020-07-18T03:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.653+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae0064050 con 0x7f0ae013eee0 2020-07-18T03:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.654+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae0062380 con 0x7f0ae013eee0 2020-07-18T03:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.654+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0ad8010070 con 0x7f0ae013eee0 2020-07-18T03:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.654+0000 7f0ad7fff700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0abc04ffb0 0x7f0abc052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.654+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0ad8067b50 con 0x7f0ae013eee0 2020-07-18T03:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.654+0000 7f0adf7fe700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0abc04ffb0 0x7f0abc052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.655+0000 7f0adf7fe700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0abc04ffb0 0x7f0abc052470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f0ad000b000 tx=0x7f0ad0009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.658+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0ad8040260 con 0x7f0ae013eee0 2020-07-18T03:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.864+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ae013ebd0 con 0x7f0ae013eee0 2020-07-18T03:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.864+0000 7f0ad7fff700 1 -- 172.21.15.47:0/1953174678 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f0ad8043880 con 0x7f0ae013eee0 2020-07-18T03:29:03.866 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:03.866 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0abc04ffb0 msgr2=0x7f0abc052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0abc04ffb0 0x7f0abc052470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f0ad000b000 tx=0x7f0ad0009040).stop 2020-07-18T03:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ae013eee0 msgr2=0x7f0ae013f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ae013eee0 0x7f0ae013f350 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f0ad8001450 tx=0x7f0ad8007d40).stop 2020-07-18T03:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 shutdown_connections 2020-07-18T03:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0abc04ffb0 0x7f0abc052470 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/1953174678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ae013eee0 0x7f0ae013f350 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 --2- 172.21.15.47:0/1953174678 >> v2:172.21.15.18:3300/0 conn(0x7f0ae00ade90 0x7f0ae0063b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 >> 172.21.15.47:0/1953174678 conn(0x7f0ae00b9e50 msgr2=0x7f0ae0142230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 shutdown_connections 2020-07-18T03:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:03.866+0000 7f0ae4f9e700 1 -- 172.21.15.47:0/1953174678 wait complete. 2020-07-18T03:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:29:03.878 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2020-07-18T03:29:04.012 INFO:teuthology.orchestra.run.smithi047.stdout:[{"id":5211,"entity":{"name":{"type":"client","num":5211},"addr":{"type":"any","addr":"172.21.15.47:0","nonce":1657157254}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":91.173085404000005,"requests_in_flight":0,"completed_requests":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.81005946077730739,"halflife":300},"inst":"client.5211 172.21.15.47:0/1657157254","completed_requests":[],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"admin","hostname":"smithi047","pid":"35209","root":"/"}},{"id":5416,"entity":{"name":{"type":"client","num":5416},"addr":{"type":"any","addr":"192.168.0.2:0","nonce":3207407312}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":130.90425267500001,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.73900830922418537,"halflife":300},"inst":"client.5416 192.168.0.2:0/3207407312","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"1","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"34555","root":"/"}}] 2020-07-18T03:29:04.014 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "admin", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "pid": "35209", "root": "/" }, "completed_requests": [], "entity": { "addr": { "addr": "172.21.15.47:0", "nonce": 1657157254, "type": "any" }, "name": { "num": 5211, "type": "client" } }, "id": 5211, "inst": "client.5211 172.21.15.47:0/1657157254", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.8100594607773074 }, "state": "stale", "uptime": 91.173085404, "used_inos": [] }, { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "1", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "34555", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.2:0", "nonce": 3207407312, "type": "any" }, "name": { "num": 5416, "type": "client" } }, "id": 5416, "inst": "client.5416 192.168.0.2:0/3207407312", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.7390083092241854 }, "state": "stale", "uptime": 130.904252675, "used_inos": [] } ] 2020-07-18T03:29:04.015 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:29:04.015 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:04.075 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:04.135 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:29:04.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.137 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-07-18T03:29:04.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:29:04.157 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:29:04.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.158 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:04.212 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:04.212 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:04.213 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.214 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.214 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:04.214 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42647sec preferred_lft 42647sec 2020-07-18T03:29:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:04.215 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.216 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.216 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.222 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:04.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.268 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-07-18T03:29:04.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:29:04.323 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:29:04.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.325 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:04.377 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:04.377 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:04.379 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.379 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.380 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.381 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.381 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42647sec preferred_lft 42647sec 2020-07-18T03:29:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.383 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.383 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.383 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.384 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.385 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:04.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.432 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:29:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-07-18T03:29:04.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:29:04.452 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-07-18T03:29:04.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.453 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:29:04.508 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.510 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.510 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42646sec preferred_lft 42646sec 2020-07-18T03:29:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:29:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.511 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.512 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.513 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:29:04.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.568 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:29:04.625 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2020-07-18T03:29:04.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:29:04.626 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-07-18T03:29:04.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.627 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:29:04.683 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.685 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.686 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42646sec preferred_lft 42646sec 2020-07-18T03:29:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:29:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:04.687 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.688 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:04.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:04.689 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:29:04.742 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn' 2020-07-18T03:29:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.864+0000 7ff7e8fc0700 1 Processor -- start 2020-07-18T03:29:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.864+0000 7ff7e8fc0700 1 -- start start 2020-07-18T03:29:04.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7e40b41c0 0x7ff7e40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:04.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e41428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:04.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40b3ac0 con 0x7ff7e40b5370 2020-07-18T03:29:04.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40b5a10 con 0x7ff7e40ad7c0 2020-07-18T03:29:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40b0040 con 0x7ff7e40b41c0 2020-07-18T03:29:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7e40b41c0 0x7ff7e40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7e40b5370 0x7ff7e405fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41554/0 (socket says 172.21.15.47:41554) 2020-07-18T03:29:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e259c700 1 -- 172.21.15.47:0/4277440450 learned_addr learned my addr 172.21.15.47:0/4277440450 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e2d9d700 1 --2- 172.21.15.47:0/4277440450 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e41428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e1d9b700 1 -- 172.21.15.47:0/4277440450 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 msgr2=0x7ff7e41428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e1d9b700 1 --2- 172.21.15.47:0/4277440450 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e41428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e1d9b700 1 -- 172.21.15.47:0/4277440450 >> v1:172.21.15.47:6789/0 conn(0x7ff7e40b5370 legacy=0x7ff7e405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e1d9b700 1 -- 172.21.15.47:0/4277440450 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40b3d20 con 0x7ff7e40b41c0 2020-07-18T03:29:04.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e159a700 1 -- 172.21.15.47:0/4277440450 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612230306 0 0) 0x7ff7e40b3ac0 con 0x7ff7e40b5370 2020-07-18T03:29:04.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e1d9b700 1 --2- 172.21.15.47:0/4277440450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e40b41c0 0x7ff7e40ad2d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7ff7e40afb30 tx=0x7ff7cc006e60).ready entity=mon.1 client_cookie=1206b4eaa4849f6d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:04.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e159a700 1 -- 172.21.15.47:0/4277440450 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7cc014070 con 0x7ff7e40b41c0 2020-07-18T03:29:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e159a700 1 -- 172.21.15.47:0/4277440450 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00aa90 con 0x7ff7e40b41c0 2020-07-18T03:29:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e159a700 1 -- 172.21.15.47:0/4277440450 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7cc00adf0 con 0x7ff7e40b41c0 2020-07-18T03:29:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/4277440450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e40b41c0 msgr2=0x7ff7e40ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/4277440450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e40b41c0 0x7ff7e40ad2d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7ff7e40afb30 tx=0x7ff7cc006e60).stop 2020-07-18T03:29:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/4277440450 shutdown_connections 2020-07-18T03:29:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.865+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/4277440450 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e41428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.866+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/4277440450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e40b41c0 0x7ff7e40ad2d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:04.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.866+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/4277440450 >> 172.21.15.47:0/4277440450 conn(0x7ff7e40b9340 msgr2=0x7ff7e40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:04.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.866+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/4277440450 shutdown_connections 2020-07-18T03:29:04.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.866+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/4277440450 wait complete. 2020-07-18T03:29:04.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.867+0000 7ff7e8fc0700 1 Processor -- start 2020-07-18T03:29:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.867+0000 7ff7e8fc0700 1 -- start start 2020-07-18T03:29:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e8fc0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e4153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e8fc0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7e4143450 0x7ff7e4065190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e8fc0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e4143af0 con 0x7ff7e40b5370 2020-07-18T03:29:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e8fc0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e41497c0 con 0x7ff7e40ad7c0 2020-07-18T03:29:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e8fc0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e4143d90 con 0x7ff7e4143450 2020-07-18T03:29:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7e4143450 0x7ff7e4065190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7e4143450 0x7ff7e4065190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60476/0 (socket says 172.21.15.47:60476) 2020-07-18T03:29:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e1d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7e40b5370 0x7ff7e4148d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41562/0 (socket says 172.21.15.47:41562) 2020-07-18T03:29:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 -- 172.21.15.47:0/2614110693 learned_addr learned my addr 172.21.15.47:0/2614110693 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e259c700 1 --2- 172.21.15.47:0/2614110693 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e4153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 -- 172.21.15.47:0/2614110693 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 msgr2=0x7ff7e4153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 --2- 172.21.15.47:0/2614110693 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e4153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 -- 172.21.15.47:0/2614110693 >> v1:172.21.15.47:6789/0 conn(0x7ff7e40b5370 legacy=0x7ff7e4148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837159281 0 0) 0x7ff7e4143af0 con 0x7ff7e40b5370 2020-07-18T03:29:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 -- 172.21.15.47:0/2614110693 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e4144030 con 0x7ff7e4143450 2020-07-18T03:29:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e259c700 1 --2- 172.21.15.47:0/2614110693 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e4153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:04.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e2d9d700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e4143450 0x7ff7e4065190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ff7dc00a000 tx=0x7ff7dc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:04.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7dc009a90 con 0x7ff7e4143450 2020-07-18T03:29:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc009bf0 con 0x7ff7e4143450 2020-07-18T03:29:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.868+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e4143850 con 0x7ff7e4143450 2020-07-18T03:29:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.869+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7dc013480 con 0x7ff7e4143450 2020-07-18T03:29:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.869+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e4065680 con 0x7ff7e4143450 2020-07-18T03:29:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.869+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4057450 con 0x7ff7e4143450 2020-07-18T03:29:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.870+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff7dc0135e0 con 0x7ff7e4143450 2020-07-18T03:29:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.870+0000 7ff7da7fc700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff7c004ff50 0x7ff7c0052410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.870+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff7dc0686b0 con 0x7ff7e4143450 2020-07-18T03:29:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.870+0000 7ff7e259c700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff7c004ff50 0x7ff7c0052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.870+0000 7ff7e259c700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff7c004ff50 0x7ff7c0052410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff7d400b000 tx=0x7ff7d4009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:04.873+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff7dc040de0 con 0x7ff7e4143450 2020-07-18T03:29:05.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.087+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]} v 0) v1 -- 0x7ff7e40b01a0 con 0x7ff7e4143450 2020-07-18T03:29:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.372+0000 7ff7da7fc700 1 -- 172.21.15.47:0/2614110693 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]}]=0 v150) v1 ==== 147+0+0 (secure 0 0 0) 0x7ff7dc044400 con 0x7ff7e4143450 2020-07-18T03:29:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff7c004ff50 msgr2=0x7ff7c0052410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff7c004ff50 0x7ff7c0052410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff7d400b000 tx=0x7ff7d4009040).stop 2020-07-18T03:29:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e4143450 msgr2=0x7ff7e4065190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e4143450 0x7ff7e4065190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ff7dc00a000 tx=0x7ff7dc008040).stop 2020-07-18T03:29:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 shutdown_connections 2020-07-18T03:29:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e4143450 0x7ff7e4065190 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/2614110693 >> v2:172.21.15.18:3300/0 conn(0x7ff7e40ad7c0 0x7ff7e4153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 --2- 172.21.15.47:0/2614110693 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff7c004ff50 0x7ff7c0052410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 >> 172.21.15.47:0/2614110693 conn(0x7ff7e40b9340 msgr2=0x7ff7e4141e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 shutdown_connections 2020-07-18T03:29:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.374+0000 7ff7e8fc0700 1 -- 172.21.15.47:0/2614110693 wait complete. 2020-07-18T03:29:05.387 INFO:tasks.cephfs_test_runner:test_abort_conn (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2020-07-18T03:29:05.388 INFO:tasks.cephfs_test_runner:Starting test: test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) 2020-07-18T03:29:05.388 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic' 2020-07-18T03:29:05.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.502+0000 7fb8d1e4d700 1 Processor -- start 2020-07-18T03:29:05.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.502+0000 7fb8d1e4d700 1 -- start start 2020-07-18T03:29:05.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8d1e4d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb8cc0b2e10 0x7fb8cc0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:05.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8d1e4d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0afbd0 con 0x7fb8cc0b1560 2020-07-18T03:29:05.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8d1e4d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc1491c0 con 0x7fb8cc0b1a10 2020-07-18T03:29:05.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8d1e4d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc144e30 con 0x7fb8cc0b2e10 2020-07-18T03:29:05.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8cbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8cc0b1a10 0x7fb8cc1454f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41572/0 (socket says 172.21.15.47:41572) 2020-07-18T03:29:05.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8cbfff700 1 -- 172.21.15.47:0/1963508397 learned_addr learned my addr 172.21.15.47:0/1963508397 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:05.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8cb7fe700 1 --2- 172.21.15.47:0/1963508397 >> v2:172.21.15.18:3300/0 conn(0x7fb8cc0b2e10 0x7fb8cc0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316123683 0 0) 0x7fb8cc1491c0 con 0x7fb8cc0b1a10 2020-07-18T03:29:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b8003240 con 0x7fb8cc0b1a10 2020-07-18T03:29:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3101903223 0 0) 0x7fb8b8003240 con 0x7fb8cc0b1a10 2020-07-18T03:29:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 >> v2:172.21.15.18:3300/0 conn(0x7fb8cc0b2e10 msgr2=0x7fb8cc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 --2- 172.21.15.47:0/1963508397 >> v2:172.21.15.18:3300/0 conn(0x7fb8cc0b2e10 0x7fb8cc0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 >> v1:172.21.15.18:6789/0 conn(0x7fb8cc0b1560 legacy=0x7fb8cc0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc145050 con 0x7fb8cc0b1a10 2020-07-18T03:29:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fb8c4001a50 con 0x7fb8cc0b1a10 2020-07-18T03:29:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 >> v1:172.21.15.47:6789/0 conn(0x7fb8cc0b1a10 legacy=0x7fb8cc1454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8b8003760 0x7fb8b8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8b80044e0 0x7fb8b8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8004e50 0x7fb8b800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.503+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc144e30 con 0x7fb8b8004e50 2020-07-18T03:29:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b800bc20 con 0x7fb8b80044e0 2020-07-18T03:29:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b800bd60 con 0x7fb8b8003760 2020-07-18T03:29:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8b80044e0 0x7fb8b8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8cbfff700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8004e50 0x7fb8b800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8cb7fe700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8b8003760 0x7fb8b8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 -- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8b8003760 msgr2=0x7fb8b8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8b8003760 0x7fb8b8003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 -- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8004e50 msgr2=0x7fb8b800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8004e50 0x7fb8b800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 -- 172.21.15.47:0/1963508397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8003240 con 0x7fb8b80044e0 2020-07-18T03:29:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8cbfff700 1 -- 172.21.15.47:0/1963508397 reap_dead start 2020-07-18T03:29:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8caffd700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8b80044e0 0x7fb8b8004960 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fb8cc0b5d30 tx=0x7fb8c0007b70).ready entity=mon.1 client_cookie=b63b3916fb580dc3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c000c870 con 0x7fb8b80044e0 2020-07-18T03:29:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c000c9d0 con 0x7fb8b80044e0 2020-07-18T03:29:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8ca7fc700 1 -- 172.21.15.47:0/1963508397 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c000cd30 con 0x7fb8b80044e0 2020-07-18T03:29:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/1963508397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8b80044e0 msgr2=0x7fb8b8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8b80044e0 0x7fb8b8004960 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fb8cc0b5d30 tx=0x7fb8c0007b70).stop 2020-07-18T03:29:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/1963508397 shutdown_connections 2020-07-18T03:29:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 --2- 172.21.15.47:0/1963508397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8b80044e0 0x7fb8b8004960 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/1963508397 >> 172.21.15.47:0/1963508397 conn(0x7fb8cc0b9340 msgr2=0x7fb8cc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/1963508397 shutdown_connections 2020-07-18T03:29:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.504+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/1963508397 wait complete. 2020-07-18T03:29:05.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.506+0000 7fb8d1e4d700 1 Processor -- start 2020-07-18T03:29:05.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.506+0000 7fb8d1e4d700 1 -- start start 2020-07-18T03:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8d1e4d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8cc066620 0x7fb8cc13fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8d1e4d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc148230 con 0x7fb8cc153db0 2020-07-18T03:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8d1e4d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc147f90 con 0x7fb8cc066240 2020-07-18T03:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8d1e4d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0649c0 con 0x7fb8cc066620 2020-07-18T03:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8cb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8cc066240 0x7fb8cc13f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41580/0 (socket says 172.21.15.47:41580) 2020-07-18T03:29:05.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8cb7fe700 1 -- 172.21.15.47:0/316518290 learned_addr learned my addr 172.21.15.47:0/316518290 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:05.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8caffd700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8cc066620 0x7fb8cc13fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:05.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755021437 0 0) 0x7fb8cc147f90 con 0x7fb8cc066240 2020-07-18T03:29:05.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a8003240 con 0x7fb8cc066240 2020-07-18T03:29:05.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230873104 0 0) 0x7fb8cc148230 con 0x7fb8cc153db0 2020-07-18T03:29:05.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8cc147f90 con 0x7fb8cc153db0 2020-07-18T03:29:05.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8caffd700 1 -- 172.21.15.47:0/316518290 >> v1:172.21.15.47:6789/0 conn(0x7fb8cc066240 legacy=0x7fb8cc13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2185197 0 0) 0x7fb8a8003240 con 0x7fb8cc066240 2020-07-18T03:29:05.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8caffd700 1 -- 172.21.15.47:0/316518290 >> v1:172.21.15.18:6789/0 conn(0x7fb8cc153db0 legacy=0x7fb8cc146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:05.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8caffd700 1 -- 172.21.15.47:0/316518290 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc147880 con 0x7fb8cc066620 2020-07-18T03:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.507+0000 7fb8caffd700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8cc066620 0x7fb8cc13fd00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb8bc00bb00 tx=0x7fb8c0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.508+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8bc00b960 con 0x7fb8cc066620 2020-07-18T03:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.508+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0013e10 con 0x7fb8cc066620 2020-07-18T03:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.508+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc147ad0 con 0x7fb8cc066620 2020-07-18T03:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.508+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c000c870 con 0x7fb8cc066620 2020-07-18T03:29:05.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.508+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc146b30 con 0x7fb8cc066620 2020-07-18T03:29:05.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.509+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc057450 con 0x7fb8cc066620 2020-07-18T03:29:05.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.509+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb8c0010030 con 0x7fb8cc066620 2020-07-18T03:29:05.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.509+0000 7fb8b37fe700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb8a80504f0 0x7fb8a80529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:05.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.509+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb8c0068f10 con 0x7fb8cc066620 2020-07-18T03:29:05.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.509+0000 7fb8cb7fe700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb8a80504f0 0x7fb8a80529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:05.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.510+0000 7fb8cb7fe700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb8a80504f0 0x7fb8a80529b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb8bc006f50 tx=0x7fb8bc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:05.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.514+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb8c00416a0 con 0x7fb8cc066620 2020-07-18T03:29:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:05.725+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]} v 0) v1 -- 0x7fb8cc0afbd0 con 0x7fb8cc066620 2020-07-18T03:29:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.376+0000 7fb8b37fe700 1 -- 172.21.15.47:0/316518290 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]}]=0 v151) v1 ==== 145+0+0 (secure 0 0 0) 0x7fb8cc0afbd0 con 0x7fb8cc066620 2020-07-18T03:29:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb8a80504f0 msgr2=0x7fb8a80529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb8a80504f0 0x7fb8a80529b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb8bc006f50 tx=0x7fb8bc008040).stop 2020-07-18T03:29:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8cc066620 msgr2=0x7fb8cc13fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8cc066620 0x7fb8cc13fd00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb8bc00bb00 tx=0x7fb8c0016000).stop 2020-07-18T03:29:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 shutdown_connections 2020-07-18T03:29:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb8a80504f0 0x7fb8a80529b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 --2- 172.21.15.47:0/316518290 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8cc066620 0x7fb8cc13fd00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 >> 172.21.15.47:0/316518290 conn(0x7fb8cc0b9340 msgr2=0x7fb8cc05f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 shutdown_connections 2020-07-18T03:29:06.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.378+0000 7fb8d1e4d700 1 -- 172.21.15.47:0/316518290 wait complete. 2020-07-18T03:29:06.393 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:29:06.394 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:06.454 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:06.508 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:29:06.509 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-07-18T03:29:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.630+0000 7f72f270d700 1 Processor -- start 2020-07-18T03:29:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.630+0000 7f72f270d700 1 -- start start 2020-07-18T03:29:06.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f270d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f72ec0ade90 0x7f72ec142f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f270d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72ec0b0950 0x7f72ec147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f270d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec143400 con 0x7f72ec0b04c0 2020-07-18T03:29:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f270d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec0afbd0 con 0x7f72ec0ade90 2020-07-18T03:29:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f270d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec0afe70 con 0x7f72ec0b0950 2020-07-18T03:29:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f72ec0ade90 0x7f72ec142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f1f0c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72ec0b0950 0x7f72ec147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f72ec0ade90 0x7f72ec142f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:46180/0 (socket says 172.21.15.47:46180) 2020-07-18T03:29:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 -- 172.21.15.47:0/865612176 learned_addr learned my addr 172.21.15.47:0/865612176 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f0f0a700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f72ec0b04c0 0x7f72ec146b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60356/0 (socket says 172.21.15.47:60356) 2020-07-18T03:29:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 -- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72ec0b0950 msgr2=0x7f72ec147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 --2- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72ec0b0950 0x7f72ec147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 -- 172.21.15.47:0/865612176 >> v1:172.21.15.18:6789/0 conn(0x7f72ec0b04c0 legacy=0x7f72ec146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.631+0000 7f72f170b700 1 -- 172.21.15.47:0/865612176 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec0ad9d0 con 0x7f72ec0ade90 2020-07-18T03:29:06.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f170b700 1 --2- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72ec0ade90 0x7f72ec142f10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f72ec0b0110 tx=0x7f72e4009270).ready entity=mon.0 client_cookie=242e78333cf18027 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:06.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72ebfff700 1 -- 172.21.15.47:0/865612176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e4009e90 con 0x7f72ec0ade90 2020-07-18T03:29:06.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72ebfff700 1 -- 172.21.15.47:0/865612176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e400bbc0 con 0x7f72ec0ade90 2020-07-18T03:29:06.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72ebfff700 1 -- 172.21.15.47:0/865612176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e400bd30 con 0x7f72ec0ade90 2020-07-18T03:29:06.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 -- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72ec0ade90 msgr2=0x7f72ec142f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 --2- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72ec0ade90 0x7f72ec142f10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f72ec0b0110 tx=0x7f72e4009270).stop 2020-07-18T03:29:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 -- 172.21.15.47:0/865612176 shutdown_connections 2020-07-18T03:29:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 --2- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72ec0b0950 0x7f72ec147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 --2- 172.21.15.47:0/865612176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72ec0ade90 0x7f72ec142f10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 -- 172.21.15.47:0/865612176 >> 172.21.15.47:0/865612176 conn(0x7f72ec0b9e50 msgr2=0x7f72ec0b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:06.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 -- 172.21.15.47:0/865612176 shutdown_connections 2020-07-18T03:29:06.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.632+0000 7f72f270d700 1 -- 172.21.15.47:0/865612176 wait complete. 2020-07-18T03:29:06.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.633+0000 7f72f270d700 1 Processor -- start 2020-07-18T03:29:06.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f270d700 1 -- start start 2020-07-18T03:29:06.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f270d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f72ec0b0950 0x7f72ec143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:06.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f270d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72ec158a50 0x7f72ec13a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:06.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f270d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec159420 con 0x7f72ec0b04c0 2020-07-18T03:29:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f270d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec1596c0 con 0x7f72ec0b0950 2020-07-18T03:29:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f270d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec155630 con 0x7f72ec158a50 2020-07-18T03:29:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f1f0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72ec158a50 0x7f72ec13a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.634+0000 7f72f1f0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72ec158a50 0x7f72ec13a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60510/0 (socket says 172.21.15.47:60510) 2020-07-18T03:29:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f1f0c700 1 -- 172.21.15.47:0/3454854258 learned_addr learned my addr 172.21.15.47:0/3454854258 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f0f0a700 1 --2- 172.21.15.47:0/3454854258 >> v2:172.21.15.18:3300/0 conn(0x7f72ec0b0950 0x7f72ec143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f1f0c700 1 -- 172.21.15.47:0/3454854258 >> v2:172.21.15.18:3300/0 conn(0x7f72ec0b0950 msgr2=0x7f72ec143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f1f0c700 1 --2- 172.21.15.47:0/3454854258 >> v2:172.21.15.18:3300/0 conn(0x7f72ec0b0950 0x7f72ec143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f1f0c700 1 -- 172.21.15.47:0/3454854258 >> v1:172.21.15.18:6789/0 conn(0x7f72ec0b04c0 legacy=0x7f72ec0660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f1f0c700 1 -- 172.21.15.47:0/3454854258 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec1557a0 con 0x7f72ec158a50 2020-07-18T03:29:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f1f0c700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72ec158a50 0x7f72ec13a5e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f72dc00c000 tx=0x7f72dc006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72dc00ca10 con 0x7f72ec158a50 2020-07-18T03:29:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec158ec0 con 0x7f72ec158a50 2020-07-18T03:29:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00cb70 con 0x7f72ec158a50 2020-07-18T03:29:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72dc0123f0 con 0x7f72ec158a50 2020-07-18T03:29:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.635+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec13aad0 con 0x7f72ec158a50 2020-07-18T03:29:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.636+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec067e20 con 0x7f72ec158a50 2020-07-18T03:29:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.636+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f72dc0125f0 con 0x7f72ec158a50 2020-07-18T03:29:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.636+0000 7f72e97fa700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f72c404ff30 0x7f72c40523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.637+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72dc0682d0 con 0x7f72ec158a50 2020-07-18T03:29:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.637+0000 7f72f170b700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f72c404ff30 0x7f72c40523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:06.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.637+0000 7f72f170b700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f72c404ff30 0x7f72c40523f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f72ec140790 tx=0x7f72e4016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.640+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f72dc040980 con 0x7f72ec158a50 2020-07-18T03:29:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.797+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f72ec0aef80 con 0x7f72ec158a50 2020-07-18T03:29:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.798+0000 7f72e97fa700 1 -- 172.21.15.47:0/3454854258 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v33) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f72dc043fa0 con 0x7f72ec158a50 2020-07-18T03:29:06.801 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:06.801 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:06.801 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 33, 2020-07-18T03:29:06.801 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", 2020-07-18T03:29:06.802 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2020-07-18T03:25:24.387359+0000", 2020-07-18T03:29:06.802 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2020-07-18T03:26:41.771920+0000", 2020-07-18T03:29:06.802 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:29:06.802 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:29:06.802 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-07-18T03:29:06.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2020-07-18T03:29:06.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2020-07-18T03:29:06.803 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2020-07-18T03:29:06.803 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2020-07-18T03:29:06.803 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2020-07-18T03:29:06.804 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2020-07-18T03:29:06.804 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:06.804 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2020-07-18T03:29:06.804 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2020-07-18T03:29:06.804 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-07-18T03:29:06.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2020-07-18T03:29:06.809 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2020-07-18T03:29:06.809 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 6, 2020-07-18T03:29:06.809 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2020-07-18T03:29:06.810 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "luminous", 2020-07-18T03:29:06.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2020-07-18T03:29:06.810 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "pacific", 2020-07-18T03:29:06.810 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2020-07-18T03:29:06.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.810 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2020-07-18T03:29:06.811 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2020-07-18T03:29:06.811 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:27.725131+0000", 2020-07-18T03:29:06.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:06.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:06.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:06.812 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:06.812 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:06.812 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:06.812 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:06.812 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:06.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2020-07-18T03:29:06.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2020-07-18T03:29:06.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2020-07-18T03:29:06.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2020-07-18T03:29:06.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2020-07-18T03:29:06.814 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:06.814 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:06.814 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:06.814 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:06.814 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:06.815 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:06.815 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:06.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.815 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "15", 2020-07-18T03:29:06.816 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:06.816 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:06.816 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:06.816 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:06.817 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:06.817 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:06.817 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:06.817 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:06.817 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:06.818 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:06.818 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:06.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:06.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:06.818 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:06.819 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:06.819 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:06.819 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:06.819 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:06.819 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:06.820 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:06.820 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:06.820 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:06.820 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:06.821 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:06.821 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:06.822 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:06.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:06.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:06.823 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:06.823 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:06.823 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:06.823 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:06.824 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:06.824 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:06.824 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:29:06.824 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2020-07-18T03:29:06.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.825 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:06.825 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2020-07-18T03:29:06.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.826 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2020-07-18T03:29:06.826 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2020-07-18T03:29:06.826 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:37.896092+0000", 2020-07-18T03:29:06.826 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:06.826 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:06.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:06.827 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:06.827 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:06.827 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:06.827 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:06.827 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:06.828 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:29:06.828 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:29:06.828 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:29:06.828 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:29:06.828 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:29:06.829 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:06.829 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:06.829 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:06.829 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:06.829 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:06.830 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:06.830 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:06.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.830 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "18", 2020-07-18T03:29:06.830 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:06.830 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:06.831 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:06.831 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:06.831 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:06.831 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:06.831 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:06.832 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:06.832 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:06.832 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:06.832 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:06.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:06.833 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:06.833 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:06.833 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:06.833 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:06.833 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:06.834 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:06.834 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:06.834 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:06.834 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:06.835 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:06.835 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:06.835 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:06.835 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:06.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:06.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.836 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:06.836 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:06.836 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:06.836 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:06.837 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:06.837 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:06.837 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:06.837 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:06.837 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:06.839 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:06.839 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:06.839 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:29:06.839 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:06.840 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2020-07-18T03:29:06.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.841 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 5, 2020-07-18T03:29:06.841 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2020-07-18T03:29:06.841 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:26:38.665431+0000", 2020-07-18T03:29:06.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:06.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:06.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:06.842 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:06.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:06.842 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:06.843 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:06.843 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:06.843 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:29:06.843 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:29:06.843 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:29:06.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:29:06.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:29:06.844 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:06.844 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:06.844 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:06.845 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:06.845 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:06.845 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:06.845 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:06.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.845 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "32", 2020-07-18T03:29:06.846 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:06.846 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:06.846 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:06.846 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:06.846 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:06.847 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:06.847 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:06.847 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:06.847 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:06.847 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:06.848 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:06.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:06.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:06.848 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:06.848 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:06.853 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:06.853 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:29:06.853 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2020-07-18T03:29:06.853 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2020-07-18T03:29:06.853 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2020-07-18T03:29:06.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.854 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:06.854 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:29:06.854 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2020-07-18T03:29:06.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.855 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.855 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 6, 2020-07-18T03:29:06.855 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2020-07-18T03:29:06.856 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:26:39.748890+0000", 2020-07-18T03:29:06.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:06.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:06.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:06.856 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:06.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:06.857 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:06.857 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:06.857 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:06.857 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:29:06.857 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:29:06.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:29:06.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:29:06.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:29:06.858 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:06.858 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:06.859 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:06.859 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:06.859 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:06.859 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:06.859 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:06.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "33", 2020-07-18T03:29:06.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:06.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:06.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:06.860 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:06.861 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:06.861 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:06.861 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:06.861 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:06.861 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:06.862 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:06.862 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:06.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:06.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:06.862 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:06.863 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:06.863 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:06.863 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:06.863 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:06.863 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:06.864 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:06.864 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:06.864 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:06.864 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:06.864 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:06.864 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:06.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:06.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.865 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:06.865 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:06.865 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:06.866 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:06.866 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:06.866 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:06.866 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:06.866 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:06.867 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:06.867 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:06.867 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:06.867 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:29:06.867 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:06.868 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:29:06.868 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2020-07-18T03:29:06.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.869 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:06.869 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2020-07-18T03:29:06.869 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.869 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:29:06.869 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", 2020-07-18T03:29:06.869 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.870 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.870 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.870 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.870 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.870 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.871 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.871 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 31, 2020-07-18T03:29:06.871 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.871 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.871 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.872 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.872 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2020-07-18T03:29:06.872 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.873 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2020-07-18T03:29:06.873 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.873 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.874 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.874 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.874 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.874 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.875 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2020-07-18T03:29:06.875 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.875 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.876 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2020-07-18T03:29:06.876 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.876 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.876 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.877 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.877 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.877 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.877 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2020-07-18T03:29:06.878 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.878 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.878 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.878 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2020-07-18T03:29:06.879 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.879 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.879 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.879 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.880 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.880 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2020-07-18T03:29:06.880 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.881 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.881 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2020-07-18T03:29:06.881 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:06.881 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.881 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.882 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6821/23865", 2020-07-18T03:29:06.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6825/23865", 2020-07-18T03:29:06.882 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6832/23865", 2020-07-18T03:29:06.882 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6829/23865", 2020-07-18T03:29:06.883 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.883 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.883 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.883 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.884 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.884 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:29:06.884 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", 2020-07-18T03:29:06.884 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.884 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.885 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.885 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.885 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.885 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.885 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.885 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 13, 2020-07-18T03:29:06.886 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.886 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.886 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.886 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.886 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.888 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2020-07-18T03:29:06.888 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.889 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2020-07-18T03:29:06.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.889 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.889 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2020-07-18T03:29:06.891 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.891 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2020-07-18T03:29:06.891 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.892 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.892 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2020-07-18T03:29:06.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2020-07-18T03:29:06.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.896 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.896 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2020-07-18T03:29:06.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2020-07-18T03:29:06.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:06.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.898 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/23853", 2020-07-18T03:29:06.899 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/23853", 2020-07-18T03:29:06.899 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/23853", 2020-07-18T03:29:06.899 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/23853", 2020-07-18T03:29:06.899 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.899 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.900 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.900 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:29:06.901 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "dc0b1a97-efca-483d-839c-37c286699283", 2020-07-18T03:29:06.901 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.901 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.901 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.901 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.902 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.902 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.902 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.902 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 22, 2020-07-18T03:29:06.902 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.902 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.903 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2020-07-18T03:29:06.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2020-07-18T03:29:06.905 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.905 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.905 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.905 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2020-07-18T03:29:06.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2020-07-18T03:29:06.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.908 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.908 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2020-07-18T03:29:06.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2020-07-18T03:29:06.910 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.910 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.911 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.911 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2020-07-18T03:29:06.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2020-07-18T03:29:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/23872", 2020-07-18T03:29:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6824/23872", 2020-07-18T03:29:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/23872", 2020-07-18T03:29:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6828/23872", 2020-07-18T03:29:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.916 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.916 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:29:06.916 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", 2020-07-18T03:29:06.917 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.917 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.917 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.917 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.917 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.918 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.918 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.918 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 31, 2020-07-18T03:29:06.918 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.918 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.919 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.919 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2020-07-18T03:29:06.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2020-07-18T03:29:06.920 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.921 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.921 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.921 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2020-07-18T03:29:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2020-07-18T03:29:06.923 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.923 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.924 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2020-07-18T03:29:06.925 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2020-07-18T03:29:06.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.926 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2020-07-18T03:29:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.928 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.928 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2020-07-18T03:29:06.928 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:06.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.929 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.929 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/23854", 2020-07-18T03:29:06.929 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/23854", 2020-07-18T03:29:06.930 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/23854", 2020-07-18T03:29:06.930 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/23854", 2020-07-18T03:29:06.930 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.930 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.930 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.930 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.931 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:29:06.931 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", 2020-07-18T03:29:06.931 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.932 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.932 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.932 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.932 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.932 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.933 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.933 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 30, 2020-07-18T03:29:06.933 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.933 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.933 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6808", 2020-07-18T03:29:06.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6809", 2020-07-18T03:29:06.935 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.936 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.936 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6810", 2020-07-18T03:29:06.937 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6811", 2020-07-18T03:29:06.938 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.938 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.938 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.939 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6814", 2020-07-18T03:29:06.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6815", 2020-07-18T03:29:06.940 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6812", 2020-07-18T03:29:06.942 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.943 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6813", 2020-07-18T03:29:06.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:06.943 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.943 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.944 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6809/24092", 2020-07-18T03:29:06.944 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6811/24092", 2020-07-18T03:29:06.944 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6815/24092", 2020-07-18T03:29:06.944 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6813/24092", 2020-07-18T03:29:06.945 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.945 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.945 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.945 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.946 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:29:06.946 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", 2020-07-18T03:29:06.946 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.946 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.946 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.947 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.947 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.947 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 31, 2020-07-18T03:29:06.948 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.948 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.948 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6800", 2020-07-18T03:29:06.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6801", 2020-07-18T03:29:06.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.950 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.951 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6802", 2020-07-18T03:29:06.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6803", 2020-07-18T03:29:06.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.953 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.953 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6806", 2020-07-18T03:29:06.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6807", 2020-07-18T03:29:06.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.956 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.956 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6804", 2020-07-18T03:29:06.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6805", 2020-07-18T03:29:06.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:06.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.958 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.959 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6801/24091", 2020-07-18T03:29:06.959 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6803/24091", 2020-07-18T03:29:06.959 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6807/24091", 2020-07-18T03:29:06.959 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6805/24091", 2020-07-18T03:29:06.959 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.959 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.960 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.960 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:29:06.961 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", 2020-07-18T03:29:06.961 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.961 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.961 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.961 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.962 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.962 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.962 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.962 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 31, 2020-07-18T03:29:06.962 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.962 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6816", 2020-07-18T03:29:06.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6817", 2020-07-18T03:29:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6819", 2020-07-18T03:29:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6821", 2020-07-18T03:29:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6828", 2020-07-18T03:29:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6830", 2020-07-18T03:29:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6823", 2020-07-18T03:29:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6826", 2020-07-18T03:29:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6817/24093", 2020-07-18T03:29:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6821/24093", 2020-07-18T03:29:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6830/24093", 2020-07-18T03:29:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6826/24093", 2020-07-18T03:29:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:29:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", 2020-07-18T03:29:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 31, 2020-07-18T03:29:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6818", 2020-07-18T03:29:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6820", 2020-07-18T03:29:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6822", 2020-07-18T03:29:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6824", 2020-07-18T03:29:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6829", 2020-07-18T03:29:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6831", 2020-07-18T03:29:06.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.987 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.988 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:06.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:06.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:06.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6825", 2020-07-18T03:29:06.988 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:06.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6827", 2020-07-18T03:29:06.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:06.990 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.990 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.990 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6820/24094", 2020-07-18T03:29:06.990 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6824/24094", 2020-07-18T03:29:06.991 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6831/24094", 2020-07-18T03:29:06.991 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6827/24094", 2020-07-18T03:29:06.991 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:06.991 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:06.991 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:06.992 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:06.992 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:06.992 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:06.993 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2020-07-18T03:29:06.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.993 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:29:06.993 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:06.993 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.933644+0000", 2020-07-18T03:29:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.995 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:29:06.995 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:06.995 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:06.995 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:06.996 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:06.996 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:06.996 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.929073+0000", 2020-07-18T03:29:06.996 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:06.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.997 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:29:06.997 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:06.997 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:06.997 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:06.997 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:06.998 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:06.998 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.974285+0000", 2020-07-18T03:29:06.998 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:06.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:06.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:06.999 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:29:06.999 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:06.999 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:06.999 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:06.999 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:06.999 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:07.000 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.909674+0000", 2020-07-18T03:29:07.000 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:07.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:07.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:07.000 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:29:07.001 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:07.001 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:07.001 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:07.001 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:07.001 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:07.002 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.935590+0000", 2020-07-18T03:29:07.002 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:07.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:07.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:07.002 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:29:07.003 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:07.003 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:07.003 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:07.003 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:07.003 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:07.004 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.916457+0000", 2020-07-18T03:29:07.004 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:07.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:07.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:07.005 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:29:07.005 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:07.005 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:07.005 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:07.005 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:07.006 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:07.006 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.990672+0000", 2020-07-18T03:29:07.006 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:07.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:07.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:07.007 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:29:07.007 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:07.007 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:07.007 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:07.007 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:07.008 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:07.008 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.948777+0000", 2020-07-18T03:29:07.008 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:07.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:07.008 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:07.008 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2020-07-18T03:29:07.009 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2020-07-18T03:29:07.009 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2020-07-18T03:29:07.009 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2020-07-18T03:29:07.009 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2020-07-18T03:29:07.009 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2020-07-18T03:29:07.010 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2020-07-18T03:29:07.010 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2020-07-18T03:29:07.010 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:29:07.010 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2020-07-18T03:29:07.010 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2020-07-18T03:29:07.011 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:29:07.011 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2020-07-18T03:29:07.011 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:29:07.011 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:29:07.011 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2020-07-18T03:29:07.011 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2020-07-18T03:29:07.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:07.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2020-07-18T03:29:07.012 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:29:07.012 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:29:07.012 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2020-07-18T03:29:07.013 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:29:07.013 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:29:07.013 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2020-07-18T03:29:07.013 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:07.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:07.013 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2020-07-18T03:29:07.014 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2020-07-18T03:29:07.014 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2020-07-18T03:29:07.014 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2020-07-18T03:29:07.014 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2020-07-18T03:29:07.014 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f72c404ff30 msgr2=0x7f72c40523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f72c404ff30 0x7f72c40523f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f72ec140790 tx=0x7f72e4016000).stop 2020-07-18T03:29:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72ec158a50 msgr2=0x7f72ec13a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72ec158a50 0x7f72ec13a5e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f72dc00c000 tx=0x7f72dc006dd0).stop 2020-07-18T03:29:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 shutdown_connections 2020-07-18T03:29:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f72c404ff30 0x7f72c40523f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 --2- 172.21.15.47:0/3454854258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72ec158a50 0x7f72ec13a5e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 --2- 172.21.15.47:0/3454854258 >> v2:172.21.15.18:3300/0 conn(0x7f72ec0b0950 0x7f72ec143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 >> 172.21.15.47:0/3454854258 conn(0x7f72ec0b9e50 msgr2=0x7f72ec0b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 shutdown_connections 2020-07-18T03:29:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:06.800+0000 7f72f270d700 1 -- 172.21.15.47:0/3454854258 wait complete. 2020-07-18T03:29:07.019 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.133+0000 7f3c89f64700 1 Processor -- start 2020-07-18T03:29:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.134+0000 7f3c89f64700 1 -- start start 2020-07-18T03:29:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.134+0000 7f3c89f64700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3c840ad100 0x7f3c840b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.134+0000 7f3c89f64700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c84147a00 0x7f3c84147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.134+0000 7f3c89f64700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c84147330 con 0x7f3c8405f5a0 2020-07-18T03:29:07.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.134+0000 7f3c89f64700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c840ae530 con 0x7f3c840ad100 2020-07-18T03:29:07.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.134+0000 7f3c89f64700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b57e0 con 0x7f3c84147a00 2020-07-18T03:29:07.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3c840ad100 0x7f3c840b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c82ffd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c84147a00 0x7f3c84147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c83fff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f3c8405f5a0 0x7f3c8405fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60368/0 (socket says 172.21.15.47:60368) 2020-07-18T03:29:07.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3c840ad100 0x7f3c840b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:46194/0 (socket says 172.21.15.47:46194) 2020-07-18T03:29:07.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c83fff700 1 -- 172.21.15.47:0/312188074 learned_addr learned my addr 172.21.15.47:0/312188074 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 -- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c84147a00 msgr2=0x7f3c84147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 --2- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c84147a00 0x7f3c84147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 -- 172.21.15.47:0/312188074 >> v1:172.21.15.18:6789/0 conn(0x7f3c8405f5a0 legacy=0x7f3c8405fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 -- 172.21.15.47:0/312188074 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840b5a80 con 0x7f3c840ad100 2020-07-18T03:29:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c827fc700 1 -- 172.21.15.47:0/312188074 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983626378 0 0) 0x7f3c84147330 con 0x7f3c8405f5a0 2020-07-18T03:29:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c837fe700 1 --2- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c840ad100 0x7f3c840b4c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f3c74002f30 tx=0x7f3c74006dd0).ready entity=mon.0 client_cookie=15ae30f66769260 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.135+0000 7f3c827fc700 1 -- 172.21.15.47:0/312188074 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c74009760 con 0x7f3c840ad100 2020-07-18T03:29:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c827fc700 1 -- 172.21.15.47:0/312188074 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c740098c0 con 0x7f3c840ad100 2020-07-18T03:29:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c827fc700 1 -- 172.21.15.47:0/312188074 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c74009c20 con 0x7f3c840ad100 2020-07-18T03:29:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 -- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c840ad100 msgr2=0x7f3c840b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 --2- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c840ad100 0x7f3c840b4c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f3c74002f30 tx=0x7f3c74006dd0).stop 2020-07-18T03:29:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 -- 172.21.15.47:0/312188074 shutdown_connections 2020-07-18T03:29:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 --2- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c840ad100 0x7f3c840b4c10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 --2- 172.21.15.47:0/312188074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c84147a00 0x7f3c84147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 -- 172.21.15.47:0/312188074 >> 172.21.15.47:0/312188074 conn(0x7f3c840b9340 msgr2=0x7f3c840b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 -- 172.21.15.47:0/312188074 shutdown_connections 2020-07-18T03:29:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.136+0000 7f3c89f64700 1 -- 172.21.15.47:0/312188074 wait complete. 2020-07-18T03:29:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.137+0000 7f3c89f64700 1 Processor -- start 2020-07-18T03:29:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.137+0000 7f3c89f64700 1 -- start start 2020-07-18T03:29:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c89f64700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8405f5a0 0x7f3c840660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c89f64700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3c84147a00 0x7f3c8413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c89f64700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8413fd50 con 0x7f3c8413d010 2020-07-18T03:29:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c89f64700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c840b1a40 con 0x7f3c84147a00 2020-07-18T03:29:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c89f64700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b1ce0 con 0x7f3c8405f5a0 2020-07-18T03:29:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c83fff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f3c8413d010 0x7f3c8413ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60374/0 (socket says 172.21.15.47:60374) 2020-07-18T03:29:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c83fff700 1 -- 172.21.15.47:0/3616713921 learned_addr learned my addr 172.21.15.47:0/3616713921 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c837fe700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8405f5a0 0x7f3c840660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c82ffd700 1 --2- 172.21.15.47:0/3616713921 >> v2:172.21.15.18:3300/0 conn(0x7f3c84147a00 0x7f3c8413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695597596 0 0) 0x7f3c8413fd50 con 0x7f3c8413d010 2020-07-18T03:29:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.138+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c5c003240 con 0x7f3c8413d010 2020-07-18T03:29:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c837fe700 1 -- 172.21.15.47:0/3616713921 >> v2:172.21.15.18:3300/0 conn(0x7f3c84147a00 msgr2=0x7f3c8413f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c837fe700 1 --2- 172.21.15.47:0/3616713921 >> v2:172.21.15.18:3300/0 conn(0x7f3c84147a00 0x7f3c8413f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c837fe700 1 -- 172.21.15.47:0/3616713921 >> v1:172.21.15.18:6789/0 conn(0x7f3c8413d010 legacy=0x7f3c8413ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c837fe700 1 -- 172.21.15.47:0/3616713921 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84066590 con 0x7f3c8405f5a0 2020-07-18T03:29:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c837fe700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8405f5a0 0x7f3c840660a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f3c74002f30 tx=0x7f3c74015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c74021070 con 0x7f3c8405f5a0 2020-07-18T03:29:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7401c6e0 con 0x7f3c8405f5a0 2020-07-18T03:29:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8413f9d0 con 0x7f3c8405f5a0 2020-07-18T03:29:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c7401ca40 con 0x7f3c8405f5a0 2020-07-18T03:29:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.139+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8413b700 con 0x7f3c8405f5a0 2020-07-18T03:29:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.140+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c840b18a0 con 0x7f3c8405f5a0 2020-07-18T03:29:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.140+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3c7400f070 con 0x7f3c8405f5a0 2020-07-18T03:29:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.141+0000 7f3c637fe700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c5c050360 0x7f3c5c052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.141+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c74069170 con 0x7f3c8405f5a0 2020-07-18T03:29:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.141+0000 7f3c82ffd700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c5c050360 0x7f3c5c052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.141+0000 7f3c82ffd700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c5c050360 0x7f3c5c052820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f3c7800b000 tx=0x7f3c78006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.144+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3c840b18a0 con 0x7f3c8405f5a0 2020-07-18T03:29:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.355+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c8413b700 con 0x7f3c8405f5a0 2020-07-18T03:29:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.356+0000 7f3c637fe700 1 -- 172.21.15.47:0/3616713921 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f3c8413b700 con 0x7f3c8405f5a0 2020-07-18T03:29:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:07.358 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:26:41.771684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5101":{"gid":5101,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3109073551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3109073551},{"type":"v1","addr":"172.21.15.47:6835","nonce":3109073551}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:29:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c5c050360 msgr2=0x7f3c5c052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c5c050360 0x7f3c5c052820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f3c7800b000 tx=0x7f3c78006dd0).stop 2020-07-18T03:29:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8405f5a0 msgr2=0x7f3c840660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8405f5a0 0x7f3c840660a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f3c74002f30 tx=0x7f3c74015000).stop 2020-07-18T03:29:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 shutdown_connections 2020-07-18T03:29:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c5c050360 0x7f3c5c052820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 --2- 172.21.15.47:0/3616713921 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8405f5a0 0x7f3c840660a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 --2- 172.21.15.47:0/3616713921 >> v2:172.21.15.18:3300/0 conn(0x7f3c84147a00 0x7f3c8413f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 >> 172.21.15.47:0/3616713921 conn(0x7f3c840b9340 msgr2=0x7f3c84060200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 shutdown_connections 2020-07-18T03:29:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.358+0000 7f3c89f64700 1 -- 172.21.15.47:0/3616713921 wait complete. 2020-07-18T03:29:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2020-07-18T03:29:07.370 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-07-18T03:29:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.483+0000 7f3db8e4e700 1 Processor -- start 2020-07-18T03:29:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.483+0000 7f3db8e4e700 1 -- start start 2020-07-18T03:29:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db8e4e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3db40ad020 0x7f3db40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db8e4e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db4144700 con 0x7f3db40af320 2020-07-18T03:29:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db8e4e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db41449a0 con 0x7f3db413dbc0 2020-07-18T03:29:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db8e4e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db40b59e0 con 0x7f3db40ad020 2020-07-18T03:29:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3db40ad020 0x7f3db40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3db40ad020 0x7f3db40b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60526/0 (socket says 172.21.15.47:60526) 2020-07-18T03:29:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db1d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3db413dbc0 0x7f3db413e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41614/0 (socket says 172.21.15.47:41614) 2020-07-18T03:29:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db259c700 1 -- 172.21.15.47:0/2148453366 learned_addr learned my addr 172.21.15.47:0/2148453366 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db259c700 1 -- 172.21.15.47:0/2148453366 >> v1:172.21.15.47:6789/0 conn(0x7f3db413dbc0 legacy=0x7f3db413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db259c700 1 -- 172.21.15.47:0/2148453366 >> v1:172.21.15.18:6789/0 conn(0x7f3db40af320 legacy=0x7f3db40af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db159a700 1 -- 172.21.15.47:0/2148453366 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170070835 0 0) 0x7f3db41449a0 con 0x7f3db413dbc0 2020-07-18T03:29:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.484+0000 7f3db259c700 1 -- 172.21.15.47:0/2148453366 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db40b5b20 con 0x7f3db40ad020 2020-07-18T03:29:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db259c700 1 --2- 172.21.15.47:0/2148453366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db40ad020 0x7f3db40b54f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3db4060910 tx=0x7f3da8006e90).ready entity=mon.1 client_cookie=9db7534a669fa05b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db159a700 1 -- 172.21.15.47:0/2148453366 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3da8009980 con 0x7f3db40ad020 2020-07-18T03:29:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db159a700 1 -- 172.21.15.47:0/2148453366 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8009ae0 con 0x7f3db40ad020 2020-07-18T03:29:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db159a700 1 -- 172.21.15.47:0/2148453366 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3da8009e40 con 0x7f3db40ad020 2020-07-18T03:29:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 -- 172.21.15.47:0/2148453366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db40ad020 msgr2=0x7f3db40b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/2148453366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db40ad020 0x7f3db40b54f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3db4060910 tx=0x7f3da8006e90).stop 2020-07-18T03:29:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 -- 172.21.15.47:0/2148453366 shutdown_connections 2020-07-18T03:29:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/2148453366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db40ad020 0x7f3db40b54f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 -- 172.21.15.47:0/2148453366 >> 172.21.15.47:0/2148453366 conn(0x7f3db40b93c0 msgr2=0x7f3db40b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 -- 172.21.15.47:0/2148453366 shutdown_connections 2020-07-18T03:29:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.485+0000 7f3db8e4e700 1 -- 172.21.15.47:0/2148453366 wait complete. 2020-07-18T03:29:07.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.486+0000 7f3db8e4e700 1 Processor -- start 2020-07-18T03:29:07.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.486+0000 7f3db8e4e700 1 -- start start 2020-07-18T03:29:07.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db8e4e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3db40af320 0x7f3db414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db8e4e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3db414aeb0 0x7f3db41560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db8e4e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db413f130 con 0x7f3db413dbc0 2020-07-18T03:29:07.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db8e4e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db413f3d0 con 0x7f3db414aeb0 2020-07-18T03:29:07.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db8e4e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db413e950 con 0x7f3db40af320 2020-07-18T03:29:07.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db1d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3db413dbc0 0x7f3db41559a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41622/0 (socket says 172.21.15.47:41622) 2020-07-18T03:29:07.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db1d9b700 1 -- 172.21.15.47:0/678803847 learned_addr learned my addr 172.21.15.47:0/678803847 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db2d9d700 1 --2- 172.21.15.47:0/678803847 >> v2:172.21.15.47:3300/0 conn(0x7f3db414aeb0 0x7f3db41560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db259c700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3db40af320 0x7f3db414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653811095 0 0) 0x7f3db413f130 con 0x7f3db413dbc0 2020-07-18T03:29:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d88003240 con 0x7f3db413dbc0 2020-07-18T03:29:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db2d9d700 1 -- 172.21.15.47:0/678803847 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3db40af320 msgr2=0x7f3db414a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db2d9d700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3db40af320 0x7f3db414a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db2d9d700 1 -- 172.21.15.47:0/678803847 >> v1:172.21.15.47:6789/0 conn(0x7f3db413dbc0 legacy=0x7f3db41559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.487+0000 7f3db2d9d700 1 -- 172.21.15.47:0/678803847 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db41565a0 con 0x7f3db414aeb0 2020-07-18T03:29:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.488+0000 7f3db2d9d700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db414aeb0 0x7f3db41560b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f3dac0022d0 tx=0x7f3dac007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.488+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dac010510 con 0x7f3db414aeb0 2020-07-18T03:29:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.488+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac010670 con 0x7f3db414aeb0 2020-07-18T03:29:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.488+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db413ed50 con 0x7f3db414aeb0 2020-07-18T03:29:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.488+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dac0107e0 con 0x7f3db414aeb0 2020-07-18T03:29:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.488+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db41491a0 con 0x7f3db414aeb0 2020-07-18T03:29:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.489+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db40b0260 con 0x7f3db414aeb0 2020-07-18T03:29:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.489+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3dac010070 con 0x7f3db414aeb0 2020-07-18T03:29:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.489+0000 7f3da27fc700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d88050370 0x7f3d88052830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.489+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3dac067610 con 0x7f3db414aeb0 2020-07-18T03:29:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.489+0000 7f3db259c700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d88050370 0x7f3d88052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.490+0000 7f3db259c700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d88050370 0x7f3d88052830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3da801c000 tx=0x7f3da8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.493+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3dac03fd20 con 0x7f3db414aeb0 2020-07-18T03:29:07.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:07.702+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3db413f510 con 0x7f3db414aeb0 2020-07-18T03:29:08.378 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:29:08.376+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:29:08.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.380+0000 7f3da27fc700 1 -- 172.21.15.47:0/678803847 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3dac043340 con 0x7f3db414aeb0 2020-07-18T03:29:08.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.382+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d88050370 msgr2=0x7f3d88052830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.382+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d88050370 0x7f3d88052830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3da801c000 tx=0x7f3da8015000).stop 2020-07-18T03:29:08.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db414aeb0 msgr2=0x7f3db41560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db414aeb0 0x7f3db41560b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f3dac0022d0 tx=0x7f3dac007e40).stop 2020-07-18T03:29:08.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 shutdown_connections 2020-07-18T03:29:08.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3db414aeb0 0x7f3db41560b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3db40af320 0x7f3db414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 --2- 172.21.15.47:0/678803847 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3d88050370 0x7f3d88052830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 >> 172.21.15.47:0/678803847 conn(0x7f3db40b93c0 msgr2=0x7f3db40b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 shutdown_connections 2020-07-18T03:29:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.383+0000 7f3db8e4e700 1 -- 172.21.15.47:0/678803847 wait complete. 2020-07-18T03:29:08.387 INFO:tasks.ceph.mds.a.smithi047.stderr: -3830> 2020-07-18T03:26:45.387+0000 7f3d1ccfa700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:29:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-07-18T03:29:08.398 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:08.411 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:29:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.519+0000 7f5134a6c700 1 Processor -- start 2020-07-18T03:29:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.519+0000 7f5134a6c700 1 -- start start 2020-07-18T03:29:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f5134a6c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 0x7f5130147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f5134a6c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f51301433e0 0x7f5130143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f5134a6c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5130147330 con 0x7f51300ad100 2020-07-18T03:29:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f5134a6c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300ae530 con 0x7f51301433e0 2020-07-18T03:29:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f5134a6c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51300b57e0 con 0x7f5130147a00 2020-07-18T03:29:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f512e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51300ad100 0x7f51300b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41640/0 (socket says 172.21.15.47:41640) 2020-07-18T03:29:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f512e59c700 1 -- 172.21.15.47:0/380060375 learned_addr learned my addr 172.21.15.47:0/380060375 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f512ed9d700 1 --2- 172.21.15.47:0/380060375 >> v2:172.21.15.18:3300/0 conn(0x7f51301433e0 0x7f5130143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f512dd9b700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 0x7f5130147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623362846 0 0) 0x7f5130147330 con 0x7f51300ad100 2020-07-18T03:29:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.520+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5114003240 con 0x7f51300ad100 2020-07-18T03:29:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 431061214 0 0) 0x7f5114003240 con 0x7f51300ad100 2020-07-18T03:29:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 msgr2=0x7f5130147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 0x7f5130147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 >> v2:172.21.15.18:3300/0 conn(0x7f51301433e0 msgr2=0x7f5130143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 --2- 172.21.15.47:0/380060375 >> v2:172.21.15.18:3300/0 conn(0x7f51301433e0 0x7f5130143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300b5a80 con 0x7f51300ad100 2020-07-18T03:29:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f51240016d0 con 0x7f51300ad100 2020-07-18T03:29:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 >> v1:172.21.15.47:6789/0 conn(0x7f51300ad100 legacy=0x7f51300b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114003730 0x7f5114003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51140044e0 0x7f5114004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 0x7f5114004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300ae530 con 0x7f5114003730 2020-07-18T03:29:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300b57e0 con 0x7f51140044e0 2020-07-18T03:29:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5114004270 con 0x7f5130147a00 2020-07-18T03:29:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512ed9d700 1 --2- 172.21.15.47:0/380060375 >> v2:172.21.15.18:3300/0 conn(0x7f51301433e0 0x7f5130143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51140044e0 0x7f5114004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512e59c700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114003730 0x7f5114003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512ed9d700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 0x7f5114004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 -- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 msgr2=0x7f5114004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5130147a00 0x7f5114004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 -- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114003730 msgr2=0x7f5114003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114003730 0x7f5114003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 -- 172.21.15.47:0/380060375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114003240 con 0x7f51140044e0 2020-07-18T03:29:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512dd9b700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51140044e0 0x7f5114004960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f512000c000 tx=0x7f5120008040).ready entity=mon.1 client_cookie=ecef40a36b401eef server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f512000b2d0 con 0x7f51140044e0 2020-07-18T03:29:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512000e8a0 con 0x7f51140044e0 2020-07-18T03:29:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.521+0000 7f512d59a700 1 -- 172.21.15.47:0/380060375 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f512000ec00 con 0x7f51140044e0 2020-07-18T03:29:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f5134a6c700 1 -- 172.21.15.47:0/380060375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51140044e0 msgr2=0x7f5114004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f5134a6c700 1 --2- 172.21.15.47:0/380060375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51140044e0 0x7f5114004960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f512000c000 tx=0x7f5120008040).stop 2020-07-18T03:29:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f512ed9d700 1 -- 172.21.15.47:0/380060375 reap_dead start 2020-07-18T03:29:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f5134a6c700 1 -- 172.21.15.47:0/380060375 shutdown_connections 2020-07-18T03:29:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f5134a6c700 1 -- 172.21.15.47:0/380060375 >> 172.21.15.47:0/380060375 conn(0x7f51300b9340 msgr2=0x7f51300b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f5134a6c700 1 -- 172.21.15.47:0/380060375 shutdown_connections 2020-07-18T03:29:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.522+0000 7f5134a6c700 1 -- 172.21.15.47:0/380060375 wait complete. 2020-07-18T03:29:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.523+0000 7f5134a6c700 1 Processor -- start 2020-07-18T03:29:08.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.523+0000 7f5134a6c700 1 -- start start 2020-07-18T03:29:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f5134a6c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51300b5e40 0x7f5130066130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f5134a6c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f51300b6220 0x7f5130141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f5134a6c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300b2650 con 0x7f5130142170 2020-07-18T03:29:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f5134a6c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51301425c0 con 0x7f51300b6220 2020-07-18T03:29:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f5134a6c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300b2880 con 0x7f51300b5e40 2020-07-18T03:29:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51300b5e40 0x7f5130066130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51300b5e40 0x7f5130066130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60562/0 (socket says 172.21.15.47:60562) 2020-07-18T03:29:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 -- 172.21.15.47:0/1276098155 learned_addr learned my addr 172.21.15.47:0/1276098155 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512dd9b700 1 --2- 172.21.15.47:0/1276098155 >> v2:172.21.15.18:3300/0 conn(0x7f51300b6220 0x7f5130141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 -- 172.21.15.47:0/1276098155 >> v2:172.21.15.18:3300/0 conn(0x7f51300b6220 msgr2=0x7f5130141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 --2- 172.21.15.47:0/1276098155 >> v2:172.21.15.18:3300/0 conn(0x7f51300b6220 0x7f5130141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 -- 172.21.15.47:0/1276098155 >> v1:172.21.15.47:6789/0 conn(0x7f5130142170 legacy=0x7f5130153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 -- 172.21.15.47:0/1276098155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300b2b20 con 0x7f51300b5e40 2020-07-18T03:29:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f512e59c700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51300b5e40 0x7f5130066130 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f512400d000 tx=0x7f51300b03e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f512400fa90 con 0x7f51300b5e40 2020-07-18T03:29:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.524+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400fbf0 con 0x7f51300b5e40 2020-07-18T03:29:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.525+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300b2dc0 con 0x7f51300b5e40 2020-07-18T03:29:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.525+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5124012350 con 0x7f51300b5e40 2020-07-18T03:29:08.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.525+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130062ca0 con 0x7f51300b5e40 2020-07-18T03:29:08.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.526+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130057450 con 0x7f51300b5e40 2020-07-18T03:29:08.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.526+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f512401b030 con 0x7f51300b5e40 2020-07-18T03:29:08.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.526+0000 7f511e7fc700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f510404ffb0 0x7f5104052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.526+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5124067740 con 0x7f51300b5e40 2020-07-18T03:29:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.526+0000 7f512dd9b700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f510404ffb0 0x7f5104052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.526+0000 7f512dd9b700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f510404ffb0 0x7f5104052470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5120008000 tx=0x7f512001a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.529+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f512403fe80 con 0x7f51300b5e40 2020-07-18T03:29:08.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.737+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f513006a100 con 0x7f51300b5e40 2020-07-18T03:29:08.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.738+0000 7f511e7fc700 1 -- 172.21.15.47:0/1276098155 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+1772 (secure 0 0 0) 0x7f5124019020 con 0x7f51300b5e40 2020-07-18T03:29:08.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:08.741 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4592,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/1757326790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1757326790},{"type":"v1","addr":"172.21.15.18:6833","nonce":1757326790}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:26:40.770533+0000","modified":"2020-07-18T03:29:08.377610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-07-18T03:29:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f510404ffb0 msgr2=0x7f5104052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f510404ffb0 0x7f5104052470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5120008000 tx=0x7f512001a000).stop 2020-07-18T03:29:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51300b5e40 msgr2=0x7f5130066130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51300b5e40 0x7f5130066130 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f512400d000 tx=0x7f51300b03e0).stop 2020-07-18T03:29:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 shutdown_connections 2020-07-18T03:29:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f510404ffb0 0x7f5104052470 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 --2- 172.21.15.47:0/1276098155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51300b5e40 0x7f5130066130 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 --2- 172.21.15.47:0/1276098155 >> v2:172.21.15.18:3300/0 conn(0x7f51300b6220 0x7f5130141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 >> 172.21.15.47:0/1276098155 conn(0x7f51300b9340 msgr2=0x7f51301469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:08.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 shutdown_connections 2020-07-18T03:29:08.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.739+0000 7f5134a6c700 1 -- 172.21.15.47:0/1276098155 wait complete. 2020-07-18T03:29:08.747 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2020-07-18T03:29:08.752 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-07-18T03:29:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.912+0000 7f11038bd700 1 Processor -- start 2020-07-18T03:29:08.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.912+0000 7f11038bd700 1 -- start start 2020-07-18T03:29:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.913+0000 7f11038bd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10fc0b59b0 0x7f10fc0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.913+0000 7f11038bd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc147de0 con 0x7f10fc0ad330 2020-07-18T03:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.913+0000 7f11038bd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc0ae000 con 0x7f10fc0b6300 2020-07-18T03:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.913+0000 7f11038bd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc148010 con 0x7f10fc0b59b0 2020-07-18T03:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.913+0000 7f1101e5a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10fc0b6300 0x7f10fc0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41662/0 (socket says 172.21.15.47:41662) 2020-07-18T03:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.913+0000 7f1101e5a700 1 -- 172.21.15.47:0/1224278103 learned_addr learned my addr 172.21.15.47:0/1224278103 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15197776 0 0) 0x7f10fc0ae000 con 0x7f10fc0b6300 2020-07-18T03:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1100e58700 1 --2- 172.21.15.47:0/1224278103 >> v2:172.21.15.18:3300/0 conn(0x7f10fc0b59b0 0x7f10fc0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4003240 con 0x7f10fc0b6300 2020-07-18T03:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3384856490 0 0) 0x7f10e4003240 con 0x7f10fc0b6300 2020-07-18T03:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 >> v2:172.21.15.18:3300/0 conn(0x7f10fc0b59b0 msgr2=0x7f10fc0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 --2- 172.21.15.47:0/1224278103 >> v2:172.21.15.18:3300/0 conn(0x7f10fc0b59b0 0x7f10fc0b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 >> v1:172.21.15.18:6789/0 conn(0x7f10fc0ad330 legacy=0x7f10fc143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0ad710 con 0x7f10fc0b6300 2020-07-18T03:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f10f4001a50 con 0x7f10fc0b6300 2020-07-18T03:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 >> v1:172.21.15.47:6789/0 conn(0x7f10fc0b6300 legacy=0x7f10fc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e4003730 0x7f10e4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10e40044e0 0x7f10e4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4004e50 0x7f10e400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc148010 con 0x7f10e40044e0 2020-07-18T03:29:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4004350 con 0x7f10e4004e50 2020-07-18T03:29:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400bc50 con 0x7f10e4003730 2020-07-18T03:29:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101e5a700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4004e50 0x7f10e400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1100e58700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10e40044e0 0x7f10e4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101659700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e4003730 0x7f10e4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101e5a700 1 -- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e4003730 msgr2=0x7f10e4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101e5a700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e4003730 0x7f10e4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101e5a700 1 -- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10e40044e0 msgr2=0x7f10e4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101e5a700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10e40044e0 0x7f10e4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1101e5a700 1 -- 172.21.15.47:0/1224278103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4003240 con 0x7f10e4004e50 2020-07-18T03:29:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.914+0000 7f1100e58700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10e40044e0 0x7f10e4004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f1101e5a700 1 -- 172.21.15.47:0/1224278103 reap_dead start 2020-07-18T03:29:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f1101e5a700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4004e50 0x7f10e400b730 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f10f4001760 tx=0x7f10f4007ad0).ready entity=mon.1 client_cookie=99c40c2444e49952 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10f4005b80 con 0x7f10e4004e50 2020-07-18T03:29:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4005ce0 con 0x7f10e4004e50 2020-07-18T03:29:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f10ebfff700 1 -- 172.21.15.47:0/1224278103 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10f40072c0 con 0x7f10e4004e50 2020-07-18T03:29:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 -- 172.21.15.47:0/1224278103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4004e50 msgr2=0x7f10e400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4004e50 0x7f10e400b730 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f10f4001760 tx=0x7f10f4007ad0).stop 2020-07-18T03:29:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 -- 172.21.15.47:0/1224278103 shutdown_connections 2020-07-18T03:29:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 --2- 172.21.15.47:0/1224278103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4004e50 0x7f10e400b730 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 -- 172.21.15.47:0/1224278103 >> 172.21.15.47:0/1224278103 conn(0x7f10fc05f5a0 msgr2=0x7f10fc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 -- 172.21.15.47:0/1224278103 shutdown_connections 2020-07-18T03:29:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.915+0000 7f11038bd700 1 -- 172.21.15.47:0/1224278103 wait complete. 2020-07-18T03:29:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.916+0000 7f11038bd700 1 Processor -- start 2020-07-18T03:29:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.916+0000 7f11038bd700 1 -- start start 2020-07-18T03:29:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f11038bd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10fc1405a0 0x7f10fc0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f11038bd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc062e50 con 0x7f10fc146850 2020-07-18T03:29:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f11038bd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc0630f0 con 0x7f10fc140980 2020-07-18T03:29:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f11038bd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc0b36a0 con 0x7f10fc1405a0 2020-07-18T03:29:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f1100e58700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10fc140980 0x7f10fc13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41672/0 (socket says 172.21.15.47:41672) 2020-07-18T03:29:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f1100e58700 1 -- 172.21.15.47:0/4117984278 learned_addr learned my addr 172.21.15.47:0/4117984278 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057515027 0 0) 0x7f10fc0630f0 con 0x7f10fc140980 2020-07-18T03:29:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.917+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10d4003240 con 0x7f10fc140980 2020-07-18T03:29:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101659700 1 --2- 172.21.15.47:0/4117984278 >> v2:172.21.15.18:3300/0 conn(0x7f10fc1405a0 0x7f10fc0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1534685344 0 0) 0x7f10d4003240 con 0x7f10fc140980 2020-07-18T03:29:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10fc0630f0 con 0x7f10fc140980 2020-07-18T03:29:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f10f800bca0 con 0x7f10fc140980 2020-07-18T03:29:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 >> v2:172.21.15.18:3300/0 conn(0x7f10fc1405a0 msgr2=0x7f10fc0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 --2- 172.21.15.47:0/4117984278 >> v2:172.21.15.18:3300/0 conn(0x7f10fc1405a0 0x7f10fc0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 >> v1:172.21.15.47:6789/0 conn(0x7f10fc140980 legacy=0x7f10fc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 >> v1:172.21.15.18:6789/0 conn(0x7f10fc146850 legacy=0x7f10fc0b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10d4003800 0x7f10d4003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d40044e0 0x7f10d4004960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d4004e50 0x7f10d400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b36a0 con 0x7f10d4003800 2020-07-18T03:29:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4004280 con 0x7f10d4004e50 2020-07-18T03:29:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d400bc20 con 0x7f10d40044e0 2020-07-18T03:29:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d4004e50 0x7f10d400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1100e58700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d40044e0 0x7f10d4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101659700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10d4003800 0x7f10d4003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 -- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d40044e0 msgr2=0x7f10d4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d40044e0 0x7f10d4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 -- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10d4003800 msgr2=0x7f10d4003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10d4003800 0x7f10d4003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0b8190 con 0x7f10d4004e50 2020-07-18T03:29:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc1563a0 con 0x7f10d4004e50 2020-07-18T03:29:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc156810 con 0x7f10d4004e50 2020-07-18T03:29:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.918+0000 7f1101e5a700 1 -- 172.21.15.47:0/4117984278 reap_dead start 2020-07-18T03:29:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.919+0000 7f1101e5a700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d4004e50 0x7f10d400b730 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f10f4001510 tx=0x7f10f4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.919+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10f4005b80 con 0x7f10d4004e50 2020-07-18T03:29:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.919+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4005e50 con 0x7f10d4004e50 2020-07-18T03:29:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.919+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10f4020480 con 0x7f10d4004e50 2020-07-18T03:29:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.919+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10fc05b120 con 0x7f10d4004e50 2020-07-18T03:29:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.920+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f10f4010070 con 0x7f10d4004e50 2020-07-18T03:29:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.920+0000 7f10e97fa700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10d4053380 0x7f10d4055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.920+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f10f4069160 con 0x7f10d4004e50 2020-07-18T03:29:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.920+0000 7f1101659700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10d4053380 0x7f10d4055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.920+0000 7f1101659700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10d4053380 0x7f10d4055840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f10ec003000 tx=0x7f10ec007b70).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:08.923+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f10f4041820 con 0x7f10d4004e50 2020-07-18T03:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.139+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f10fc0690d0 con 0x7f10d4004e50 2020-07-18T03:29:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.384+0000 7f10e97fa700 1 -- 172.21.15.47:0/4117984278 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f10f4044e40 con 0x7f10d4004e50 2020-07-18T03:29:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.385+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10d4053380 msgr2=0x7f10d4055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.385+0000 7f11038bd700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10d4053380 0x7f10d4055840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f10ec003000 tx=0x7f10ec007b70).stop 2020-07-18T03:29:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.385+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d4004e50 msgr2=0x7f10d400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.385+0000 7f11038bd700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d4004e50 0x7f10d400b730 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f10f4001510 tx=0x7f10f4015000).stop 2020-07-18T03:29:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.386+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 shutdown_connections 2020-07-18T03:29:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.386+0000 7f11038bd700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10d4053380 0x7f10d4055840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.386+0000 7f11038bd700 1 --2- 172.21.15.47:0/4117984278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d4004e50 0x7f10d400b730 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.386+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 >> 172.21.15.47:0/4117984278 conn(0x7f10fc05f5a0 msgr2=0x7f10fc0afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.386+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 shutdown_connections 2020-07-18T03:29:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.386+0000 7f11038bd700 1 -- 172.21.15.47:0/4117984278 wait complete. 2020-07-18T03:29:09.400 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-07-18T03:29:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.518+0000 7fd705ba1700 1 Processor -- start 2020-07-18T03:29:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.518+0000 7fd705ba1700 1 -- start start 2020-07-18T03:29:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.518+0000 7fd705ba1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 0x7fd700143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd705ba1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd7000b3a10 0x7fd7000b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd705ba1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd700147de0 con 0x7fd7000b59b0 2020-07-18T03:29:09.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd705ba1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7000ae000 con 0x7fd7000b3a10 2020-07-18T03:29:09.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd705ba1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd700148010 con 0x7fd7000ad330 2020-07-18T03:29:09.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6feffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd7000b59b0 0x7fd7000b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41686/0 (socket says 172.21.15.47:41686) 2020-07-18T03:29:09.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6feffd700 1 -- 172.21.15.47:0/2106184355 learned_addr learned my addr 172.21.15.47:0/2106184355 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:09.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fffff700 1 --2- 172.21.15.47:0/2106184355 >> v2:172.21.15.18:3300/0 conn(0x7fd7000b3a10 0x7fd7000b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6ff7fe700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 0x7fd700143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569422278 0 0) 0x7fd700147de0 con 0x7fd7000b59b0 2020-07-18T03:29:09.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ec003240 con 0x7fd7000b59b0 2020-07-18T03:29:09.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3780462671 0 0) 0x7fd6ec003240 con 0x7fd7000b59b0 2020-07-18T03:29:09.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 msgr2=0x7fd700143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:09.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 0x7fd700143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 >> v2:172.21.15.18:3300/0 conn(0x7fd7000b3a10 msgr2=0x7fd7000b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:09.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 --2- 172.21.15.47:0/2106184355 >> v2:172.21.15.18:3300/0 conn(0x7fd7000b3a10 0x7fd7000b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7000ad710 con 0x7fd7000b59b0 2020-07-18T03:29:09.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fd6f40016d0 con 0x7fd7000b59b0 2020-07-18T03:29:09.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 >> v1:172.21.15.47:6789/0 conn(0x7fd7000b59b0 legacy=0x7fd7000b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:09.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 0x7fd6ec003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6ec003c40 0x7fd6ec0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec0049d0 0x7fd6ec004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000ae000 con 0x7fd6ec003c40 2020-07-18T03:29:09.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700148010 con 0x7fd6ec0049d0 2020-07-18T03:29:09.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.519+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec004200 con 0x7fd7000ad330 2020-07-18T03:29:09.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec0049d0 0x7fd6ec004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6ff7fe700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 0x7fd6ec003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6feffd700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6ec003c40 0x7fd6ec0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 -- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 msgr2=0x7fd6ec003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:09.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd7000ad330 0x7fd6ec003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 -- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6ec003c40 msgr2=0x7fd6ec0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:09.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6ec003c40 0x7fd6ec0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 -- 172.21.15.47:0/2106184355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec003240 con 0x7fd6ec0049d0 2020-07-18T03:29:09.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec0049d0 0x7fd6ec004e50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fd6f800c000 tx=0x7fd6f80091d0).ready entity=mon.1 client_cookie=1fde746c0be18f35 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:09.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6f800ec30 con 0x7fd6ec0049d0 2020-07-18T03:29:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f800ed90 con 0x7fd6ec0049d0 2020-07-18T03:29:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fe7fc700 1 -- 172.21.15.47:0/2106184355 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6f80125e0 con 0x7fd6ec0049d0 2020-07-18T03:29:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd705ba1700 1 -- 172.21.15.47:0/2106184355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec0049d0 msgr2=0x7fd6ec004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd705ba1700 1 --2- 172.21.15.47:0/2106184355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec0049d0 0x7fd6ec004e50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fd6f800c000 tx=0x7fd6f80091d0).stop 2020-07-18T03:29:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd6fffff700 1 -- 172.21.15.47:0/2106184355 reap_dead start 2020-07-18T03:29:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd705ba1700 1 -- 172.21.15.47:0/2106184355 shutdown_connections 2020-07-18T03:29:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd705ba1700 1 -- 172.21.15.47:0/2106184355 >> 172.21.15.47:0/2106184355 conn(0x7fd70005f5a0 msgr2=0x7fd70005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd705ba1700 1 -- 172.21.15.47:0/2106184355 shutdown_connections 2020-07-18T03:29:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.520+0000 7fd705ba1700 1 -- 172.21.15.47:0/2106184355 wait complete. 2020-07-18T03:29:09.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.522+0000 7fd705ba1700 1 Processor -- start 2020-07-18T03:29:09.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.522+0000 7fd705ba1700 1 -- start start 2020-07-18T03:29:09.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd705ba1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd70013b790 0x7fd700143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd705ba1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd700139c10 0x7fd700138840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd705ba1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd700143f60 con 0x7fd70013bb70 2020-07-18T03:29:09.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd705ba1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd700137c90 con 0x7fd70013b790 2020-07-18T03:29:09.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd705ba1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd700144200 con 0x7fd700139c10 2020-07-18T03:29:09.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6feffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd70013bb70 0x7fd700138130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41700/0 (socket says 172.21.15.47:41700) 2020-07-18T03:29:09.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6feffd700 1 -- 172.21.15.47:0/1777758711 learned_addr learned my addr 172.21.15.47:0/1777758711 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:09.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6ff7fe700 1 --2- 172.21.15.47:0/1777758711 >> v2:172.21.15.18:3300/0 conn(0x7fd70013b790 0x7fd700143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6fffff700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd700139c10 0x7fd700138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940966513 0 0) 0x7fd700143f60 con 0x7fd70013bb70 2020-07-18T03:29:09.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8003240 con 0x7fd70013bb70 2020-07-18T03:29:09.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6ff7fe700 1 -- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd700139c10 msgr2=0x7fd700138840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:09.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6ff7fe700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd700139c10 0x7fd700138840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:09.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6ff7fe700 1 -- 172.21.15.47:0/1777758711 >> v1:172.21.15.47:6789/0 conn(0x7fd70013bb70 legacy=0x7fd700138130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:09.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6ff7fe700 1 -- 172.21.15.47:0/1777758711 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700138d30 con 0x7fd70013b790 2020-07-18T03:29:09.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6fffff700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd700139c10 0x7fd700138840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:09.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6ff7fe700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd70013b790 0x7fd700143850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd6f0008000 tx=0x7fd6f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:09.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6f000f800 con 0x7fd70013b790 2020-07-18T03:29:09.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000f960 con 0x7fd70013b790 2020-07-18T03:29:09.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700138ec0 con 0x7fd70013b790 2020-07-18T03:29:09.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.523+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6f000fcc0 con 0x7fd70013b790 2020-07-18T03:29:09.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.524+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7001376d0 con 0x7fd70013b790 2020-07-18T03:29:09.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.524+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70005b120 con 0x7fd70013b790 2020-07-18T03:29:09.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.525+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd6f0014440 con 0x7fd70013b790 2020-07-18T03:29:09.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.525+0000 7fd6df7fe700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d8050310 0x7fd6d80527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:09.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.525+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd6f0068370 con 0x7fd70013b790 2020-07-18T03:29:09.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.525+0000 7fd6feffd700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d8050310 0x7fd6d80527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:09.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.525+0000 7fd6feffd700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d8050310 0x7fd6d80527d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fd6f400d000 tx=0x7fd6f40077d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:09.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.529+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd6f00409d0 con 0x7fd70013b790 2020-07-18T03:29:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:09.687+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7000690d0 con 0x7fd70013b790 2020-07-18T03:29:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.391+0000 7fd6df7fe700 1 -- 172.21.15.47:0/1777758711 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd6f0018020 con 0x7fd70013b790 2020-07-18T03:29:10.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d8050310 msgr2=0x7fd6d80527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:10.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d8050310 0x7fd6d80527d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fd6f400d000 tx=0x7fd6f40077d0).stop 2020-07-18T03:29:10.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd70013b790 msgr2=0x7fd700143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:10.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd70013b790 0x7fd700143850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fd6f0008000 tx=0x7fd6f0008040).stop 2020-07-18T03:29:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 shutdown_connections 2020-07-18T03:29:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd700139c10 0x7fd700138840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd70013b790 0x7fd700143850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 --2- 172.21.15.47:0/1777758711 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d8050310 0x7fd6d80527d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 >> 172.21.15.47:0/1777758711 conn(0x7fd70005f5a0 msgr2=0x7fd7000b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 shutdown_connections 2020-07-18T03:29:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.393+0000 7fd705ba1700 1 -- 172.21.15.47:0/1777758711 wait complete. 2020-07-18T03:29:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2020-07-18T03:29:10.419 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-07-18T03:29:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.544+0000 7f06e8732700 1 Processor -- start 2020-07-18T03:29:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.544+0000 7f06e8732700 1 -- start start 2020-07-18T03:29:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e8732700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f06e00ad330 0x7f06e0143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e8732700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b59b0 0x7f06e00b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e8732700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e00b6300 0x7f06e00b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e8732700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e0147de0 con 0x7f06e00ad330 2020-07-18T03:29:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e8732700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e00ae000 con 0x7f06e00b6300 2020-07-18T03:29:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e8732700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0148010 con 0x7f06e00b59b0 2020-07-18T03:29:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e00b6300 0x7f06e00b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e00b6300 0x7f06e00b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60618/0 (socket says 172.21.15.47:60618) 2020-07-18T03:29:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 -- 172.21.15.47:0/927091292 learned_addr learned my addr 172.21.15.47:0/927091292 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e64ce700 1 --2- 172.21.15.47:0/927091292 >> v2:172.21.15.18:3300/0 conn(0x7f06e00ad330 0x7f06e0143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e5ccd700 1 --2- 172.21.15.47:0/927091292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b59b0 0x7f06e00b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 -- 172.21.15.47:0/927091292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b59b0 msgr2=0x7f06e00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 --2- 172.21.15.47:0/927091292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b59b0 0x7f06e00b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 -- 172.21.15.47:0/927091292 >> v2:172.21.15.18:3300/0 conn(0x7f06e00ad330 msgr2=0x7f06e0143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:10.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 --2- 172.21.15.47:0/927091292 >> v2:172.21.15.18:3300/0 conn(0x7f06e00ad330 0x7f06e0143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 -- 172.21.15.47:0/927091292 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e00ad710 con 0x7f06e00b6300 2020-07-18T03:29:10.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.545+0000 7f06e6ccf700 1 --2- 172.21.15.47:0/927091292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b6300 0x7f06e00b3a10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f06d80032b0 tx=0x7f06d8009270).ready entity=mon.1 client_cookie=a887f2fc48a5eae7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e54cc700 1 -- 172.21.15.47:0/927091292 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06d8009e60 con 0x7f06e00b6300 2020-07-18T03:29:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e54cc700 1 -- 172.21.15.47:0/927091292 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d800bb50 con 0x7f06e00b6300 2020-07-18T03:29:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e54cc700 1 -- 172.21.15.47:0/927091292 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06d800bcc0 con 0x7f06e00b6300 2020-07-18T03:29:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 -- 172.21.15.47:0/927091292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b6300 msgr2=0x7f06e00b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 --2- 172.21.15.47:0/927091292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b6300 0x7f06e00b3a10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f06d80032b0 tx=0x7f06d8009270).stop 2020-07-18T03:29:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 -- 172.21.15.47:0/927091292 shutdown_connections 2020-07-18T03:29:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 --2- 172.21.15.47:0/927091292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b6300 0x7f06e00b3a10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 --2- 172.21.15.47:0/927091292 >> v2:172.21.15.18:3300/0 conn(0x7f06e00ad330 0x7f06e0143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 --2- 172.21.15.47:0/927091292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b59b0 0x7f06e00b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 -- 172.21.15.47:0/927091292 >> 172.21.15.47:0/927091292 conn(0x7f06e005f5a0 msgr2=0x7f06e005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 -- 172.21.15.47:0/927091292 shutdown_connections 2020-07-18T03:29:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.546+0000 7f06e8732700 1 -- 172.21.15.47:0/927091292 wait complete. 2020-07-18T03:29:10.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.547+0000 7f06e8732700 1 Processor -- start 2020-07-18T03:29:10.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.547+0000 7f06e8732700 1 -- start start 2020-07-18T03:29:10.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e8732700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e00b59b0 0x7f06e01467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:10.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e8732700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b1530 0x7f06e01426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:10.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e8732700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e01459e0 con 0x7f06e00ad330 2020-07-18T03:29:10.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e8732700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e0145c80 con 0x7f06e00b59b0 2020-07-18T03:29:10.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e8732700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0146120 con 0x7f06e00b1530 2020-07-18T03:29:10.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e00b59b0 0x7f06e01467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06e00b59b0 0x7f06e01467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60626/0 (socket says 172.21.15.47:60626) 2020-07-18T03:29:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 -- 172.21.15.47:0/2166762428 learned_addr learned my addr 172.21.15.47:0/2166762428 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e6ccf700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b1530 0x7f06e01426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 -- 172.21.15.47:0/2166762428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b1530 msgr2=0x7f06e01426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b1530 0x7f06e01426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 -- 172.21.15.47:0/2166762428 >> v1:172.21.15.18:6789/0 conn(0x7f06e00ad330 legacy=0x7f06e00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06e5ccd700 1 -- 172.21.15.47:0/2166762428 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e00b2d70 con 0x7f06e00b59b0 2020-07-18T03:29:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.548+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202945225 0 0) 0x7f06e01459e0 con 0x7f06e00ad330 2020-07-18T03:29:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.549+0000 7f06e5ccd700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b59b0 0x7f06e01467e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f06dc00c000 tx=0x7f06dc006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.549+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06dc00ca10 con 0x7f06e00b59b0 2020-07-18T03:29:10.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.549+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc00cb70 con 0x7f06e00b59b0 2020-07-18T03:29:10.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.549+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e0146cd0 con 0x7f06e00b59b0 2020-07-18T03:29:10.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.549+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06dc0123f0 con 0x7f06e00b59b0 2020-07-18T03:29:10.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.549+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e0142dd0 con 0x7f06e00b59b0 2020-07-18T03:29:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.550+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e00b49f0 con 0x7f06e00b59b0 2020-07-18T03:29:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.550+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f06dc0125a0 con 0x7f06e00b59b0 2020-07-18T03:29:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.550+0000 7f06d27fc700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06b804fef0 0x7f06b80523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.550+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f06dc0680c0 con 0x7f06e00b59b0 2020-07-18T03:29:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.550+0000 7f06e64ce700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06b804fef0 0x7f06b80523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.551+0000 7f06e64ce700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06b804fef0 0x7f06b80523b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f06d4002f30 tx=0x7f06d4006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.554+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f06dc040950 con 0x7f06e00b59b0 2020-07-18T03:29:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:10.712+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06e00b0230 con 0x7f06e00b59b0 2020-07-18T03:29:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.394+0000 7f06d27fc700 1 -- 172.21.15.47:0/2166762428 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 173+0+0 (secure 0 0 0) 0x7f06dc043f70 con 0x7f06e00b59b0 2020-07-18T03:29:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06b804fef0 msgr2=0x7f06b80523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06b804fef0 0x7f06b80523b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f06d4002f30 tx=0x7f06d4006dd0).stop 2020-07-18T03:29:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b59b0 msgr2=0x7f06e01467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b59b0 0x7f06e01467e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f06dc00c000 tx=0x7f06dc006dd0).stop 2020-07-18T03:29:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 shutdown_connections 2020-07-18T03:29:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06b804fef0 0x7f06b80523b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06e00b1530 0x7f06e01426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 --2- 172.21.15.47:0/2166762428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06e00b59b0 0x7f06e01467e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 >> 172.21.15.47:0/2166762428 conn(0x7f06e005f5a0 msgr2=0x7f06e00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 shutdown_connections 2020-07-18T03:29:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.397+0000 7f06e8732700 1 -- 172.21.15.47:0/2166762428 wait complete. 2020-07-18T03:29:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2020-07-18T03:29:11.418 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:29:11.419 INFO:tasks.ceph.mds.a:Stopping old one... 2020-07-18T03:29:11.419 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:29:11.419 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:29:11.420 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:29:11.420 INFO:tasks.ceph.mds.b:Stopping old one... 2020-07-18T03:29:11.421 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:29:11.421 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:29:11.429 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:29:11.430 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:29:11.431 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:29:11.432 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:29:11.475 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:29:11.476 DEBUG:teuthology.parallel:result is None 2020-07-18T03:29:11.477 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:29:11.477 DEBUG:teuthology.parallel:result is None 2020-07-18T03:29:11.478 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-07-18T03:29:11.568 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:29:11.569 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:29:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.610+0000 7f9931c21700 1 Processor -- start 2020-07-18T03:29:11.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.611+0000 7f9931c21700 1 -- start start 2020-07-18T03:29:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f9931c21700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f992c0aea60 0x7f992c142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f9931c21700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c0b5830 con 0x7f992c05ffb0 2020-07-18T03:29:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f9931c21700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c146da0 con 0x7f992c143330 2020-07-18T03:29:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f9931c21700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c0b5a60 con 0x7f992c0aea60 2020-07-18T03:29:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f992c0aea60 0x7f992c142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f992c143330 0x7f992c1471d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41736/0 (socket says 172.21.15.47:41736) 2020-07-18T03:29:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992bfff700 1 -- 172.21.15.47:0/1269102605 learned_addr learned my addr 172.21.15.47:0/1269102605 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992a7fc700 1 -- 172.21.15.47:0/1269102605 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925635547 0 0) 0x7f992c0b5830 con 0x7f992c05ffb0 2020-07-18T03:29:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992affd700 1 -- 172.21.15.47:0/1269102605 >> v1:172.21.15.47:6789/0 conn(0x7f992c143330 legacy=0x7f992c1471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992affd700 1 -- 172.21.15.47:0/1269102605 >> v1:172.21.15.18:6789/0 conn(0x7f992c05ffb0 legacy=0x7f992c0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992affd700 1 -- 172.21.15.47:0/1269102605 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c060390 con 0x7f992c0aea60 2020-07-18T03:29:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992affd700 1 --2- 172.21.15.47:0/1269102605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c0aea60 0x7f992c142e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f9924001450 tx=0x7f9920006dd0).ready entity=mon.1 client_cookie=51ae8a2fed2bdc42 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:11.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992a7fc700 1 -- 172.21.15.47:0/1269102605 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9920009730 con 0x7f992c0aea60 2020-07-18T03:29:11.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992a7fc700 1 -- 172.21.15.47:0/1269102605 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920009890 con 0x7f992c0aea60 2020-07-18T03:29:11.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f992a7fc700 1 -- 172.21.15.47:0/1269102605 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9920009bf0 con 0x7f992c0aea60 2020-07-18T03:29:11.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f9931c21700 1 -- 172.21.15.47:0/1269102605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c0aea60 msgr2=0x7f992c142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.612+0000 7f9931c21700 1 --2- 172.21.15.47:0/1269102605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c0aea60 0x7f992c142e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f9924001450 tx=0x7f9920006dd0).stop 2020-07-18T03:29:11.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.613+0000 7f9931c21700 1 -- 172.21.15.47:0/1269102605 shutdown_connections 2020-07-18T03:29:11.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.613+0000 7f9931c21700 1 --2- 172.21.15.47:0/1269102605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c0aea60 0x7f992c142e40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:11.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.613+0000 7f9931c21700 1 -- 172.21.15.47:0/1269102605 >> 172.21.15.47:0/1269102605 conn(0x7f992c0b93c0 msgr2=0x7f992c0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:11.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.613+0000 7f9931c21700 1 -- 172.21.15.47:0/1269102605 shutdown_connections 2020-07-18T03:29:11.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.613+0000 7f9931c21700 1 -- 172.21.15.47:0/1269102605 wait complete. 2020-07-18T03:29:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.614+0000 7f9931c21700 1 Processor -- start 2020-07-18T03:29:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.614+0000 7f9931c21700 1 -- start start 2020-07-18T03:29:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f9931c21700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f992c05ffb0 0x7f992c144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f9931c21700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f992c143330 0x7f992c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f9931c21700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c140520 con 0x7f992c13db70 2020-07-18T03:29:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f9931c21700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c13f6d0 con 0x7f992c143330 2020-07-18T03:29:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f9931c21700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c13f970 con 0x7f992c05ffb0 2020-07-18T03:29:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f992affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f992c143330 0x7f992c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f992affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f992c143330 0x7f992c0b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60652/0 (socket says 172.21.15.47:60652) 2020-07-18T03:29:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f992affd700 1 -- 172.21.15.47:0/4185731438 learned_addr learned my addr 172.21.15.47:0/4185731438 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.615+0000 7f992b7fe700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f992c05ffb0 0x7f992c144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f992affd700 1 -- 172.21.15.47:0/4185731438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f992c05ffb0 msgr2=0x7f992c144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f992affd700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f992c05ffb0 0x7f992c144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f992affd700 1 -- 172.21.15.47:0/4185731438 >> v1:172.21.15.18:6789/0 conn(0x7f992c13db70 legacy=0x7f992c13fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f992affd700 1 -- 172.21.15.47:0/4185731438 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c1409c0 con 0x7f992c143330 2020-07-18T03:29:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f992b7fe700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f992c05ffb0 0x7f992c144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f992affd700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c143330 0x7f992c0b4730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f9924001450 tx=0x7f9920015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992001d5d0 con 0x7f992c143330 2020-07-18T03:29:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992001d730 con 0x7f992c143330 2020-07-18T03:29:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c0b4c20 con 0x7f992c143330 2020-07-18T03:29:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992001da90 con 0x7f992c143330 2020-07-18T03:29:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.616+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c062f50 con 0x7f992c143330 2020-07-18T03:29:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.617+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c057450 con 0x7f992c143330 2020-07-18T03:29:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.617+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9920025030 con 0x7f992c143330 2020-07-18T03:29:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.618+0000 7f990b7fe700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f990404ffa0 0x7f9904052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.618+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f99200692e0 con 0x7f992c143330 2020-07-18T03:29:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.618+0000 7f992b7fe700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f990404ffa0 0x7f9904052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.618+0000 7f992b7fe700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f990404ffa0 0x7f9904052460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f991c008000 tx=0x7f991c009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.621+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9920041d50 con 0x7f992c143330 2020-07-18T03:29:11.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:11.798+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f992c0621f0 con 0x7f992c143330 2020-07-18T03:29:12.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.580+0000 7f990b7fe700 1 -- 172.21.15.47:0/4185731438 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9920045370 con 0x7f992c143330 2020-07-18T03:29:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f990404ffa0 msgr2=0x7f9904052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f990404ffa0 0x7f9904052460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f991c008000 tx=0x7f991c009040).stop 2020-07-18T03:29:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c143330 msgr2=0x7f992c0b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c143330 0x7f992c0b4730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f9924001450 tx=0x7f9920015000).stop 2020-07-18T03:29:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 shutdown_connections 2020-07-18T03:29:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f990404ffa0 0x7f9904052460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f992c05ffb0 0x7f992c144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 --2- 172.21.15.47:0/4185731438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f992c143330 0x7f992c0b4730 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 >> 172.21.15.47:0/4185731438 conn(0x7f992c0b93c0 msgr2=0x7f992c0b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 shutdown_connections 2020-07-18T03:29:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.582+0000 7f9931c21700 1 -- 172.21.15.47:0/4185731438 wait complete. 2020-07-18T03:29:12.586 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2020-07-18T03:29:12.597 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2020-07-18T03:29:12.597 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-07-18T03:29:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.717+0000 7fc7f2a56700 1 Processor -- start 2020-07-18T03:29:12.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.717+0000 7fc7f2a56700 1 -- start start 2020-07-18T03:29:12.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f2a56700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b2e10 0x7fc7ec0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:12.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f2a56700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7ec0b1a10 0x7fc7ec1454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:12.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f2a56700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ec0afbd0 con 0x7fc7ec0b1560 2020-07-18T03:29:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f2a56700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ec144e30 con 0x7fc7ec0b2e10 2020-07-18T03:29:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f2a56700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ec1491c0 con 0x7fc7ec0b1a10 2020-07-18T03:29:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f0ff3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7ec0b1a10 0x7fc7ec1454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7e3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc7ec0b1560 0x7fc7ec0b2130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41750/0 (socket says 172.21.15.47:41750) 2020-07-18T03:29:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7e3fff700 1 -- 172.21.15.47:0/1318998626 learned_addr learned my addr 172.21.15.47:0/1318998626 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:12.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7ebfff700 1 --2- 172.21.15.47:0/1318998626 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b2e10 0x7fc7ec0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:12.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/1318998626 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b2e10 msgr2=0x7fc7ec0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:12.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f0ff3700 1 --2- 172.21.15.47:0/1318998626 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b2e10 0x7fc7ec0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/1318998626 >> v1:172.21.15.47:6789/0 conn(0x7fc7ec0b1560 legacy=0x7fc7ec0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.718+0000 7fc7eb7fe700 1 -- 172.21.15.47:0/1318998626 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053058517 0 0) 0x7fc7d40010d0 con 0x7fc7ec0b1560 2020-07-18T03:29:12.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/1318998626 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec145050 con 0x7fc7ec0b1a10 2020-07-18T03:29:12.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f0ff3700 1 --2- 172.21.15.47:0/1318998626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1a10 0x7fc7ec1454f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fc7ec0ba340 tx=0x7fc7e40091d0).ready entity=mon.1 client_cookie=13e6c260562be776 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:12.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7eb7fe700 1 -- 172.21.15.47:0/1318998626 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e400b950 con 0x7fc7ec0b1a10 2020-07-18T03:29:12.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7eb7fe700 1 -- 172.21.15.47:0/1318998626 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e400bab0 con 0x7fc7ec0b1a10 2020-07-18T03:29:12.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7eb7fe700 1 -- 172.21.15.47:0/1318998626 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e400bc20 con 0x7fc7ec0b1a10 2020-07-18T03:29:12.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 -- 172.21.15.47:0/1318998626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1a10 msgr2=0x7fc7ec1454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/1318998626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1a10 0x7fc7ec1454f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fc7ec0ba340 tx=0x7fc7e40091d0).stop 2020-07-18T03:29:12.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 -- 172.21.15.47:0/1318998626 shutdown_connections 2020-07-18T03:29:12.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/1318998626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1a10 0x7fc7ec1454f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/1318998626 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b2e10 0x7fc7ec0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 -- 172.21.15.47:0/1318998626 >> 172.21.15.47:0/1318998626 conn(0x7fc7ec0b9340 msgr2=0x7fc7ec0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:12.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 -- 172.21.15.47:0/1318998626 shutdown_connections 2020-07-18T03:29:12.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.719+0000 7fc7f2a56700 1 -- 172.21.15.47:0/1318998626 wait complete. 2020-07-18T03:29:12.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.720+0000 7fc7f2a56700 1 Processor -- start 2020-07-18T03:29:12.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 -- start start 2020-07-18T03:29:12.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b1560 0x7fc7ec137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:12.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7ec0b2e10 0x7fc7ec065d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:12.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7ec066210 0x7fc7ec146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:12.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ec13b2f0 con 0x7fc7ec0b1560 2020-07-18T03:29:12.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ec0ad1a0 con 0x7fc7ec066210 2020-07-18T03:29:12.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f2a56700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec137e50 con 0x7fc7ec0b2e10 2020-07-18T03:29:12.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f0ff3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7ec066210 0x7fc7ec146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f0ff3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7ec066210 0x7fc7ec146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60670/0 (socket says 172.21.15.47:60670) 2020-07-18T03:29:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/2710823345 learned_addr learned my addr 172.21.15.47:0/2710823345 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7ebfff700 1 --2- 172.21.15.47:0/2710823345 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b1560 0x7fc7ec137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.721+0000 7fc7e3fff700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7ec0b2e10 0x7fc7ec065d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/2710823345 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7ec0b2e10 msgr2=0x7fc7ec065d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f0ff3700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7ec0b2e10 0x7fc7ec065d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/2710823345 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b1560 msgr2=0x7fc7ec137960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f0ff3700 1 --2- 172.21.15.47:0/2710823345 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b1560 0x7fc7ec137960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f0ff3700 1 -- 172.21.15.47:0/2710823345 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec13a310 con 0x7fc7ec066210 2020-07-18T03:29:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f0ff3700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec066210 0x7fc7ec146520 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fc7ec0ba340 tx=0x7fc7e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e40107a0 con 0x7fc7ec066210 2020-07-18T03:29:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4010900 con 0x7fc7ec066210 2020-07-18T03:29:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e4010c60 con 0x7fc7ec066210 2020-07-18T03:29:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ec13a4a0 con 0x7fc7ec066210 2020-07-18T03:29:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.722+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ec146c50 con 0x7fc7ec066210 2020-07-18T03:29:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.723+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ec057450 con 0x7fc7ec066210 2020-07-18T03:29:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.723+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc7e401e070 con 0x7fc7ec066210 2020-07-18T03:29:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.723+0000 7fc7e8ff9700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc7c804ffa0 0x7fc7c8052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.724+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc7e4068a70 con 0x7fc7ec066210 2020-07-18T03:29:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.724+0000 7fc7ebfff700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc7c804ffa0 0x7fc7c8052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.724+0000 7fc7ebfff700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc7c804ffa0 0x7fc7c8052460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc7dc00b000 tx=0x7fc7dc006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.727+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc7e4041540 con 0x7fc7ec066210 2020-07-18T03:29:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.941+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc7ec060870 con 0x7fc7ec066210 2020-07-18T03:29:12.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.941+0000 7fc7e8ff9700 1 -- 172.21.15.47:0/2710823345 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fc7e4068e60 con 0x7fc7ec066210 2020-07-18T03:29:12.944 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:12.944 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:12.944 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2020-07-18T03:29:12.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.945 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2020-07-18T03:29:12.945 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g==", 2020-07-18T03:29:12.945 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.945 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:12.946 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.946 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.946 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.946 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.947 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2020-07-18T03:29:12.947 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ==", 2020-07-18T03:29:12.947 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.947 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:12.948 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.948 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.949 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2020-07-18T03:29:12.949 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA==", 2020-07-18T03:29:12.949 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.950 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.950 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.951 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2020-07-18T03:29:12.951 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ==", 2020-07-18T03:29:12.951 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.951 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.952 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2020-07-18T03:29:12.952 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA==", 2020-07-18T03:29:12.953 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.953 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.954 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2020-07-18T03:29:12.954 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ==", 2020-07-18T03:29:12.954 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.955 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.956 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2020-07-18T03:29:12.956 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ==", 2020-07-18T03:29:12.957 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.957 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.958 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2020-07-18T03:29:12.958 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA==", 2020-07-18T03:29:12.958 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.959 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.960 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2020-07-18T03:29:12.960 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg==", 2020-07-18T03:29:12.960 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.960 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.961 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.961 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.961 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.962 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2020-07-18T03:29:12.962 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg==", 2020-07-18T03:29:12.962 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.962 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.962 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.963 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.963 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.964 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2020-07-18T03:29:12.964 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw==", 2020-07-18T03:29:12.964 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:12.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:12.965 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:12.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.965 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2020-07-18T03:29:12.966 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g==", 2020-07-18T03:29:12.966 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:12.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:12.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:12.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.967 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2020-07-18T03:29:12.967 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w==", 2020-07-18T03:29:12.968 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:12.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:12.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:12.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.969 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2020-07-18T03:29:12.969 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog==", 2020-07-18T03:29:12.969 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.970 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:12.970 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:12.970 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:12.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.971 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2020-07-18T03:29:12.971 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfi9GyDxAACUH4y/92DOhKqZyh9ejEnA==", 2020-07-18T03:29:12.971 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.972 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:29:12.972 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:12.972 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:12.972 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.973 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2020-07-18T03:29:12.973 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ==", 2020-07-18T03:29:12.974 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.974 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:29:12.974 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:29:12.974 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2020-07-18T03:29:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw==", 2020-07-18T03:29:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:29:12.976 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:29:12.976 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:12.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:12.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:12.977 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:12.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc7c804ffa0 msgr2=0x7fc7c8052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc7c804ffa0 0x7fc7c8052460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc7dc00b000 tx=0x7fc7dc006dd0).stop 2020-07-18T03:29:12.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec066210 msgr2=0x7fc7ec146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:12.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec066210 0x7fc7ec146520 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fc7ec0ba340 tx=0x7fc7e4016000).stop 2020-07-18T03:29:12.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 shutdown_connections 2020-07-18T03:29:12.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc7c804ffa0 0x7fc7c8052460 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec066210 0x7fc7ec146520 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/2710823345 >> v2:172.21.15.18:3300/0 conn(0x7fc7ec0b1560 0x7fc7ec137960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 --2- 172.21.15.47:0/2710823345 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc7ec0b2e10 0x7fc7ec065d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:12.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 >> 172.21.15.47:0/2710823345 conn(0x7fc7ec0b9340 msgr2=0x7fc7ec0b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:12.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 shutdown_connections 2020-07-18T03:29:12.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:12.943+0000 7fc7f2a56700 1 -- 172.21.15.47:0/2710823345 wait complete. 2020-07-18T03:29:12.982 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.095+0000 7f7d44ed7700 1 Processor -- start 2020-07-18T03:29:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.095+0000 7f7d44ed7700 1 -- start start 2020-07-18T03:29:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d44ed7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d44ed7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d40147a00 0x7f7d40147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d44ed7700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7d4005f5a0 0x7f7d4005fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d44ed7700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d400ae530 con 0x7f7d4005f5a0 2020-07-18T03:29:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d44ed7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d40147330 con 0x7f7d40147a00 2020-07-18T03:29:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d44ed7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400b57e0 con 0x7f7d400ad100 2020-07-18T03:29:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d3dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d40147a00 0x7f7d40147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d3dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d40147a00 0x7f7d40147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60676/0 (socket says 172.21.15.47:60676) 2020-07-18T03:29:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.096+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/822700536 learned_addr learned my addr 172.21.15.47:0/822700536 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3e59c700 1 --2- 172.21.15.47:0/822700536 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3ed9d700 1 --2- 172.21.15.47:0/822700536 >> v2:172.21.15.18:3300/0 conn(0x7f7d4005f5a0 0x7f7d4005fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/822700536 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 msgr2=0x7f7d400b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3dd9b700 1 --2- 172.21.15.47:0/822700536 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/822700536 >> v2:172.21.15.18:3300/0 conn(0x7f7d4005f5a0 msgr2=0x7f7d4005fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3dd9b700 1 --2- 172.21.15.47:0/822700536 >> v2:172.21.15.18:3300/0 conn(0x7f7d4005f5a0 0x7f7d4005fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/822700536 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400b5a80 con 0x7f7d40147a00 2020-07-18T03:29:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3dd9b700 1 --2- 172.21.15.47:0/822700536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d40147a00 0x7f7d40147e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f7d300032b0 tx=0x7f7d30009270).ready entity=mon.1 client_cookie=657de084c048e78 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3d59a700 1 -- 172.21.15.47:0/822700536 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3000bc00 con 0x7f7d40147a00 2020-07-18T03:29:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3d59a700 1 -- 172.21.15.47:0/822700536 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3000bd60 con 0x7f7d40147a00 2020-07-18T03:29:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d3d59a700 1 -- 172.21.15.47:0/822700536 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d30011400 con 0x7f7d40147a00 2020-07-18T03:29:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 -- 172.21.15.47:0/822700536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d40147a00 msgr2=0x7f7d40147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/822700536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d40147a00 0x7f7d40147e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f7d300032b0 tx=0x7f7d30009270).stop 2020-07-18T03:29:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 -- 172.21.15.47:0/822700536 shutdown_connections 2020-07-18T03:29:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/822700536 >> v2:172.21.15.18:3300/0 conn(0x7f7d4005f5a0 0x7f7d4005fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/822700536 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/822700536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d40147a00 0x7f7d40147e40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 -- 172.21.15.47:0/822700536 >> 172.21.15.47:0/822700536 conn(0x7f7d400b9340 msgr2=0x7f7d400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 -- 172.21.15.47:0/822700536 shutdown_connections 2020-07-18T03:29:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.097+0000 7f7d44ed7700 1 -- 172.21.15.47:0/822700536 wait complete. 2020-07-18T03:29:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.099+0000 7f7d44ed7700 1 Processor -- start 2020-07-18T03:29:13.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.099+0000 7f7d44ed7700 1 -- start start 2020-07-18T03:29:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.099+0000 7f7d44ed7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.099+0000 7f7d44ed7700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d401455e0 con 0x7f7d4005f5a0 2020-07-18T03:29:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.099+0000 7f7d44ed7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d40139f70 con 0x7f7d400b21e0 2020-07-18T03:29:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.099+0000 7f7d44ed7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401453d0 con 0x7f7d400ad100 2020-07-18T03:29:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d400b21e0 0x7f7d40138a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41772/0 (socket says 172.21.15.47:41772) 2020-07-18T03:29:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3ed9d700 1 -- 172.21.15.47:0/3761183063 learned_addr learned my addr 172.21.15.47:0/3761183063 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3dd9b700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790197006 0 0) 0x7f7d40139f70 con 0x7f7d400b21e0 2020-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d14003240 con 0x7f7d400b21e0 2020-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788994976 0 0) 0x7f7d401455e0 con 0x7f7d4005f5a0 2020-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d40139f70 con 0x7f7d4005f5a0 2020-07-18T03:29:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/3761183063 >> v1:172.21.15.47:6789/0 conn(0x7f7d400b21e0 legacy=0x7f7d40138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/3761183063 >> v1:172.21.15.18:6789/0 conn(0x7f7d4005f5a0 legacy=0x7f7d400b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3dd9b700 1 -- 172.21.15.47:0/3761183063 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400b2670 con 0x7f7d400ad100 2020-07-18T03:29:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d3dd9b700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b0bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7d300032b0 tx=0x7f7d30016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d300107a0 con 0x7f7d400ad100 2020-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30010900 con 0x7f7d400ad100 2020-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d400b2800 con 0x7f7d400ad100 2020-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d30010c60 con 0x7f7d400ad100 2020-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.100+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d40146b20 con 0x7f7d400ad100 2020-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.101+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d40057450 con 0x7f7d400ad100 2020-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.102+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7d30027020 con 0x7f7d400ad100 2020-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.102+0000 7f7d2e7fc700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d140504f0 0x7f7d140529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.102+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7d30068cf0 con 0x7f7d400ad100 2020-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.102+0000 7f7d3e59c700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d140504f0 0x7f7d140529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.103+0000 7f7d3e59c700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d140504f0 0x7f7d140529b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f7d34007000 tx=0x7f7d34008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.107+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7d300417c0 con 0x7f7d400ad100 2020-07-18T03:29:13.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.264+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d400b2800 con 0x7f7d400ad100 2020-07-18T03:29:13.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.266+0000 7f7d2e7fc700 1 -- 172.21.15.47:0/3761183063 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13417 (secure 0 0 0) 0x7f7d30015070 con 0x7f7d400ad100 2020-07-18T03:29:13.269 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:13.269 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:12.576398+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:13.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d140504f0 msgr2=0x7f7d140529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d140504f0 0x7f7d140529b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f7d34007000 tx=0x7f7d34008040).stop 2020-07-18T03:29:13.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 msgr2=0x7f7d400b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b0bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7d300032b0 tx=0x7f7d30016000).stop 2020-07-18T03:29:13.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 shutdown_connections 2020-07-18T03:29:13.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d140504f0 0x7f7d140529b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 --2- 172.21.15.47:0/3761183063 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7d400ad100 0x7f7d400b0bf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 >> 172.21.15.47:0/3761183063 conn(0x7f7d400b9340 msgr2=0x7f7d400b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:13.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 shutdown_connections 2020-07-18T03:29:13.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.267+0000 7f7d44ed7700 1 -- 172.21.15.47:0/3761183063 wait complete. 2020-07-18T03:29:13.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-07-18T03:29:13.281 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:29:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.396+0000 7f765ec8f700 1 Processor -- start 2020-07-18T03:29:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.396+0000 7f765ec8f700 1 -- start start 2020-07-18T03:29:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f765ec8f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7658141fb0 0x7f7658142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f765ec8f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7658061db0 con 0x7f76580b04b0 2020-07-18T03:29:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f765ec8f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7658062050 con 0x7f7658144e70 2020-07-18T03:29:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f765ec8f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7658145250 con 0x7f7658141fb0 2020-07-18T03:29:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f765ca2b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7658144e70 0x7f76580b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41776/0 (socket says 172.21.15.47:41776) 2020-07-18T03:29:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f765ca2b700 1 -- 172.21.15.47:0/3131135775 learned_addr learned my addr 172.21.15.47:0/3131135775 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f7657fff700 1 --2- 172.21.15.47:0/3131135775 >> v2:172.21.15.18:3300/0 conn(0x7f7658141fb0 0x7f7658142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205095983 0 0) 0x7f7658062050 con 0x7f7658144e70 2020-07-18T03:29:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.397+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7644003240 con 0x7f7658144e70 2020-07-18T03:29:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3110844987 0 0) 0x7f7644003240 con 0x7f7658144e70 2020-07-18T03:29:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 >> v2:172.21.15.18:3300/0 conn(0x7f7658141fb0 msgr2=0x7f7658142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 --2- 172.21.15.47:0/3131135775 >> v2:172.21.15.18:3300/0 conn(0x7f7658141fb0 0x7f7658142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 >> v1:172.21.15.18:6789/0 conn(0x7f76580b04b0 legacy=0x7f76580b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76580622f0 con 0x7f7658144e70 2020-07-18T03:29:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f76480016d0 con 0x7f7658144e70 2020-07-18T03:29:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 >> v1:172.21.15.47:6789/0 conn(0x7f7658144e70 legacy=0x7f76580b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7644003730 0x7f7644003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76440044e0 0x7f7644004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7644004e50 0x7f764400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658145250 con 0x7f7644003730 2020-07-18T03:29:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644004350 con 0x7f76440044e0 2020-07-18T03:29:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764400bc50 con 0x7f7644004e50 2020-07-18T03:29:13.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76440044e0 0x7f7644004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f765ca2b700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7644003730 0x7f7644003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f765d22c700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7644004e50 0x7f764400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 -- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7644004e50 msgr2=0x7f764400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7644004e50 0x7f764400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 -- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7644003730 msgr2=0x7f7644003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7644003730 0x7f7644003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f765d22c700 1 -- 172.21.15.47:0/3131135775 reap_dead start 2020-07-18T03:29:13.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 -- 172.21.15.47:0/3131135775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644003240 con 0x7f76440044e0 2020-07-18T03:29:13.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f765ca2b700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7644003730 0x7f7644003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.398+0000 7f7657fff700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76440044e0 0x7f7644004960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f764c00c000 tx=0x7f764c008040).ready entity=mon.1 client_cookie=11be754617110c16 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764c00e970 con 0x7f76440044e0 2020-07-18T03:29:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00ead0 con 0x7f76440044e0 2020-07-18T03:29:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f76577fe700 1 -- 172.21.15.47:0/3131135775 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764c00ee30 con 0x7f76440044e0 2020-07-18T03:29:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 -- 172.21.15.47:0/3131135775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76440044e0 msgr2=0x7f7644004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76440044e0 0x7f7644004960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f764c00c000 tx=0x7f764c008040).stop 2020-07-18T03:29:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 -- 172.21.15.47:0/3131135775 shutdown_connections 2020-07-18T03:29:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 --2- 172.21.15.47:0/3131135775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76440044e0 0x7f7644004960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 -- 172.21.15.47:0/3131135775 >> 172.21.15.47:0/3131135775 conn(0x7f765805f5a0 msgr2=0x7f765805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 -- 172.21.15.47:0/3131135775 shutdown_connections 2020-07-18T03:29:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.399+0000 7f765ec8f700 1 -- 172.21.15.47:0/3131135775 wait complete. 2020-07-18T03:29:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.400+0000 7f765ec8f700 1 Processor -- start 2020-07-18T03:29:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.400+0000 7f765ec8f700 1 -- start start 2020-07-18T03:29:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765ec8f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f765812e070 0x7f7658066740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765ec8f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7658066000 con 0x7f7658148e50 2020-07-18T03:29:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765ec8f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76580662a0 con 0x7f7658148a70 2020-07-18T03:29:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765ec8f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7658065e20 con 0x7f765812e070 2020-07-18T03:29:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765ca2b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7658148a70 0x7f765812d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41788/0 (socket says 172.21.15.47:41788) 2020-07-18T03:29:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765ca2b700 1 -- 172.21.15.47:0/1275732902 learned_addr learned my addr 172.21.15.47:0/1275732902 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.401+0000 7f765d22c700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f765812e070 0x7f7658066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.402+0000 7f765d22c700 1 -- 172.21.15.47:0/1275732902 >> v1:172.21.15.47:6789/0 conn(0x7f7658148a70 legacy=0x7f765812d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.402+0000 7f765d22c700 1 -- 172.21.15.47:0/1275732902 >> v1:172.21.15.18:6789/0 conn(0x7f7658148e50 legacy=0x7f7658145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.402+0000 7f765d22c700 1 -- 172.21.15.47:0/1275732902 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658148560 con 0x7f765812e070 2020-07-18T03:29:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.402+0000 7f765d22c700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f765812e070 0x7f7658066740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f76500011d0 tx=0x7f7650009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.404+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76500032c0 con 0x7f765812e070 2020-07-18T03:29:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.404+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7650010040 con 0x7f765812e070 2020-07-18T03:29:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.404+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7658065970 con 0x7f765812e070 2020-07-18T03:29:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.404+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7650001930 con 0x7f765812e070 2020-07-18T03:29:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.404+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658064350 con 0x7f765812e070 2020-07-18T03:29:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.405+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658066430 con 0x7f765812e070 2020-07-18T03:29:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.405+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f765001c060 con 0x7f765812e070 2020-07-18T03:29:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.406+0000 7f7654ff9700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f763004ffa0 0x7f7630052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.406+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f76500671c0 con 0x7f765812e070 2020-07-18T03:29:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.406+0000 7f765ca2b700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f763004ffa0 0x7f7630052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.406+0000 7f765ca2b700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f763004ffa0 0x7f7630052460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7648008000 tx=0x7f7658061f40).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.409+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f765003fc10 con 0x7f765812e070 2020-07-18T03:29:13.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.568+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f76580b1600 con 0x7f765812e070 2020-07-18T03:29:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.568+0000 7f7654ff9700 1 -- 172.21.15.47:0/1275732902 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v38) v1 ==== 110+0+10 (secure 0 0 0) 0x7f765001a020 con 0x7f765812e070 2020-07-18T03:29:13.571 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:29:13.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f763004ffa0 msgr2=0x7f7630052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f763004ffa0 0x7f7630052460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7648008000 tx=0x7f7658061f40).stop 2020-07-18T03:29:13.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f765812e070 msgr2=0x7f7658066740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f765812e070 0x7f7658066740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f76500011d0 tx=0x7f7650009040).stop 2020-07-18T03:29:13.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 shutdown_connections 2020-07-18T03:29:13.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f765812e070 0x7f7658066740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 --2- 172.21.15.47:0/1275732902 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f763004ffa0 0x7f7630052460 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 >> 172.21.15.47:0/1275732902 conn(0x7f765805f5a0 msgr2=0x7f76580b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:13.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 shutdown_connections 2020-07-18T03:29:13.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.570+0000 7f765ec8f700 1 -- 172.21.15.47:0/1275732902 wait complete. 2020-07-18T03:29:13.583 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:29:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.698+0000 7f251c080700 1 Processor -- start 2020-07-18T03:29:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f251c080700 1 -- start start 2020-07-18T03:29:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f251c080700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2514061c60 0x7f25140620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f251c080700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f25140b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f251c080700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140af4b0 con 0x7f2514142a20 2020-07-18T03:29:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f251c080700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140b6d80 con 0x7f2514061c60 2020-07-18T03:29:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f251c080700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140af710 con 0x7f25140b3af0 2020-07-18T03:29:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f2519e1c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2514142a20 0x7f2514142e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41796/0 (socket says 172.21.15.47:41796) 2020-07-18T03:29:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.699+0000 7f2519e1c700 1 -- 172.21.15.47:0/745906815 learned_addr learned my addr 172.21.15.47:0/745906815 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f251a61d700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f25140b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f251961b700 1 --2- 172.21.15.47:0/745906815 >> v2:172.21.15.18:3300/0 conn(0x7f2514061c60 0x7f25140620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605195782 0 0) 0x7f25140af4b0 con 0x7f2514142a20 2020-07-18T03:29:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24fc003240 con 0x7f2514142a20 2020-07-18T03:29:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 408740229 0 0) 0x7f24fc003240 con 0x7f2514142a20 2020-07-18T03:29:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 msgr2=0x7f25140b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f25140b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 >> v2:172.21.15.18:3300/0 conn(0x7f2514061c60 msgr2=0x7f25140620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 --2- 172.21.15.47:0/745906815 >> v2:172.21.15.18:3300/0 conn(0x7f2514061c60 0x7f25140620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140af9b0 con 0x7f2514142a20 2020-07-18T03:29:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f25080016d0 con 0x7f2514142a20 2020-07-18T03:29:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f251a61d700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f25140b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 >> v1:172.21.15.47:6789/0 conn(0x7f2514142a20 legacy=0x7f2514142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24fc0037b0 0x7f24fc003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24fc0044e0 0x7f24fc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f24fc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b6d80 con 0x7f24fc0037b0 2020-07-18T03:29:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140af710 con 0x7f24fc0044e0 2020-07-18T03:29:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc005340 con 0x7f25140b3af0 2020-07-18T03:29:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f251961b700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24fc0044e0 0x7f24fc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f2519e1c700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24fc0037b0 0x7f24fc003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.700+0000 7f251a61d700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f24fc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251961b700 1 -- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 msgr2=0x7f24fc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251961b700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25140b3af0 0x7f24fc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251961b700 1 -- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24fc0037b0 msgr2=0x7f24fc003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251961b700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24fc0037b0 0x7f24fc003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251961b700 1 -- 172.21.15.47:0/745906815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc003240 con 0x7f24fc0044e0 2020-07-18T03:29:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f2519e1c700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24fc0037b0 0x7f24fc003bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251961b700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24fc0044e0 0x7f24fc004960 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f251000c000 tx=0x7f2510008040).ready entity=mon.1 client_cookie=dede0cd48cc429ab server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f251000ea90 con 0x7f24fc0044e0 2020-07-18T03:29:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251000ebf0 con 0x7f24fc0044e0 2020-07-18T03:29:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f2518e1a700 1 -- 172.21.15.47:0/745906815 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25100123f0 con 0x7f24fc0044e0 2020-07-18T03:29:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251c080700 1 -- 172.21.15.47:0/745906815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24fc0044e0 msgr2=0x7f24fc004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251c080700 1 --2- 172.21.15.47:0/745906815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24fc0044e0 0x7f24fc004960 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f251000c000 tx=0x7f2510008040).stop 2020-07-18T03:29:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251a61d700 1 -- 172.21.15.47:0/745906815 reap_dead start 2020-07-18T03:29:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251c080700 1 -- 172.21.15.47:0/745906815 shutdown_connections 2020-07-18T03:29:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251c080700 1 -- 172.21.15.47:0/745906815 >> 172.21.15.47:0/745906815 conn(0x7f251405f5a0 msgr2=0x7f251405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251c080700 1 -- 172.21.15.47:0/745906815 shutdown_connections 2020-07-18T03:29:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.701+0000 7f251c080700 1 -- 172.21.15.47:0/745906815 wait complete. 2020-07-18T03:29:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.702+0000 7f251c080700 1 Processor -- start 2020-07-18T03:29:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f251c080700 1 -- start start 2020-07-18T03:29:13.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f251c080700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2514140890 0x7f2514063980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f251c080700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f25140b20f0 0x7f25140b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f251c080700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2514065fa0 con 0x7f2514140c70 2020-07-18T03:29:13.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f251c080700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2514066240 con 0x7f25140b20f0 2020-07-18T03:29:13.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f251c080700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140b1c50 con 0x7f2514140890 2020-07-18T03:29:13.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f2519e1c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2514140890 0x7f2514063980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f2519e1c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2514140890 0x7f2514063980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60720/0 (socket says 172.21.15.47:60720) 2020-07-18T03:29:13.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.703+0000 7f2519e1c700 1 -- 172.21.15.47:0/3568458007 learned_addr learned my addr 172.21.15.47:0/3568458007 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:13.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2519e1c700 1 -- 172.21.15.47:0/3568458007 >> v2:172.21.15.18:3300/0 conn(0x7f25140b20f0 msgr2=0x7f25140b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f251a61d700 1 --2- 172.21.15.47:0/3568458007 >> v2:172.21.15.18:3300/0 conn(0x7f25140b20f0 0x7f25140b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2519e1c700 1 --2- 172.21.15.47:0/3568458007 >> v2:172.21.15.18:3300/0 conn(0x7f25140b20f0 0x7f25140b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2519e1c700 1 -- 172.21.15.47:0/3568458007 >> v1:172.21.15.18:6789/0 conn(0x7f2514140c70 legacy=0x7f251413fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2519e1c700 1 -- 172.21.15.47:0/3568458007 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514066730 con 0x7f2514140890 2020-07-18T03:29:13.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2519e1c700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2514140890 0x7f2514063980 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f2514064550 tx=0x7f2508001e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f250800fbc0 con 0x7f2514140890 2020-07-18T03:29:13.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250800fd20 con 0x7f2514140890 2020-07-18T03:29:13.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2508012540 con 0x7f2514140890 2020-07-18T03:29:13.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2514065c20 con 0x7f2514140890 2020-07-18T03:29:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.704+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25140b2a20 con 0x7f2514140890 2020-07-18T03:29:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.705+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f250801b030 con 0x7f2514140890 2020-07-18T03:29:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.705+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25140b86f0 con 0x7f2514140890 2020-07-18T03:29:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.705+0000 7f2505ffb700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f24ec04ff60 0x7f24ec052420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.705+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2508067640 con 0x7f2514140890 2020-07-18T03:29:13.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.705+0000 7f251961b700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f24ec04ff60 0x7f24ec052420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.706+0000 7f251961b700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f24ec04ff60 0x7f24ec052420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f251000c000 tx=0x7f2510019000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:13.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.709+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2508040110 con 0x7f2514140890 2020-07-18T03:29:13.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.866+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f251405b120 con 0x7f2514140890 2020-07-18T03:29:13.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.866+0000 7f2505ffb700 1 -- 172.21.15.47:0/3568458007 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2508019070 con 0x7f2514140890 2020-07-18T03:29:13.869 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.867+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f24ec04ff60 msgr2=0x7f24ec052420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.867+0000 7f251c080700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f24ec04ff60 0x7f24ec052420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f251000c000 tx=0x7f2510019000).stop 2020-07-18T03:29:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.867+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2514140890 msgr2=0x7f2514063980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.867+0000 7f251c080700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2514140890 0x7f2514063980 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f2514064550 tx=0x7f2508001e50).stop 2020-07-18T03:29:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 shutdown_connections 2020-07-18T03:29:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f24ec04ff60 0x7f24ec052420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 --2- 172.21.15.47:0/3568458007 >> v2:172.21.15.18:3300/0 conn(0x7f25140b20f0 0x7f25140b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 --2- 172.21.15.47:0/3568458007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2514140890 0x7f2514063980 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 >> 172.21.15.47:0/3568458007 conn(0x7f251405f5a0 msgr2=0x7f251405d270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 shutdown_connections 2020-07-18T03:29:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.868+0000 7f251c080700 1 -- 172.21.15.47:0/3568458007 wait complete. 2020-07-18T03:29:13.880 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-07-18T03:29:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.993+0000 7fb39e0dc700 1 Processor -- start 2020-07-18T03:29:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.993+0000 7fb39e0dc700 1 -- start start 2020-07-18T03:29:13.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb39e0dc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980628a0 0x7fb39813db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb39e0dc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3980af080 0x7fb3980af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:13.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb39e0dc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3981441f0 con 0x7fb3980ad990 2020-07-18T03:29:13.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb39e0dc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb398143fe0 con 0x7fb3980af080 2020-07-18T03:29:13.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb39e0dc700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb398144490 con 0x7fb3980628a0 2020-07-18T03:29:13.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3980af080 0x7fb3980af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:13.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb396ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb3980ad990 0x7fb398061b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41818/0 (socket says 172.21.15.47:41818) 2020-07-18T03:29:13.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3980af080 0x7fb3980af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60732/0 (socket says 172.21.15.47:60732) 2020-07-18T03:29:13.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb396ffd700 1 -- 172.21.15.47:0/2140655566 learned_addr learned my addr 172.21.15.47:0/2140655566 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:14.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb3977fe700 1 --2- 172.21.15.47:0/2140655566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980628a0 0x7fb39813db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 -- 172.21.15.47:0/2140655566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980628a0 msgr2=0x7fb39813db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:14.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 --2- 172.21.15.47:0/2140655566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980628a0 0x7fb39813db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 -- 172.21.15.47:0/2140655566 >> v1:172.21.15.47:6789/0 conn(0x7fb3980ad990 legacy=0x7fb398061b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 -- 172.21.15.47:0/2140655566 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb398143c80 con 0x7fb3980af080 2020-07-18T03:29:14.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb3967fc700 1 -- 172.21.15.47:0/2140655566 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158960800 0 0) 0x7fb3981441f0 con 0x7fb3980ad990 2020-07-18T03:29:14.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.994+0000 7fb397fff700 1 --2- 172.21.15.47:0/2140655566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3980af080 0x7fb3980af510 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fb3900032b0 tx=0x7fb3900091d0).ready entity=mon.1 client_cookie=5690ba769541e925 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:14.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb3967fc700 1 -- 172.21.15.47:0/2140655566 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39000bb90 con 0x7fb3980af080 2020-07-18T03:29:14.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb3967fc700 1 -- 172.21.15.47:0/2140655566 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39000bcf0 con 0x7fb3980af080 2020-07-18T03:29:14.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb3967fc700 1 -- 172.21.15.47:0/2140655566 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb390010660 con 0x7fb3980af080 2020-07-18T03:29:14.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2140655566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3980af080 msgr2=0x7fb3980af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 --2- 172.21.15.47:0/2140655566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3980af080 0x7fb3980af510 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fb3900032b0 tx=0x7fb3900091d0).stop 2020-07-18T03:29:14.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2140655566 shutdown_connections 2020-07-18T03:29:14.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 --2- 172.21.15.47:0/2140655566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3980af080 0x7fb3980af510 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 --2- 172.21.15.47:0/2140655566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980628a0 0x7fb39813db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2140655566 >> 172.21.15.47:0/2140655566 conn(0x7fb3980b92e0 msgr2=0x7fb3980b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:14.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2140655566 shutdown_connections 2020-07-18T03:29:14.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.995+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2140655566 wait complete. 2020-07-18T03:29:14.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.996+0000 7fb39e0dc700 1 Processor -- start 2020-07-18T03:29:14.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.996+0000 7fb39e0dc700 1 -- start start 2020-07-18T03:29:14.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb39e0dc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 0x7fb398159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb39e0dc700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb398159880 con 0x7fb3980628a0 2020-07-18T03:29:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb39e0dc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb398155700 con 0x7fb39813e400 2020-07-18T03:29:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb39e0dc700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981559a0 con 0x7fb3980ad990 2020-07-18T03:29:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb397fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb39813e400 0x7fb3980b64e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41826/0 (socket says 172.21.15.47:41826) 2020-07-18T03:29:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb397fff700 1 -- 172.21.15.47:0/2838847511 learned_addr learned my addr 172.21.15.47:0/2838847511 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb396ffd700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 0x7fb398159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664646399 0 0) 0x7fb398155700 con 0x7fb39813e400 2020-07-18T03:29:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.997+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb370003240 con 0x7fb39813e400 2020-07-18T03:29:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3422744862 0 0) 0x7fb370003240 con 0x7fb39813e400 2020-07-18T03:29:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb398155700 con 0x7fb39813e400 2020-07-18T03:29:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fb39000be60 con 0x7fb39813e400 2020-07-18T03:29:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 msgr2=0x7fb398159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 0x7fb398159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 >> v1:172.21.15.47:6789/0 conn(0x7fb39813e400 legacy=0x7fb3980b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 >> v1:172.21.15.18:6789/0 conn(0x7fb3980628a0 legacy=0x7fb398155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 0x7fb370003800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb370003cf0 0x7fb3700044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3700049d0 0x7fb370004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981559a0 con 0x7fb370003cf0 2020-07-18T03:29:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370004270 con 0x7fb3700049d0 2020-07-18T03:29:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37000b730 con 0x7fb3980ad990 2020-07-18T03:29:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3700049d0 0x7fb370004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb3977fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 0x7fb370003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb396ffd700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb370003cf0 0x7fb3700044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 -- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 msgr2=0x7fb370003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb3980ad990 0x7fb370003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 -- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb370003cf0 msgr2=0x7fb3700044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb370003cf0 0x7fb3700044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3981553f0 con 0x7fb3700049d0 2020-07-18T03:29:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb398159390 con 0x7fb3700049d0 2020-07-18T03:29:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb398152610 con 0x7fb3700049d0 2020-07-18T03:29:14.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.998+0000 7fb397fff700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3700049d0 0x7fb370004e50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb3900033f0 tx=0x7fb390018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:14.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.999+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39001d070 con 0x7fb3700049d0 2020-07-18T03:29:14.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.999+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb390022420 con 0x7fb3700049d0 2020-07-18T03:29:14.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.999+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39000bb90 con 0x7fb3700049d0 2020-07-18T03:29:14.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.999+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3980b57b0 con 0x7fb3700049d0 2020-07-18T03:29:14.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:13.999+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb390022780 con 0x7fb3700049d0 2020-07-18T03:29:14.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.000+0000 7fb3777fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb370058350 0x7fb37005a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.000+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb3900693b0 con 0x7fb3700049d0 2020-07-18T03:29:14.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.000+0000 7fb3977fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb370058350 0x7fb37005a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.000+0000 7fb3977fe700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb370058350 0x7fb37005a810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fb38800e000 tx=0x7fb388006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:14.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.003+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb390041e80 con 0x7fb3700049d0 2020-07-18T03:29:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.171+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb398060050 con 0x7fb3700049d0 2020-07-18T03:29:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.171+0000 7fb3777fe700 1 -- 172.21.15.47:0/2838847511 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb3900454a0 con 0x7fb3700049d0 2020-07-18T03:29:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:14.175 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2020-07-18T03:29:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2020-07-18T03:29:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2020-07-18T03:29:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2020-07-18T03:29:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2020-07-18T03:29:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:14.180 INFO:teuthology.orchestra.run.smithi047.stdout:] 2020-07-18T03:29:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb370058350 msgr2=0x7fb37005a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb370058350 0x7fb37005a810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fb38800e000 tx=0x7fb388006dd0).stop 2020-07-18T03:29:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3700049d0 msgr2=0x7fb370004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 --2- 172.21.15.47:0/2838847511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3700049d0 0x7fb370004e50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb3900033f0 tx=0x7fb390018000).stop 2020-07-18T03:29:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb397fff700 1 -- 172.21.15.47:0/2838847511 reap_dead start 2020-07-18T03:29:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb397fff700 1 -- 172.21.15.47:0/2838847511 reap_dead start 2020-07-18T03:29:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 shutdown_connections 2020-07-18T03:29:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 >> 172.21.15.47:0/2838847511 conn(0x7fb3980b92e0 msgr2=0x7fb39805f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 shutdown_connections 2020-07-18T03:29:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.173+0000 7fb39e0dc700 1 -- 172.21.15.47:0/2838847511 wait complete. 2020-07-18T03:29:14.187 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-07-18T03:29:14.200 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-07-18T03:29:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.317+0000 7f7d9a68a700 1 Processor -- start 2020-07-18T03:29:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.317+0000 7f7d9a68a700 1 -- start start 2020-07-18T03:29:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d9a68a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d940b3a10 0x7f7d940b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d9a68a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d94147de0 con 0x7f7d940b59b0 2020-07-18T03:29:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d9a68a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d940ae000 con 0x7f7d940ad330 2020-07-18T03:29:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d9a68a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d94148010 con 0x7f7d940b3a10 2020-07-18T03:29:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d98c27700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d940b3a10 0x7f7d940b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d98c27700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d940b3a10 0x7f7d940b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60752/0 (socket says 172.21.15.47:60752) 2020-07-18T03:29:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d93fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d940ad330 0x7f7d94143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41836/0 (socket says 172.21.15.47:41836) 2020-07-18T03:29:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.318+0000 7f7d98c27700 1 -- 172.21.15.47:0/74686055 learned_addr learned my addr 172.21.15.47:0/74686055 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d98c27700 1 -- 172.21.15.47:0/74686055 >> v1:172.21.15.47:6789/0 conn(0x7f7d940ad330 legacy=0x7f7d94143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d98c27700 1 -- 172.21.15.47:0/74686055 >> v1:172.21.15.18:6789/0 conn(0x7f7d940b59b0 legacy=0x7f7d940b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d98c27700 1 -- 172.21.15.47:0/74686055 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940ad710 con 0x7f7d940b3a10 2020-07-18T03:29:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d98c27700 1 --2- 172.21.15.47:0/74686055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b3a10 0x7f7d940b3e80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f7d8c0032b0 tx=0x7f7d8c009270).ready entity=mon.1 client_cookie=1297407e45b9b4ca server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d92ffd700 1 -- 172.21.15.47:0/74686055 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8c00bc00 con 0x7f7d940b3a10 2020-07-18T03:29:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d92ffd700 1 -- 172.21.15.47:0/74686055 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c00bd60 con 0x7f7d940b3a10 2020-07-18T03:29:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d92ffd700 1 -- 172.21.15.47:0/74686055 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8c011400 con 0x7f7d940b3a10 2020-07-18T03:29:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 -- 172.21.15.47:0/74686055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b3a10 msgr2=0x7f7d940b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/74686055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b3a10 0x7f7d940b3e80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f7d8c0032b0 tx=0x7f7d8c009270).stop 2020-07-18T03:29:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 -- 172.21.15.47:0/74686055 shutdown_connections 2020-07-18T03:29:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/74686055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b3a10 0x7f7d940b3e80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 -- 172.21.15.47:0/74686055 >> 172.21.15.47:0/74686055 conn(0x7f7d9405f5a0 msgr2=0x7f7d9405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 -- 172.21.15.47:0/74686055 shutdown_connections 2020-07-18T03:29:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.319+0000 7f7d9a68a700 1 -- 172.21.15.47:0/74686055 wait complete. 2020-07-18T03:29:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.320+0000 7f7d9a68a700 1 Processor -- start 2020-07-18T03:29:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d9a68a700 1 -- start start 2020-07-18T03:29:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d9a68a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d940b59b0 0x7f7d940b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d9a68a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7d941367b0 0x7f7d94138670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d9a68a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d941381d0 con 0x7f7d940ad330 2020-07-18T03:29:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d9a68a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d94137fc0 con 0x7f7d941367b0 2020-07-18T03:29:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d9a68a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d94137c20 con 0x7f7d940b59b0 2020-07-18T03:29:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d937fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d940b59b0 0x7f7d940b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d937fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d940b59b0 0x7f7d940b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60756/0 (socket says 172.21.15.47:60756) 2020-07-18T03:29:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d937fe700 1 -- 172.21.15.47:0/2401142359 learned_addr learned my addr 172.21.15.47:0/2401142359 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.321+0000 7f7d98c27700 1 --2- 172.21.15.47:0/2401142359 >> v2:172.21.15.18:3300/0 conn(0x7f7d941367b0 0x7f7d94138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d937fe700 1 -- 172.21.15.47:0/2401142359 >> v2:172.21.15.18:3300/0 conn(0x7f7d941367b0 msgr2=0x7f7d94138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d937fe700 1 --2- 172.21.15.47:0/2401142359 >> v2:172.21.15.18:3300/0 conn(0x7f7d941367b0 0x7f7d94138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d937fe700 1 -- 172.21.15.47:0/2401142359 >> v1:172.21.15.18:6789/0 conn(0x7f7d940ad330 legacy=0x7f7d940b9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d937fe700 1 -- 172.21.15.47:0/2401142359 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940b2660 con 0x7f7d940b59b0 2020-07-18T03:29:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d98c27700 1 --2- 172.21.15.47:0/2401142359 >> v2:172.21.15.18:3300/0 conn(0x7f7d941367b0 0x7f7d94138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d937fe700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b59b0 0x7f7d940b2170 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f7d940ae4e0 tx=0x7f7d88007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8800e4a0 con 0x7f7d940b59b0 2020-07-18T03:29:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8800e600 con 0x7f7d940b59b0 2020-07-18T03:29:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8800e960 con 0x7f7d940b59b0 2020-07-18T03:29:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d94138b60 con 0x7f7d940b59b0 2020-07-18T03:29:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.322+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d940b31d0 con 0x7f7d940b59b0 2020-07-18T03:29:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.323+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d94137d60 con 0x7f7d940b59b0 2020-07-18T03:29:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.323+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7d88019030 con 0x7f7d940b59b0 2020-07-18T03:29:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.323+0000 7f7d73fff700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d6c04ff20 0x7f7d6c0523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.323+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7d88066e70 con 0x7f7d940b59b0 2020-07-18T03:29:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.323+0000 7f7d93fff700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d6c04ff20 0x7f7d6c0523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.324+0000 7f7d93fff700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d6c04ff20 0x7f7d6c0523e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f7d940b9e00 tx=0x7f7d84007800).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.327+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7d8803f940 con 0x7f7d940b59b0 2020-07-18T03:29:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:14.486+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f7d94138360 con 0x7f7d940b59b0 2020-07-18T03:29:15.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.412+0000 7f7d73fff700 1 -- 172.21.15.47:0/2401142359 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v39) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7d88017020 con 0x7f7d940b59b0 2020-07-18T03:29:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.414+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d6c04ff20 msgr2=0x7f7d6c0523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.414+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d6c04ff20 0x7f7d6c0523e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f7d940b9e00 tx=0x7f7d84007800).stop 2020-07-18T03:29:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.414+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b59b0 msgr2=0x7f7d940b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.414+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b59b0 0x7f7d940b2170 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f7d940ae4e0 tx=0x7f7d88007c30).stop 2020-07-18T03:29:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 shutdown_connections 2020-07-18T03:29:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7d6c04ff20 0x7f7d6c0523e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/2401142359 >> v2:172.21.15.18:3300/0 conn(0x7f7d941367b0 0x7f7d94138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 --2- 172.21.15.47:0/2401142359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d940b59b0 0x7f7d940b2170 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 >> 172.21.15.47:0/2401142359 conn(0x7f7d9405f5a0 msgr2=0x7f7d94150910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 shutdown_connections 2020-07-18T03:29:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.415+0000 7f7d9a68a700 1 -- 172.21.15.47:0/2401142359 wait complete. 2020-07-18T03:29:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2020-07-18T03:29:15.431 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-07-18T03:29:15.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.553+0000 7f1aa77fd700 1 Processor -- start 2020-07-18T03:29:15.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 -- start start 2020-07-18T03:29:15.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00ad330 0x7f1aa0143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1aa00b59b0 0x7f1aa00b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa0147de0 con 0x7f1aa00b6300 2020-07-18T03:29:15.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa00ae000 con 0x7f1aa00b59b0 2020-07-18T03:29:15.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.554+0000 7f1aa77fd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0148010 con 0x7f1aa00ad330 2020-07-18T03:29:15.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1aa00b59b0 0x7f1aa00b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1aa00b59b0 0x7f1aa00b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60766/0 (socket says 172.21.15.47:60766) 2020-07-18T03:29:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 -- 172.21.15.47:0/2169389451 learned_addr learned my addr 172.21.15.47:0/2169389451 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa5d9a700 1 --2- 172.21.15.47:0/2169389451 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa5599700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00ad330 0x7f1aa0143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 -- 172.21.15.47:0/2169389451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00ad330 msgr2=0x7f1aa0143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00ad330 0x7f1aa0143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 -- 172.21.15.47:0/2169389451 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 msgr2=0x7f1aa00b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/2169389451 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 -- 172.21.15.47:0/2169389451 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa00ad710 con 0x7f1aa00b59b0 2020-07-18T03:29:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa5599700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00ad330 0x7f1aa0143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:15.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa00b59b0 0x7f1aa00b5e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f1a9c002f30 tx=0x7f1a9c006e90).ready entity=mon.1 client_cookie=9b1b4316493852a6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:15.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1a93fff700 1 -- 172.21.15.47:0/2169389451 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9c009850 con 0x7f1aa00b59b0 2020-07-18T03:29:15.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1a93fff700 1 -- 172.21.15.47:0/2169389451 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c0099b0 con 0x7f1aa00b59b0 2020-07-18T03:29:15.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1a93fff700 1 -- 172.21.15.47:0/2169389451 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9c009d10 con 0x7f1aa00b59b0 2020-07-18T03:29:15.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 -- 172.21.15.47:0/2169389451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa00b59b0 msgr2=0x7f1aa00b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa00b59b0 0x7f1aa00b5e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f1a9c002f30 tx=0x7f1a9c006e90).stop 2020-07-18T03:29:15.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 -- 172.21.15.47:0/2169389451 shutdown_connections 2020-07-18T03:29:15.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 --2- 172.21.15.47:0/2169389451 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00ad330 0x7f1aa0143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 --2- 172.21.15.47:0/2169389451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa00b59b0 0x7f1aa00b5e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.555+0000 7f1aa77fd700 1 -- 172.21.15.47:0/2169389451 >> 172.21.15.47:0/2169389451 conn(0x7f1aa005f5a0 msgr2=0x7f1aa005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:15.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.556+0000 7f1aa77fd700 1 -- 172.21.15.47:0/2169389451 shutdown_connections 2020-07-18T03:29:15.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.556+0000 7f1aa77fd700 1 -- 172.21.15.47:0/2169389451 wait complete. 2020-07-18T03:29:15.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.557+0000 7f1aa77fd700 1 Processor -- start 2020-07-18T03:29:15.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.557+0000 7f1aa77fd700 1 -- start start 2020-07-18T03:29:15.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa77fd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa77fd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 0x7f1aa013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa77fd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa01409b0 con 0x7f1aa00ad330 2020-07-18T03:29:15.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa77fd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa013e1b0 con 0x7f1aa00b6300 2020-07-18T03:29:15.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa77fd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0140c50 con 0x7f1aa013d830 2020-07-18T03:29:15.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa5599700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1aa00ad330 0x7f1aa013f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41856/0 (socket says 172.21.15.47:41856) 2020-07-18T03:29:15.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa5599700 1 -- 172.21.15.47:0/3340903277 learned_addr learned my addr 172.21.15.47:0/3340903277 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:15.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097201122 0 0) 0x7f1aa01409b0 con 0x7f1aa00ad330 2020-07-18T03:29:15.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a78003240 con 0x7f1aa00ad330 2020-07-18T03:29:15.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/3340903277 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1aa5d9a700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 0x7f1aa013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.558+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1485149589 0 0) 0x7f1a78003240 con 0x7f1aa00ad330 2020-07-18T03:29:15.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1aa01409b0 con 0x7f1aa00ad330 2020-07-18T03:29:15.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f1a940012a0 con 0x7f1aa00ad330 2020-07-18T03:29:15.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 msgr2=0x7f1aa013dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 0x7f1aa013dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 msgr2=0x7f1aa00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 --2- 172.21.15.47:0/3340903277 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b6300 0x7f1aa00b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 >> v1:172.21.15.47:6789/0 conn(0x7f1aa00ad330 legacy=0x7f1aa013f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a78003800 0x7f1a78003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a780044e0 0x7f1a78004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 0x7f1a78004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa013e1b0 con 0x7f1a78003800 2020-07-18T03:29:15.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0140c50 con 0x7f1a780044e0 2020-07-18T03:29:15.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a78005300 con 0x7f1aa013d830 2020-07-18T03:29:15.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a780044e0 0x7f1a78004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa5599700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a78003800 0x7f1a78003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa5d9a700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 0x7f1a78004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 -- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 msgr2=0x7f1a78004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013d830 0x7f1a78004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 -- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a78003800 msgr2=0x7f1a78003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:15.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a78003800 0x7f1a78003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:15.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa00b2cc0 con 0x7f1a780044e0 2020-07-18T03:29:15.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa013ec90 con 0x7f1a780044e0 2020-07-18T03:29:15.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa013de80 con 0x7f1a780044e0 2020-07-18T03:29:15.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.559+0000 7f1aa4d98700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a780044e0 0x7f1a78004920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f1a9c01c000 tx=0x7f1a9c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:15.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.560+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9c01f440 con 0x7f1a780044e0 2020-07-18T03:29:15.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.560+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c01f5a0 con 0x7f1a780044e0 2020-07-18T03:29:15.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.560+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9c01f900 con 0x7f1a780044e0 2020-07-18T03:29:15.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.560+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0060420 con 0x7f1a780044e0 2020-07-18T03:29:15.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.560+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1a9c01a040 con 0x7f1a780044e0 2020-07-18T03:29:15.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.561+0000 7f1a917fa700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a78058130 0x7f1a7805a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:15.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.561+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f1a9c069d70 con 0x7f1a780044e0 2020-07-18T03:29:15.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.561+0000 7f1aa5599700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a78058130 0x7f1a7805a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:15.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.561+0000 7f1aa5599700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a78058130 0x7f1a7805a5f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1a9400f000 tx=0x7f1a94008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:15.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.564+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1a9c042780 con 0x7f1a780044e0 2020-07-18T03:29:15.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:15.727+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f1aa005b120 con 0x7f1a780044e0 2020-07-18T03:29:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.584+0000 7f1a917fa700 1 -- 172.21.15.47:0/3340903277 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v41) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1a9c045da0 con 0x7f1a780044e0 2020-07-18T03:29:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a78058130 msgr2=0x7f1a7805a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa77fd700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a78058130 0x7f1a7805a5f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1a9400f000 tx=0x7f1a94008040).stop 2020-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a780044e0 msgr2=0x7f1a78004920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa77fd700 1 --2- 172.21.15.47:0/3340903277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a780044e0 0x7f1a78004920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f1a9c01c000 tx=0x7f1a9c015000).stop 2020-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa5d9a700 1 -- 172.21.15.47:0/3340903277 reap_dead start 2020-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa5d9a700 1 -- 172.21.15.47:0/3340903277 reap_dead start 2020-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 shutdown_connections 2020-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.586+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 >> 172.21.15.47:0/3340903277 conn(0x7f1aa005f5a0 msgr2=0x7f1aa01427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.587+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 shutdown_connections 2020-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.587+0000 7f1aa77fd700 1 -- 172.21.15.47:0/3340903277 wait complete. 2020-07-18T03:29:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2020-07-18T03:29:16.603 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-07-18T03:29:16.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.726+0000 7fe547cdb700 1 Processor -- start 2020-07-18T03:29:16.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.726+0000 7fe547cdb700 1 -- start start 2020-07-18T03:29:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe547cdb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 0x7fe5400620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe547cdb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5400af4b0 con 0x7fe540142a20 2020-07-18T03:29:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe547cdb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5400b6d80 con 0x7fe5400b3af0 2020-07-18T03:29:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe547cdb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400af710 con 0x7fe540061c60 2020-07-18T03:29:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe546278700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe5400b3af0 0x7fe5400b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41870/0 (socket says 172.21.15.47:41870) 2020-07-18T03:29:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe546278700 1 -- 172.21.15.47:0/3772543964 learned_addr learned my addr 172.21.15.47:0/3772543964 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe545276700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 0x7fe5400620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952767591 0 0) 0x7fe5400b6d80 con 0x7fe5400b3af0 2020-07-18T03:29:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe52c003240 con 0x7fe5400b3af0 2020-07-18T03:29:16.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3857812201 0 0) 0x7fe52c003240 con 0x7fe5400b3af0 2020-07-18T03:29:16.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 msgr2=0x7fe5400620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:16.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 0x7fe5400620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 >> v1:172.21.15.18:6789/0 conn(0x7fe540142a20 legacy=0x7fe540142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400af9b0 con 0x7fe5400b3af0 2020-07-18T03:29:16.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.727+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fe5300016d0 con 0x7fe5400b3af0 2020-07-18T03:29:16.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 >> v1:172.21.15.47:6789/0 conn(0x7fe5400b3af0 legacy=0x7fe5400b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe52c003730 0x7fe52c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe52c0044e0 0x7fe52c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 0x7fe52c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400af710 con 0x7fe52c003730 2020-07-18T03:29:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c005390 con 0x7fe52c0044e0 2020-07-18T03:29:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c004320 con 0x7fe540061c60 2020-07-18T03:29:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe52c0044e0 0x7fe52c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545a77700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe52c003730 0x7fe52c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe546278700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 0x7fe52c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 -- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 msgr2=0x7fe52c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe540061c60 0x7fe52c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 -- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe52c003730 msgr2=0x7fe52c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe52c003730 0x7fe52c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 -- 172.21.15.47:0/3772543964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c003240 con 0x7fe52c0044e0 2020-07-18T03:29:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe545276700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe52c0044e0 0x7fe52c004960 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fe53400c000 tx=0x7fe534008040).ready entity=mon.1 client_cookie=5241a8fde0762e65 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53400b2d0 con 0x7fe52c0044e0 2020-07-18T03:29:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400e8a0 con 0x7fe52c0044e0 2020-07-18T03:29:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe544a75700 1 -- 172.21.15.47:0/3772543964 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53400ec00 con 0x7fe52c0044e0 2020-07-18T03:29:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe547cdb700 1 -- 172.21.15.47:0/3772543964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe52c0044e0 msgr2=0x7fe52c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe547cdb700 1 --2- 172.21.15.47:0/3772543964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe52c0044e0 0x7fe52c004960 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fe53400c000 tx=0x7fe534008040).stop 2020-07-18T03:29:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe546278700 1 -- 172.21.15.47:0/3772543964 reap_dead start 2020-07-18T03:29:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe547cdb700 1 -- 172.21.15.47:0/3772543964 shutdown_connections 2020-07-18T03:29:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.728+0000 7fe547cdb700 1 -- 172.21.15.47:0/3772543964 >> 172.21.15.47:0/3772543964 conn(0x7fe54005f5a0 msgr2=0x7fe54005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.729+0000 7fe547cdb700 1 -- 172.21.15.47:0/3772543964 shutdown_connections 2020-07-18T03:29:16.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.729+0000 7fe547cdb700 1 -- 172.21.15.47:0/3772543964 wait complete. 2020-07-18T03:29:16.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.730+0000 7fe547cdb700 1 Processor -- start 2020-07-18T03:29:16.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.730+0000 7fe547cdb700 1 -- start start 2020-07-18T03:29:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe547cdb700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe54013f9f0 0x7fe5400b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe547cdb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5400b7670 0x7fe540064dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe547cdb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe540155ae0 con 0x7fe5400652b0 2020-07-18T03:29:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe547cdb700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400b0630 con 0x7fe54013f9f0 2020-07-18T03:29:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe547cdb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe540155760 con 0x7fe5400b7670 2020-07-18T03:29:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5400b7670 0x7fe540064dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe546278700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe5400652b0 0x7fe5400b0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41886/0 (socket says 172.21.15.47:41886) 2020-07-18T03:29:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5400b7670 0x7fe540064dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60796/0 (socket says 172.21.15.47:60796) 2020-07-18T03:29:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe546278700 1 -- 172.21.15.47:0/3730314686 learned_addr learned my addr 172.21.15.47:0/3730314686 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545a77700 1 --2- 172.21.15.47:0/3730314686 >> v2:172.21.15.18:3300/0 conn(0x7fe54013f9f0 0x7fe5400b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 -- 172.21.15.47:0/3730314686 >> v2:172.21.15.18:3300/0 conn(0x7fe54013f9f0 msgr2=0x7fe5400b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 --2- 172.21.15.47:0/3730314686 >> v2:172.21.15.18:3300/0 conn(0x7fe54013f9f0 0x7fe5400b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 -- 172.21.15.47:0/3730314686 >> v1:172.21.15.47:6789/0 conn(0x7fe5400652b0 legacy=0x7fe5400b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801440078 0 0) 0x7fe540155ae0 con 0x7fe5400652b0 2020-07-18T03:29:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 -- 172.21.15.47:0/3730314686 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540062f60 con 0x7fe5400b7670 2020-07-18T03:29:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe545276700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5400b7670 0x7fe540064dc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fe53400c000 tx=0x7fe53401a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53401c830 con 0x7fe5400b7670 2020-07-18T03:29:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53401c990 con 0x7fe5400b7670 2020-07-18T03:29:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.731+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe540063150 con 0x7fe5400b7670 2020-07-18T03:29:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.732+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53401ccf0 con 0x7fe5400b7670 2020-07-18T03:29:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.732+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5400b1420 con 0x7fe5400b7670 2020-07-18T03:29:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.733+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5400b0800 con 0x7fe5400b7670 2020-07-18T03:29:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.733+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe53402b060 con 0x7fe5400b7670 2020-07-18T03:29:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.734+0000 7fe53dffb700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe51804fec0 0x7fe518052380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.734+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fe534069d90 con 0x7fe5400b7670 2020-07-18T03:29:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.734+0000 7fe545a77700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe51804fec0 0x7fe518052380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.734+0000 7fe545a77700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe51804fec0 0x7fe518052380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fe538003000 tx=0x7fe538007e60).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:16.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.737+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe534042540 con 0x7fe5400b7670 2020-07-18T03:29:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.946+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe540155c70 con 0x7fe5400b7670 2020-07-18T03:29:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.951+0000 7fe53dffb700 1 -- 172.21.15.47:0/3730314686 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v16) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe5340160d0 con 0x7fe5400b7670 2020-07-18T03:29:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe51804fec0 msgr2=0x7fe518052380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe51804fec0 0x7fe518052380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fe538003000 tx=0x7fe538007e60).stop 2020-07-18T03:29:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5400b7670 msgr2=0x7fe540064dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5400b7670 0x7fe540064dc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fe53400c000 tx=0x7fe53401a000).stop 2020-07-18T03:29:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 shutdown_connections 2020-07-18T03:29:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe51804fec0 0x7fe518052380 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 --2- 172.21.15.47:0/3730314686 >> v2:172.21.15.18:3300/0 conn(0x7fe54013f9f0 0x7fe5400b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 --2- 172.21.15.47:0/3730314686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5400b7670 0x7fe540064dc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.956+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 >> 172.21.15.47:0/3730314686 conn(0x7fe54005f5a0 msgr2=0x7fe54005d270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.957+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 shutdown_connections 2020-07-18T03:29:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:16.957+0000 7fe547cdb700 1 -- 172.21.15.47:0/3730314686 wait complete. 2020-07-18T03:29:16.964 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 7 and data pool 8 2020-07-18T03:29:16.975 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.088+0000 7f66c012c700 1 Processor -- start 2020-07-18T03:29:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.089+0000 7f66c012c700 1 -- start start 2020-07-18T03:29:17.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.089+0000 7f66c012c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f66b80b41c0 0x7f66b80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.089+0000 7f66c012c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b80ad7c0 0x7f66b81428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.089+0000 7f66c012c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b80b3ac0 con 0x7f66b80b5370 2020-07-18T03:29:17.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.089+0000 7f66c012c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b80b5a10 con 0x7f66b80b41c0 2020-07-18T03:29:17.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.089+0000 7f66c012c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80b0040 con 0x7f66b80ad7c0 2020-07-18T03:29:17.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bdec8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f66b80b5370 0x7f66b805fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41898/0 (socket says 172.21.15.47:41898) 2020-07-18T03:29:17.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bdec8700 1 -- 172.21.15.47:0/297931241 learned_addr learned my addr 172.21.15.47:0/297931241 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:17.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bd6c7700 1 --2- 172.21.15.47:0/297931241 >> v2:172.21.15.18:3300/0 conn(0x7f66b80b41c0 0x7f66b80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66be6c9700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b80ad7c0 0x7f66b81428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193847886 0 0) 0x7f66b80b3ac0 con 0x7f66b80b5370 2020-07-18T03:29:17.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a0003240 con 0x7f66b80b5370 2020-07-18T03:29:17.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1052817737 0 0) 0x7f66a0003240 con 0x7f66b80b5370 2020-07-18T03:29:17.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b80ad7c0 msgr2=0x7f66b81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b80ad7c0 0x7f66b81428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 >> v2:172.21.15.18:3300/0 conn(0x7f66b80b41c0 msgr2=0x7f66b80ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 --2- 172.21.15.47:0/297931241 >> v2:172.21.15.18:3300/0 conn(0x7f66b80b41c0 0x7f66b80ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80b3d20 con 0x7f66b80b5370 2020-07-18T03:29:17.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f66ac0016d0 con 0x7f66b80b5370 2020-07-18T03:29:17.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 >> v1:172.21.15.47:6789/0 conn(0x7f66b80b5370 legacy=0x7f66b805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a0003730 0x7f66a0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66a00044e0 0x7f66a0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a0004e50 0x7f66a000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80b5a10 con 0x7f66a0003730 2020-07-18T03:29:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80b0040 con 0x7f66a0004e50 2020-07-18T03:29:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00042a0 con 0x7f66a00044e0 2020-07-18T03:29:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66be6c9700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b80ad7c0 0x7f66b81428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bd6c7700 1 --2- 172.21.15.47:0/297931241 >> v2:172.21.15.18:3300/0 conn(0x7f66b80b41c0 0x7f66b80ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66be6c9700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a0004e50 0x7f66a000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.090+0000 7f66bdec8700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a0003730 0x7f66a0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66bd6c7700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66a00044e0 0x7f66a0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 -- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66a00044e0 msgr2=0x7f66a0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66a00044e0 0x7f66a0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 -- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a0003730 msgr2=0x7f66a0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a0003730 0x7f66a0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 -- 172.21.15.47:0/297931241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0003240 con 0x7f66a0004e50 2020-07-18T03:29:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a0004e50 0x7f66a000b730 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f66b0000b20 tx=0x7f66b00091d0).ready entity=mon.1 client_cookie=acd6719832205c6a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b000ee20 con 0x7f66a0004e50 2020-07-18T03:29:17.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b00123f0 con 0x7f66a0004e50 2020-07-18T03:29:17.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66bcec6700 1 -- 172.21.15.47:0/297931241 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b0012750 con 0x7f66a0004e50 2020-07-18T03:29:17.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66c012c700 1 -- 172.21.15.47:0/297931241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a0004e50 msgr2=0x7f66a000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66c012c700 1 --2- 172.21.15.47:0/297931241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a0004e50 0x7f66a000b730 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f66b0000b20 tx=0x7f66b00091d0).stop 2020-07-18T03:29:17.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66be6c9700 1 -- 172.21.15.47:0/297931241 reap_dead start 2020-07-18T03:29:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66c012c700 1 -- 172.21.15.47:0/297931241 shutdown_connections 2020-07-18T03:29:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66c012c700 1 -- 172.21.15.47:0/297931241 >> 172.21.15.47:0/297931241 conn(0x7f66b80b9340 msgr2=0x7f66b80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66c012c700 1 -- 172.21.15.47:0/297931241 shutdown_connections 2020-07-18T03:29:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.091+0000 7f66c012c700 1 -- 172.21.15.47:0/297931241 wait complete. 2020-07-18T03:29:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.092+0000 7f66c012c700 1 Processor -- start 2020-07-18T03:29:17.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66c012c700 1 -- start start 2020-07-18T03:29:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66c012c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66b8066300 0x7f66b8156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66c012c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b8141b50 0x7f66b8156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66c012c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b8157110 con 0x7f66b8065f20 2020-07-18T03:29:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66c012c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b8147e60 con 0x7f66b8066300 2020-07-18T03:29:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66c012c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8146d70 con 0x7f66b8141b50 2020-07-18T03:29:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66bd6c7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66b8066300 0x7f66b8156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66bd6c7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66b8066300 0x7f66b8156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60824/0 (socket says 172.21.15.47:60824) 2020-07-18T03:29:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66bd6c7700 1 -- 172.21.15.47:0/3322851258 learned_addr learned my addr 172.21.15.47:0/3322851258 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.093+0000 7f66be6c9700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b8141b50 0x7f66b8156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66bd6c7700 1 -- 172.21.15.47:0/3322851258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b8141b50 msgr2=0x7f66b8156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66bd6c7700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b8141b50 0x7f66b8156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66bd6c7700 1 -- 172.21.15.47:0/3322851258 >> v1:172.21.15.47:6789/0 conn(0x7f66b8065f20 legacy=0x7f66b8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284647128 0 0) 0x7f66b8157110 con 0x7f66b8065f20 2020-07-18T03:29:17.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66bd6c7700 1 -- 172.21.15.47:0/3322851258 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8147010 con 0x7f66b8066300 2020-07-18T03:29:17.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66be6c9700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b8141b50 0x7f66b8156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:17.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66bd6c7700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66b8066300 0x7f66b8156730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f66b4008000 tx=0x7f66b4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b400e8f0 con 0x7f66b8066300 2020-07-18T03:29:17.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b400ea50 con 0x7f66b8066300 2020-07-18T03:29:17.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b81472b0 con 0x7f66b8066300 2020-07-18T03:29:17.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66b400edb0 con 0x7f66b8066300 2020-07-18T03:29:17.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.094+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b8155790 con 0x7f66b8066300 2020-07-18T03:29:17.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.095+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b80ba100 con 0x7f66b8066300 2020-07-18T03:29:17.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.095+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f66b4014470 con 0x7f66b8066300 2020-07-18T03:29:17.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.095+0000 7f66a9ffb700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f669004fe20 0x7f66900522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.096+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f66b4068190 con 0x7f66b8066300 2020-07-18T03:29:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.096+0000 7f66bdec8700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f669004fe20 0x7f66900522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.096+0000 7f66bdec8700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f669004fe20 0x7f66900522e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f66b80afb30 tx=0x7f66b80b3eb0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.099+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f66b40408c0 con 0x7f66b8066300 2020-07-18T03:29:17.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.256+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66b8147440 con 0x7f66b8066300 2020-07-18T03:29:17.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.257+0000 7f66a9ffb700 1 -- 172.21.15.47:0/3322851258 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f66b4043ee0 con 0x7f66b8066300 2020-07-18T03:29:17.260 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:17.260 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:16.949111+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:14.526774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:15.768057+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.258+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f669004fe20 msgr2=0x7f66900522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.258+0000 7f66c012c700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f669004fe20 0x7f66900522e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f66b80afb30 tx=0x7f66b80b3eb0).stop 2020-07-18T03:29:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.258+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66b8066300 msgr2=0x7f66b8156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.258+0000 7f66c012c700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66b8066300 0x7f66b8156730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f66b4008000 tx=0x7f66b4008040).stop 2020-07-18T03:29:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 shutdown_connections 2020-07-18T03:29:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66b8141b50 0x7f66b8156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f669004fe20 0x7f66900522e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 --2- 172.21.15.47:0/3322851258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66b8066300 0x7f66b8156730 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 >> 172.21.15.47:0/3322851258 conn(0x7f66b80b9340 msgr2=0x7f66b80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:17.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 shutdown_connections 2020-07-18T03:29:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.259+0000 7f66c012c700 1 -- 172.21.15.47:0/3322851258 wait complete. 2020-07-18T03:29:17.271 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.384+0000 7f18a3cc0700 1 Processor -- start 2020-07-18T03:29:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.384+0000 7f18a3cc0700 1 -- start start 2020-07-18T03:29:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a3cc0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f189c0b5370 0x7f189c05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a3cc0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c0ad9e0 0x7f189c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a3cc0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c0b3ac0 con 0x7f189c0b41c0 2020-07-18T03:29:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a3cc0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c0b5a10 con 0x7f189c0b5370 2020-07-18T03:29:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a3cc0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0b0040 con 0x7f189c0ad9e0 2020-07-18T03:29:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a1a5c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f189c0b5370 0x7f189c05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a225d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c0ad9e0 0x7f189c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a1a5c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f189c0b5370 0x7f189c05fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:46510/0 (socket says 172.21.15.47:46510) 2020-07-18T03:29:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.385+0000 7f18a1a5c700 1 -- 172.21.15.47:0/3381118032 learned_addr learned my addr 172.21.15.47:0/3381118032 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a1a5c700 1 -- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c0ad9e0 msgr2=0x7f189c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a1a5c700 1 --2- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c0ad9e0 0x7f189c1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a1a5c700 1 -- 172.21.15.47:0/3381118032 >> v1:172.21.15.18:6789/0 conn(0x7f189c0b41c0 legacy=0x7f189c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a1a5c700 1 -- 172.21.15.47:0/3381118032 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c0b3d20 con 0x7f189c0b5370 2020-07-18T03:29:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a225d700 1 --2- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c0ad9e0 0x7f189c1428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a1a5c700 1 --2- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0b5370 0x7f189c05fff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f189c0afb30 tx=0x7f188c006e90).ready entity=mon.0 client_cookie=558fc31bcc671875 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a0a5a700 1 -- 172.21.15.47:0/3381118032 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188c014070 con 0x7f189c0b5370 2020-07-18T03:29:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a0a5a700 1 -- 172.21.15.47:0/3381118032 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c009a00 con 0x7f189c0b5370 2020-07-18T03:29:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a0a5a700 1 -- 172.21.15.47:0/3381118032 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188c009d60 con 0x7f189c0b5370 2020-07-18T03:29:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 -- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0b5370 msgr2=0x7f189c05fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0b5370 0x7f189c05fff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f189c0afb30 tx=0x7f188c006e90).stop 2020-07-18T03:29:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 -- 172.21.15.47:0/3381118032 shutdown_connections 2020-07-18T03:29:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f189c0ad9e0 0x7f189c1428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/3381118032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0b5370 0x7f189c05fff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 -- 172.21.15.47:0/3381118032 >> 172.21.15.47:0/3381118032 conn(0x7f189c0b9340 msgr2=0x7f189c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 -- 172.21.15.47:0/3381118032 shutdown_connections 2020-07-18T03:29:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.386+0000 7f18a3cc0700 1 -- 172.21.15.47:0/3381118032 wait complete. 2020-07-18T03:29:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.388+0000 7f18a3cc0700 1 Processor -- start 2020-07-18T03:29:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.388+0000 7f18a3cc0700 1 -- start start 2020-07-18T03:29:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.388+0000 7f18a3cc0700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f189c0ad9e0 0x7f189c0622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.388+0000 7f18a3cc0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c062df0 con 0x7f189c0b41c0 2020-07-18T03:29:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.388+0000 7f18a3cc0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c063090 con 0x7f189c064e00 2020-07-18T03:29:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.388+0000 7f18a3cc0700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c154f80 con 0x7f189c0ad9e0 2020-07-18T03:29:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a225d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f189c064e00 0x7f189c150c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41926/0 (socket says 172.21.15.47:41926) 2020-07-18T03:29:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a225d700 1 -- 172.21.15.47:0/843410128 learned_addr learned my addr 172.21.15.47:0/843410128 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a1a5c700 1 --2- 172.21.15.47:0/843410128 >> v2:172.21.15.18:3300/0 conn(0x7f189c0ad9e0 0x7f189c0622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415921200 0 0) 0x7f189c063090 con 0x7f189c064e00 2020-07-18T03:29:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1874003240 con 0x7f189c064e00 2020-07-18T03:29:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a1a5c700 1 -- 172.21.15.47:0/843410128 >> v1:172.21.15.47:6789/0 conn(0x7f189c064e00 legacy=0x7f189c150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a1a5c700 1 -- 172.21.15.47:0/843410128 >> v1:172.21.15.18:6789/0 conn(0x7f189c0b41c0 legacy=0x7f189c154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155357771 0 0) 0x7f189c062df0 con 0x7f189c0b41c0 2020-07-18T03:29:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a1a5c700 1 -- 172.21.15.47:0/843410128 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c0627d0 con 0x7f189c0ad9e0 2020-07-18T03:29:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a1a5c700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0ad9e0 0x7f189c0622e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f189c0afb30 tx=0x7f188c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188c01d850 con 0x7f189c0ad9e0 2020-07-18T03:29:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c009890 con 0x7f189c0ad9e0 2020-07-18T03:29:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c151380 con 0x7f189c0ad9e0 2020-07-18T03:29:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.389+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188c01dba0 con 0x7f189c0ad9e0 2020-07-18T03:29:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.390+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c1517f0 con 0x7f189c0ad9e0 2020-07-18T03:29:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.391+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c0629c0 con 0x7f189c0ad9e0 2020-07-18T03:29:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.391+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f188c00e040 con 0x7f189c0ad9e0 2020-07-18T03:29:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.391+0000 7f1889ffb700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f18740502c0 0x7f1874052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.391+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f188c06aec0 con 0x7f189c0ad9e0 2020-07-18T03:29:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.391+0000 7f18a125b700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f18740502c0 0x7f1874052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.392+0000 7f18a125b700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f18740502c0 0x7f1874052780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1898001380 tx=0x7f1898007b80).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.396+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f188c043530 con 0x7f189c0ad9e0 2020-07-18T03:29:17.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.552+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f189c063b50 con 0x7f189c0ad9e0 2020-07-18T03:29:17.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.553+0000 7f1889ffb700 1 -- 172.21.15.47:0/843410128 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f189c063b50 con 0x7f189c0ad9e0 2020-07-18T03:29:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:17.556 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:16.949111+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:14.526774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:15.768057+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:17.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f18740502c0 msgr2=0x7f1874052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f18740502c0 0x7f1874052780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1898001380 tx=0x7f1898007b80).stop 2020-07-18T03:29:17.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0ad9e0 msgr2=0x7f189c0622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0ad9e0 0x7f189c0622e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f189c0afb30 tx=0x7f188c016000).stop 2020-07-18T03:29:17.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 shutdown_connections 2020-07-18T03:29:17.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f18740502c0 0x7f1874052780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 --2- 172.21.15.47:0/843410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f189c0ad9e0 0x7f189c0622e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 >> 172.21.15.47:0/843410128 conn(0x7f189c0b9340 msgr2=0x7f189c0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:17.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 shutdown_connections 2020-07-18T03:29:17.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.554+0000 7f18a3cc0700 1 -- 172.21.15.47:0/843410128 wait complete. 2020-07-18T03:29:17.568 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-07-18T03:29:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.684+0000 7f10de2c3700 1 Processor -- start 2020-07-18T03:29:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.684+0000 7f10de2c3700 1 -- start start 2020-07-18T03:29:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10de2c3700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10d8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10de2c3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10d80b59b0 0x7f10d80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10de2c3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d8147de0 con 0x7f10d80b6300 2020-07-18T03:29:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10de2c3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d80ae000 con 0x7f10d80b59b0 2020-07-18T03:29:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10de2c3700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8148010 con 0x7f10d80ad330 2020-07-18T03:29:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10dc860700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10d80b6300 0x7f10d80b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41932/0 (socket says 172.21.15.47:41932) 2020-07-18T03:29:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10dc860700 1 -- 172.21.15.47:0/451654549 learned_addr learned my addr 172.21.15.47:0/451654549 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10d7fff700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10d8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.685+0000 7f10d77fe700 1 --2- 172.21.15.47:0/451654549 >> v2:172.21.15.18:3300/0 conn(0x7f10d80b59b0 0x7f10d80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920203351 0 0) 0x7f10d8147de0 con 0x7f10d80b6300 2020-07-18T03:29:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c4003240 con 0x7f10d80b6300 2020-07-18T03:29:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 387932860 0 0) 0x7f10c4003240 con 0x7f10d80b6300 2020-07-18T03:29:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 msgr2=0x7f10d8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10d8143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 >> v2:172.21.15.18:3300/0 conn(0x7f10d80b59b0 msgr2=0x7f10d80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 --2- 172.21.15.47:0/451654549 >> v2:172.21.15.18:3300/0 conn(0x7f10d80b59b0 0x7f10d80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80ad710 con 0x7f10d80b6300 2020-07-18T03:29:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d7fff700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10d8143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f10d0001a50 con 0x7f10d80b6300 2020-07-18T03:29:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 >> v1:172.21.15.47:6789/0 conn(0x7f10d80b6300 legacy=0x7f10d80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10c4003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10c4003c20 0x7f10c40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10c40049d0 0x7f10c4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80ae000 con 0x7f10c4003c20 2020-07-18T03:29:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8148010 con 0x7f10c40049d0 2020-07-18T03:29:17.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40041e0 con 0x7f10d80ad330 2020-07-18T03:29:17.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10dc860700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10c40049d0 0x7f10c4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d7fff700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10c4003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10d77fe700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10c4003c20 0x7f10c40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10dc860700 1 -- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 msgr2=0x7f10c4003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10dc860700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10d80ad330 0x7f10c4003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10dc860700 1 -- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10c4003c20 msgr2=0x7f10c40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10dc860700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10c4003c20 0x7f10c40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.686+0000 7f10dc860700 1 -- 172.21.15.47:0/451654549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c4003240 con 0x7f10c40049d0 2020-07-18T03:29:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10dc860700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10c40049d0 0x7f10c4004e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f10d0001760 tx=0x7f10d0007ad0).ready entity=mon.1 client_cookie=7a608f014918e0b5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d0005b80 con 0x7f10c40049d0 2020-07-18T03:29:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d0005ce0 con 0x7f10c40049d0 2020-07-18T03:29:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10d6ffd700 1 -- 172.21.15.47:0/451654549 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d00072c0 con 0x7f10c40049d0 2020-07-18T03:29:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10de2c3700 1 -- 172.21.15.47:0/451654549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10c40049d0 msgr2=0x7f10c4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10de2c3700 1 --2- 172.21.15.47:0/451654549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10c40049d0 0x7f10c4004e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f10d0001760 tx=0x7f10d0007ad0).stop 2020-07-18T03:29:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10dc860700 1 -- 172.21.15.47:0/451654549 reap_dead start 2020-07-18T03:29:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10de2c3700 1 -- 172.21.15.47:0/451654549 shutdown_connections 2020-07-18T03:29:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10de2c3700 1 -- 172.21.15.47:0/451654549 >> 172.21.15.47:0/451654549 conn(0x7f10d805f5a0 msgr2=0x7f10d805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10de2c3700 1 -- 172.21.15.47:0/451654549 shutdown_connections 2020-07-18T03:29:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.687+0000 7f10de2c3700 1 -- 172.21.15.47:0/451654549 wait complete. 2020-07-18T03:29:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.688+0000 7f10de2c3700 1 Processor -- start 2020-07-18T03:29:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.688+0000 7f10de2c3700 1 -- start start 2020-07-18T03:29:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10de2c3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10d8156780 0x7f10d8143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10de2c3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10d8063290 0x7f10d8143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10de2c3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d8064990 con 0x7f10d8138e70 2020-07-18T03:29:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10de2c3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d8064780 con 0x7f10d8063290 2020-07-18T03:29:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10de2c3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d8064170 con 0x7f10d8156780 2020-07-18T03:29:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10d7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10d8156780 0x7f10d8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10d7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10d8156780 0x7f10d8143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60856/0 (socket says 172.21.15.47:60856) 2020-07-18T03:29:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10d7fff700 1 -- 172.21.15.47:0/3712484191 learned_addr learned my addr 172.21.15.47:0/3712484191 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10d77fe700 1 --2- 172.21.15.47:0/3712484191 >> v2:172.21.15.18:3300/0 conn(0x7f10d8063290 0x7f10d8143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.689+0000 7f10d7fff700 1 -- 172.21.15.47:0/3712484191 >> v2:172.21.15.18:3300/0 conn(0x7f10d8063290 msgr2=0x7f10d8143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10d7fff700 1 --2- 172.21.15.47:0/3712484191 >> v2:172.21.15.18:3300/0 conn(0x7f10d8063290 0x7f10d8143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10d7fff700 1 -- 172.21.15.47:0/3712484191 >> v1:172.21.15.18:6789/0 conn(0x7f10d8138e70 legacy=0x7f10d8156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10d7fff700 1 -- 172.21.15.47:0/3712484191 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8157270 con 0x7f10d8156780 2020-07-18T03:29:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10d77fe700 1 --2- 172.21.15.47:0/3712484191 >> v2:172.21.15.18:3300/0 conn(0x7f10d8063290 0x7f10d8143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10d7fff700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d8156780 0x7f10d8143640 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f10d80b2d90 tx=0x7f10c8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c800fa90 con 0x7f10d8156780 2020-07-18T03:29:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c800fbf0 con 0x7f10d8156780 2020-07-18T03:29:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d8157400 con 0x7f10d8156780 2020-07-18T03:29:17.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c800fd60 con 0x7f10d8156780 2020-07-18T03:29:17.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.690+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d80b1fb0 con 0x7f10d8156780 2020-07-18T03:29:17.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.691+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f10c8014440 con 0x7f10d8156780 2020-07-18T03:29:17.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.691+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d80642b0 con 0x7f10d8156780 2020-07-18T03:29:17.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.691+0000 7f10b7fff700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10b004fed0 0x7f10b0052390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:17.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.691+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f10c8068220 con 0x7f10d8156780 2020-07-18T03:29:17.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.691+0000 7f10d77fe700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10b004fed0 0x7f10b0052390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:17.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.692+0000 7f10d77fe700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10b004fed0 0x7f10b0052390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f10cc008000 tx=0x7f10cc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:17.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.695+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f10d80642b0 con 0x7f10d8156780 2020-07-18T03:29:17.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.911+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f10d80642b0 con 0x7f10d8156780 2020-07-18T03:29:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.951+0000 7f10b7fff700 1 -- 172.21.15.47:0/3712484191 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f10d80642b0 con 0x7f10d8156780 2020-07-18T03:29:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.953+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10b004fed0 msgr2=0x7f10b0052390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.953+0000 7f10de2c3700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10b004fed0 0x7f10b0052390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f10cc008000 tx=0x7f10cc008040).stop 2020-07-18T03:29:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.953+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d8156780 msgr2=0x7f10d8143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.953+0000 7f10de2c3700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d8156780 0x7f10d8143640 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f10d80b2d90 tx=0x7f10c8008040).stop 2020-07-18T03:29:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 shutdown_connections 2020-07-18T03:29:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10b004fed0 0x7f10b0052390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 --2- 172.21.15.47:0/3712484191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d8156780 0x7f10d8143640 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 --2- 172.21.15.47:0/3712484191 >> v2:172.21.15.18:3300/0 conn(0x7f10d8063290 0x7f10d8143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 >> 172.21.15.47:0/3712484191 conn(0x7f10d805f5a0 msgr2=0x7f10d80b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 shutdown_connections 2020-07-18T03:29:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:17.954+0000 7f10de2c3700 1 -- 172.21.15.47:0/3712484191 wait complete. 2020-07-18T03:29:17.966 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:18.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.080+0000 7ff62d74e700 1 Processor -- start 2020-07-18T03:29:18.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.080+0000 7ff62d74e700 1 -- start start 2020-07-18T03:29:18.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.081+0000 7ff62d74e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff6281428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.081+0000 7ff62d74e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280b3ac0 con 0x7ff6280b41c0 2020-07-18T03:29:18.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.081+0000 7ff62d74e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280b3d20 con 0x7ff6280b5370 2020-07-18T03:29:18.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.081+0000 7ff62d74e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b5a10 con 0x7ff6280ad9e0 2020-07-18T03:29:18.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.081+0000 7ff626ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff6280b5370 0x7ff62805fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41950/0 (socket says 172.21.15.47:41950) 2020-07-18T03:29:18.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.081+0000 7ff626ffd700 1 -- 172.21.15.47:0/1907413204 learned_addr learned my addr 172.21.15.47:0/1907413204 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:18.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff6277fe700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff6281428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760417082 0 0) 0x7ff6280b3d20 con 0x7ff6280b5370 2020-07-18T03:29:18.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff608003240 con 0x7ff6280b5370 2020-07-18T03:29:18.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537354129 0 0) 0x7ff6280b3ac0 con 0x7ff6280b41c0 2020-07-18T03:29:18.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6280b3d20 con 0x7ff6280b41c0 2020-07-18T03:29:18.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2728273505 0 0) 0x7ff608003240 con 0x7ff6280b5370 2020-07-18T03:29:18.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 msgr2=0x7ff6281428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:18.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff6281428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 >> v1:172.21.15.18:6789/0 conn(0x7ff6280b41c0 legacy=0x7ff6280ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280b5750 con 0x7ff6280b5370 2020-07-18T03:29:18.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7ff620001a50 con 0x7ff6280b5370 2020-07-18T03:29:18.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 >> v1:172.21.15.47:6789/0 conn(0x7ff6280b5370 legacy=0x7ff62805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6080038f0 0x7ff608003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6080044e0 0x7ff608004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff608004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b5a10 con 0x7ff6080044e0 2020-07-18T03:29:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608005370 con 0x7ff6080038f0 2020-07-18T03:29:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60800b780 con 0x7ff6280ad9e0 2020-07-18T03:29:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff626ffd700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6080038f0 0x7ff608003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff6277fe700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff608004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff6267fc700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6080044e0 0x7ff608004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff626ffd700 1 -- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 msgr2=0x7ff608004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff626ffd700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff608004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff626ffd700 1 -- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6080044e0 msgr2=0x7ff608004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff626ffd700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6080044e0 0x7ff608004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff626ffd700 1 -- 172.21.15.47:0/1907413204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608003240 con 0x7ff6080038f0 2020-07-18T03:29:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff6277fe700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff6280ad9e0 0x7ff608004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.082+0000 7ff6267fc700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6080044e0 0x7ff608004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff626ffd700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6080038f0 0x7ff608003cf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7ff620001820 tx=0x7ff6280ba090).ready entity=mon.1 client_cookie=2dfbafb195a95046 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff620005b80 con 0x7ff6080038f0 2020-07-18T03:29:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620005ce0 con 0x7ff6080038f0 2020-07-18T03:29:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff625ffb700 1 -- 172.21.15.47:0/1907413204 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6200072c0 con 0x7ff6080038f0 2020-07-18T03:29:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff62d74e700 1 -- 172.21.15.47:0/1907413204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6080038f0 msgr2=0x7ff608003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff62d74e700 1 --2- 172.21.15.47:0/1907413204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6080038f0 0x7ff608003cf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7ff620001820 tx=0x7ff6280ba090).stop 2020-07-18T03:29:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff6277fe700 1 -- 172.21.15.47:0/1907413204 reap_dead start 2020-07-18T03:29:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff62d74e700 1 -- 172.21.15.47:0/1907413204 shutdown_connections 2020-07-18T03:29:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff62d74e700 1 -- 172.21.15.47:0/1907413204 >> 172.21.15.47:0/1907413204 conn(0x7ff6280b9340 msgr2=0x7ff6280b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:18.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff62d74e700 1 -- 172.21.15.47:0/1907413204 shutdown_connections 2020-07-18T03:29:18.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.083+0000 7ff62d74e700 1 -- 172.21.15.47:0/1907413204 wait complete. 2020-07-18T03:29:18.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.084+0000 7ff62d74e700 1 Processor -- start 2020-07-18T03:29:18.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.084+0000 7ff62d74e700 1 -- start start 2020-07-18T03:29:18.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff62d74e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff6281460d0 0x7ff628063f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff62d74e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff62813fc50 0x7ff628064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff62d74e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6281571d0 con 0x7ff628145cf0 2020-07-18T03:29:18.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff62d74e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6280637d0 con 0x7ff6281460d0 2020-07-18T03:29:18.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff62d74e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff628063a70 con 0x7ff62813fc50 2020-07-18T03:29:18.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff6277fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff62813fc50 0x7ff628064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff6277fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff62813fc50 0x7ff628064400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60878/0 (socket says 172.21.15.47:60878) 2020-07-18T03:29:18.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff6277fe700 1 -- 172.21.15.47:0/4271358153 learned_addr learned my addr 172.21.15.47:0/4271358153 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:18.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.085+0000 7ff6267fc700 1 --2- 172.21.15.47:0/4271358153 >> v2:172.21.15.18:3300/0 conn(0x7ff6281460d0 0x7ff628063f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff6277fe700 1 -- 172.21.15.47:0/4271358153 >> v2:172.21.15.18:3300/0 conn(0x7ff6281460d0 msgr2=0x7ff628063f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff6277fe700 1 --2- 172.21.15.47:0/4271358153 >> v2:172.21.15.18:3300/0 conn(0x7ff6281460d0 0x7ff628063f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff6277fe700 1 -- 172.21.15.47:0/4271358153 >> v1:172.21.15.18:6789/0 conn(0x7ff628145cf0 legacy=0x7ff628135860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff6277fe700 1 -- 172.21.15.47:0/4271358153 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628140430 con 0x7ff62813fc50 2020-07-18T03:29:18.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff6277fe700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62813fc50 0x7ff628064400 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff628112a80 tx=0x7ff618008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:18.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff61800e930 con 0x7ff62813fc50 2020-07-18T03:29:18.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61800ea90 con 0x7ff62813fc50 2020-07-18T03:29:18.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff628135f70 con 0x7ff62813fc50 2020-07-18T03:29:18.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff61800edf0 con 0x7ff62813fc50 2020-07-18T03:29:18.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.086+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff628156fa0 con 0x7ff62813fc50 2020-07-18T03:29:18.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.087+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff628057450 con 0x7ff62813fc50 2020-07-18T03:29:18.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.087+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff618014470 con 0x7ff62813fc50 2020-07-18T03:29:18.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.087+0000 7ff61effd700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff604050000 0x7ff6040524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:18.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.087+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff6180683d0 con 0x7ff62813fc50 2020-07-18T03:29:18.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.087+0000 7ff626ffd700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff604050000 0x7ff6040524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:18.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.088+0000 7ff626ffd700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff604050000 0x7ff6040524c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7ff620001820 tx=0x7ff620015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:18.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.091+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff618040b00 con 0x7ff62813fc50 2020-07-18T03:29:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.300+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6280b01a0 con 0x7ff62813fc50 2020-07-18T03:29:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.301+0000 7ff61effd700 1 -- 172.21.15.47:0/4271358153 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7ff618044120 con 0x7ff62813fc50 2020-07-18T03:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff604050000 msgr2=0x7ff6040524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff604050000 0x7ff6040524c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7ff620001820 tx=0x7ff620015000).stop 2020-07-18T03:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62813fc50 msgr2=0x7ff628064400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62813fc50 0x7ff628064400 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff628112a80 tx=0x7ff618008040).stop 2020-07-18T03:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 shutdown_connections 2020-07-18T03:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff604050000 0x7ff6040524c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 --2- 172.21.15.47:0/4271358153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff62813fc50 0x7ff628064400 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 --2- 172.21.15.47:0/4271358153 >> v2:172.21.15.18:3300/0 conn(0x7ff6281460d0 0x7ff628063f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 >> 172.21.15.47:0/4271358153 conn(0x7ff6280b9340 msgr2=0x7ff6280b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 shutdown_connections 2020-07-18T03:29:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.302+0000 7ff62d74e700 1 -- 172.21.15.47:0/4271358153 wait complete. 2020-07-18T03:29:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:18.315 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:18.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.429+0000 7f8857942700 1 Processor -- start 2020-07-18T03:29:19.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.429+0000 7f8857942700 1 -- start start 2020-07-18T03:29:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8857942700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88500b5370 0x7f885005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8857942700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f88500ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8857942700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88500b3ac0 con 0x7f88500ad7c0 2020-07-18T03:29:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8857942700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88500b5a10 con 0x7f88500b41c0 2020-07-18T03:29:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8857942700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88500b0040 con 0x7f88500b5370 2020-07-18T03:29:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f88556de700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88500b5370 0x7f885005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8855edf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f88500ad7c0 0x7f88501428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41974/0 (socket says 172.21.15.47:41974) 2020-07-18T03:29:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f88556de700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88500b5370 0x7f885005fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60882/0 (socket says 172.21.15.47:60882) 2020-07-18T03:29:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.430+0000 7f8855edf700 1 -- 172.21.15.47:0/2228015687 learned_addr learned my addr 172.21.15.47:0/2228015687 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8854edd700 1 --2- 172.21.15.47:0/2228015687 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f88500ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f88556de700 1 -- 172.21.15.47:0/2228015687 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 msgr2=0x7f88500ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f88556de700 1 --2- 172.21.15.47:0/2228015687 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f88500ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f88556de700 1 -- 172.21.15.47:0/2228015687 >> v1:172.21.15.47:6789/0 conn(0x7f88500ad7c0 legacy=0x7f88501428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f883ffff700 1 -- 172.21.15.47:0/2228015687 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004413550 0 0) 0x7f88500b3ac0 con 0x7f88500ad7c0 2020-07-18T03:29:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f88556de700 1 -- 172.21.15.47:0/2228015687 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500b3d20 con 0x7f88500b5370 2020-07-18T03:29:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8854edd700 1 --2- 172.21.15.47:0/2228015687 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f88500ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f88556de700 1 --2- 172.21.15.47:0/2228015687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88500b5370 0x7f885005fff0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f88500afb30 tx=0x7f8840006dd0).ready entity=mon.1 client_cookie=bcbbfe2c416b0d26 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f883ffff700 1 -- 172.21.15.47:0/2228015687 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88400097b0 con 0x7f88500b5370 2020-07-18T03:29:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f883ffff700 1 -- 172.21.15.47:0/2228015687 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8840009910 con 0x7f88500b5370 2020-07-18T03:29:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f883ffff700 1 -- 172.21.15.47:0/2228015687 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8840009c70 con 0x7f88500b5370 2020-07-18T03:29:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 -- 172.21.15.47:0/2228015687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88500b5370 msgr2=0x7f885005fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 --2- 172.21.15.47:0/2228015687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88500b5370 0x7f885005fff0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f88500afb30 tx=0x7f8840006dd0).stop 2020-07-18T03:29:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 -- 172.21.15.47:0/2228015687 shutdown_connections 2020-07-18T03:29:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 --2- 172.21.15.47:0/2228015687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88500b5370 0x7f885005fff0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 --2- 172.21.15.47:0/2228015687 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f88500ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 -- 172.21.15.47:0/2228015687 >> 172.21.15.47:0/2228015687 conn(0x7f88500b9340 msgr2=0x7f88500b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 -- 172.21.15.47:0/2228015687 shutdown_connections 2020-07-18T03:29:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.431+0000 7f8857942700 1 -- 172.21.15.47:0/2228015687 wait complete. 2020-07-18T03:29:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.433+0000 7f8857942700 1 Processor -- start 2020-07-18T03:29:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.433+0000 7f8857942700 1 -- start start 2020-07-18T03:29:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.433+0000 7f8857942700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f885013aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.433+0000 7f8857942700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88500660d0 con 0x7f88500ad7c0 2020-07-18T03:29:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.433+0000 7f8857942700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8850066370 con 0x7f8850064fd0 2020-07-18T03:29:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.433+0000 7f8857942700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8850066610 con 0x7f88500b41c0 2020-07-18T03:29:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f8855edf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8850064fd0 0x7f885013fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41978/0 (socket says 172.21.15.47:41978) 2020-07-18T03:29:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f8855edf700 1 -- 172.21.15.47:0/1641552263 learned_addr learned my addr 172.21.15.47:0/1641552263 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f8854edd700 1 --2- 172.21.15.47:0/1641552263 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f885013aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503411745 0 0) 0x7f8850066370 con 0x7f8850064fd0 2020-07-18T03:29:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8828003240 con 0x7f8850064fd0 2020-07-18T03:29:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1519192981 0 0) 0x7f8828003240 con 0x7f8850064fd0 2020-07-18T03:29:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8850066370 con 0x7f8850064fd0 2020-07-18T03:29:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8848001be0 con 0x7f8850064fd0 2020-07-18T03:29:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 msgr2=0x7f885013aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 --2- 172.21.15.47:0/1641552263 >> v2:172.21.15.18:3300/0 conn(0x7f88500b41c0 0x7f885013aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 >> v1:172.21.15.47:6789/0 conn(0x7f8850064fd0 legacy=0x7f885013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 >> v1:172.21.15.18:6789/0 conn(0x7f88500ad7c0 legacy=0x7f885014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8828003800 0x7f8828003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88280044e0 0x7f8828004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8828004e10 0x7f882800b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8850066610 con 0x7f8828003800 2020-07-18T03:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882800bc20 con 0x7f88280044e0 2020-07-18T03:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882800bd60 con 0x7f8828004e10 2020-07-18T03:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f8854edd700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88280044e0 0x7f8828004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f88556de700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8828003800 0x7f8828003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.434+0000 7f8855edf700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8828004e10 0x7f882800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8854edd700 1 -- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8828004e10 msgr2=0x7f882800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8854edd700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8828004e10 0x7f882800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8854edd700 1 -- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8828003800 msgr2=0x7f8828003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8854edd700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8828003800 0x7f8828003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8855edf700 1 -- 172.21.15.47:0/1641552263 reap_dead start 2020-07-18T03:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8854edd700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850065df0 con 0x7f88280044e0 2020-07-18T03:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88501426c0 con 0x7f88280044e0 2020-07-18T03:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885014f390 con 0x7f88280044e0 2020-07-18T03:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f8854edd700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88280044e0 0x7f8828004920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f884c00c000 tx=0x7f884c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884c00eb00 con 0x7f88280044e0 2020-07-18T03:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00ec60 con 0x7f88280044e0 2020-07-18T03:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.435+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884c014470 con 0x7f88280044e0 2020-07-18T03:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.436+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8850057450 con 0x7f88280044e0 2020-07-18T03:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.436+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f884c0145d0 con 0x7f88280044e0 2020-07-18T03:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.436+0000 7f883d7fa700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8828053ed0 0x7f8828056390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.436+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f884c0688b0 con 0x7f88280044e0 2020-07-18T03:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.436+0000 7f88556de700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8828053ed0 0x7f8828056390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.437+0000 7f88556de700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8828053ed0 0x7f8828056390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f884001c000 tx=0x7f8840015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.440+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f884c040fa0 con 0x7f88280044e0 2020-07-18T03:29:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.597+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88500b01a0 con 0x7f88280044e0 2020-07-18T03:29:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.598+0000 7f883d7fa700 1 -- 172.21.15.47:0/1641552263 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f884c0445c0 con 0x7f88280044e0 2020-07-18T03:29:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8828053ed0 msgr2=0x7f8828056390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8828053ed0 0x7f8828056390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f884001c000 tx=0x7f8840015000).stop 2020-07-18T03:29:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88280044e0 msgr2=0x7f8828004920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88280044e0 0x7f8828004920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f884c00c000 tx=0x7f884c008040).stop 2020-07-18T03:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 shutdown_connections 2020-07-18T03:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8828053ed0 0x7f8828056390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 --2- 172.21.15.47:0/1641552263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88280044e0 0x7f8828004920 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 >> 172.21.15.47:0/1641552263 conn(0x7f88500b9340 msgr2=0x7f88500b9830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 shutdown_connections 2020-07-18T03:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:18.599+0000 7f8857942700 1 -- 172.21.15.47:0/1641552263 wait complete. 2020-07-18T03:29:19.082 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:19.082 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:17.951029+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:14.526774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:15.768057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:19.083 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:29:19.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.209+0000 7ff9db330700 1 Processor -- start 2020-07-18T03:29:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.209+0000 7ff9db330700 1 -- start start 2020-07-18T03:29:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9db330700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff9d4080510 0x7ff9d40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9db330700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9d4153ad0 0x7ff9d41575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9db330700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9d4153f80 0x7ff9d4157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9db330700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4066c20 con 0x7ff9d4080510 2020-07-18T03:29:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9db330700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4080bb0 con 0x7ff9d4153ad0 2020-07-18T03:29:19.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9db330700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d4157140 con 0x7ff9d4153f80 2020-07-18T03:29:19.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9d90cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9d4153ad0 0x7ff9d41575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9d90cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9d4153ad0 0x7ff9d41575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60902/0 (socket says 172.21.15.47:60902) 2020-07-18T03:29:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.210+0000 7ff9d90cc700 1 -- 172.21.15.47:0/2294547848 learned_addr learned my addr 172.21.15.47:0/2294547848 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9da0ce700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9d4153f80 0x7ff9d4157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d98cd700 1 --2- 172.21.15.47:0/2294547848 >> v2:172.21.15.18:3300/0 conn(0x7ff9d4080510 0x7ff9d40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d90cc700 1 -- 172.21.15.47:0/2294547848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9d4153f80 msgr2=0x7ff9d4157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d90cc700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9d4153f80 0x7ff9d4157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d90cc700 1 -- 172.21.15.47:0/2294547848 >> v2:172.21.15.18:3300/0 conn(0x7ff9d4080510 msgr2=0x7ff9d40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d90cc700 1 --2- 172.21.15.47:0/2294547848 >> v2:172.21.15.18:3300/0 conn(0x7ff9d4080510 0x7ff9d40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d90cc700 1 -- 172.21.15.47:0/2294547848 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40808f0 con 0x7ff9d4153ad0 2020-07-18T03:29:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9da0ce700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9d4153f80 0x7ff9d4157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d98cd700 1 --2- 172.21.15.47:0/2294547848 >> v2:172.21.15.18:3300/0 conn(0x7ff9d4080510 0x7ff9d40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d90cc700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153ad0 0x7ff9d41575e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff9c4002f30 tx=0x7ff9c4006e90).ready entity=mon.1 client_cookie=c4ce111b56551d2a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d88cb700 1 -- 172.21.15.47:0/2294547848 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9c4009920 con 0x7ff9d4153ad0 2020-07-18T03:29:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d88cb700 1 -- 172.21.15.47:0/2294547848 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4009a80 con 0x7ff9d4153ad0 2020-07-18T03:29:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9d88cb700 1 -- 172.21.15.47:0/2294547848 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9c4009de0 con 0x7ff9d4153ad0 2020-07-18T03:29:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 -- 172.21.15.47:0/2294547848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153ad0 msgr2=0x7ff9d41575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153ad0 0x7ff9d41575e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff9c4002f30 tx=0x7ff9c4006e90).stop 2020-07-18T03:29:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 -- 172.21.15.47:0/2294547848 shutdown_connections 2020-07-18T03:29:19.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff9d4153f80 0x7ff9d4157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 --2- 172.21.15.47:0/2294547848 >> v2:172.21.15.18:3300/0 conn(0x7ff9d4080510 0x7ff9d40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 --2- 172.21.15.47:0/2294547848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153ad0 0x7ff9d41575e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 -- 172.21.15.47:0/2294547848 >> 172.21.15.47:0/2294547848 conn(0x7ff9d405c620 msgr2=0x7ff9d40b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 -- 172.21.15.47:0/2294547848 shutdown_connections 2020-07-18T03:29:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.211+0000 7ff9db330700 1 -- 172.21.15.47:0/2294547848 wait complete. 2020-07-18T03:29:19.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.213+0000 7ff9db330700 1 Processor -- start 2020-07-18T03:29:19.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.213+0000 7ff9db330700 1 -- start start 2020-07-18T03:29:19.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9d4153f80 0x7ff9d4137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff9d412b400 0x7ff9d41384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d412afe0 con 0x7ff9d4080510 2020-07-18T03:29:19.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4129790 con 0x7ff9d412b400 2020-07-18T03:29:19.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4152b80 con 0x7ff9d4153f80 2020-07-18T03:29:19.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9d4153f80 0x7ff9d4137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9d4153f80 0x7ff9d4137fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60908/0 (socket says 172.21.15.47:60908) 2020-07-18T03:29:19.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 -- 172.21.15.47:0/3500949504 learned_addr learned my addr 172.21.15.47:0/3500949504 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:19.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9da0ce700 1 --2- 172.21.15.47:0/3500949504 >> v2:172.21.15.18:3300/0 conn(0x7ff9d412b400 0x7ff9d41384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 -- 172.21.15.47:0/3500949504 >> v2:172.21.15.18:3300/0 conn(0x7ff9d412b400 msgr2=0x7ff9d41384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 --2- 172.21.15.47:0/3500949504 >> v2:172.21.15.18:3300/0 conn(0x7ff9d412b400 0x7ff9d41384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 -- 172.21.15.47:0/3500949504 >> v1:172.21.15.18:6789/0 conn(0x7ff9d4080510 legacy=0x7ff9d4155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 -- 172.21.15.47:0/3500949504 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d4152e50 con 0x7ff9d4153f80 2020-07-18T03:29:19.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9d90cc700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153f80 0x7ff9d4137fd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7ff9c4002f30 tx=0x7ff9c4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9c401d5d0 con 0x7ff9d4153f80 2020-07-18T03:29:19.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c401d730 con 0x7ff9d4153f80 2020-07-18T03:29:19.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d412a940 con 0x7ff9d4153f80 2020-07-18T03:29:19.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9c401da90 con 0x7ff9d4153f80 2020-07-18T03:29:19.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.214+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d4138b70 con 0x7ff9d4153f80 2020-07-18T03:29:19.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.215+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d4137cc0 con 0x7ff9d4153f80 2020-07-18T03:29:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.216+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff9c4025030 con 0x7ff9d4153f80 2020-07-18T03:29:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.216+0000 7ff9c97fa700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff9ac04ff40 0x7ff9ac052400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.216+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff9c40694d0 con 0x7ff9d4153f80 2020-07-18T03:29:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.216+0000 7ff9d98cd700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff9ac04ff40 0x7ff9ac052400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.216+0000 7ff9d98cd700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff9ac04ff40 0x7ff9ac052400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7ff9d41535c0 tx=0x7ff9d0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.219+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff9c4041c60 con 0x7ff9d4153f80 2020-07-18T03:29:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.433+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff9d4152cc0 con 0x7ff9d4153f80 2020-07-18T03:29:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.437+0000 7ff9c97fa700 1 -- 172.21.15.47:0/3500949504 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff9c4045280 con 0x7ff9d4153f80 2020-07-18T03:29:19.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff9ac04ff40 msgr2=0x7ff9ac052400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff9ac04ff40 0x7ff9ac052400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7ff9d41535c0 tx=0x7ff9d0008040).stop 2020-07-18T03:29:19.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153f80 msgr2=0x7ff9d4137fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153f80 0x7ff9d4137fd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7ff9c4002f30 tx=0x7ff9c4015000).stop 2020-07-18T03:29:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 shutdown_connections 2020-07-18T03:29:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff9ac04ff40 0x7ff9ac052400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 --2- 172.21.15.47:0/3500949504 >> v2:172.21.15.18:3300/0 conn(0x7ff9d412b400 0x7ff9d41384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 --2- 172.21.15.47:0/3500949504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9d4153f80 0x7ff9d4137fd0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 >> 172.21.15.47:0/3500949504 conn(0x7ff9d405c620 msgr2=0x7ff9d4062440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 shutdown_connections 2020-07-18T03:29:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.438+0000 7ff9db330700 1 -- 172.21.15.47:0/3500949504 wait complete. 2020-07-18T03:29:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2020-07-18T03:29:19.451 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:29:19.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.570+0000 7fde4a9ba700 1 Processor -- start 2020-07-18T03:29:19.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.570+0000 7fde4a9ba700 1 -- start start 2020-07-18T03:29:19.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde4a9ba700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44153ad0 0x7fde441575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde4a9ba700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44153f80 0x7fde44157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde4a9ba700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde44066c20 con 0x7fde44080510 2020-07-18T03:29:19.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde4a9ba700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde44080bb0 con 0x7fde44153f80 2020-07-18T03:29:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde4a9ba700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde44157140 con 0x7fde44153ad0 2020-07-18T03:29:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde49758700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44153f80 0x7fde44157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde49758700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44153f80 0x7fde44157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60918/0 (socket says 172.21.15.47:60918) 2020-07-18T03:29:19.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde49758700 1 -- 172.21.15.47:0/654637761 learned_addr learned my addr 172.21.15.47:0/654637761 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:19.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde43fff700 1 --2- 172.21.15.47:0/654637761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44153ad0 0x7fde441575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.571+0000 7fde49758700 1 -- 172.21.15.47:0/654637761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44153ad0 msgr2=0x7fde441575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde49758700 1 --2- 172.21.15.47:0/654637761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44153ad0 0x7fde441575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde49758700 1 -- 172.21.15.47:0/654637761 >> v1:172.21.15.18:6789/0 conn(0x7fde44080510 legacy=0x7fde440b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde49758700 1 -- 172.21.15.47:0/654637761 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440808f0 con 0x7fde44153f80 2020-07-18T03:29:19.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde49758700 1 --2- 172.21.15.47:0/654637761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153f80 0x7fde44157ad0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fde3c0032b0 tx=0x7fde3c009270).ready entity=mon.1 client_cookie=aa4b1a782843ca8e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde437fe700 1 -- 172.21.15.47:0/654637761 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3c009e60 con 0x7fde44153f80 2020-07-18T03:29:19.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde437fe700 1 -- 172.21.15.47:0/654637761 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c00bc00 con 0x7fde44153f80 2020-07-18T03:29:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde437fe700 1 -- 172.21.15.47:0/654637761 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3c00bd70 con 0x7fde44153f80 2020-07-18T03:29:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 -- 172.21.15.47:0/654637761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153f80 msgr2=0x7fde44157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/654637761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153f80 0x7fde44157ad0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fde3c0032b0 tx=0x7fde3c009270).stop 2020-07-18T03:29:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 -- 172.21.15.47:0/654637761 shutdown_connections 2020-07-18T03:29:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/654637761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153f80 0x7fde44157ad0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/654637761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44153ad0 0x7fde441575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 -- 172.21.15.47:0/654637761 >> 172.21.15.47:0/654637761 conn(0x7fde4405c620 msgr2=0x7fde440b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 -- 172.21.15.47:0/654637761 shutdown_connections 2020-07-18T03:29:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.572+0000 7fde4a9ba700 1 -- 172.21.15.47:0/654637761 wait complete. 2020-07-18T03:29:19.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 Processor -- start 2020-07-18T03:29:19.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 -- start start 2020-07-18T03:29:19.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44080510 0x7fde44129ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44153ad0 0x7fde44129fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4415c930 con 0x7fde4415a640 2020-07-18T03:29:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde441583a0 con 0x7fde44153ad0 2020-07-18T03:29:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.574+0000 7fde4a9ba700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415cbd0 con 0x7fde44080510 2020-07-18T03:29:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44153ad0 0x7fde44129fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44153ad0 0x7fde44129fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60922/0 (socket says 172.21.15.47:60922) 2020-07-18T03:29:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 -- 172.21.15.47:0/3791139581 learned_addr learned my addr 172.21.15.47:0/3791139581 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde48f57700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44080510 0x7fde44129ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 -- 172.21.15.47:0/3791139581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44080510 msgr2=0x7fde44129ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44080510 0x7fde44129ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 -- 172.21.15.47:0/3791139581 >> v1:172.21.15.18:6789/0 conn(0x7fde4415a640 legacy=0x7fde44136b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 -- 172.21.15.47:0/3791139581 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4415ce70 con 0x7fde44153ad0 2020-07-18T03:29:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035451169 0 0) 0x7fde4415c930 con 0x7fde4415a640 2020-07-18T03:29:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde43fff700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153ad0 0x7fde44129fa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fde3800c000 tx=0x7fde38008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3800ca10 con 0x7fde44153ad0 2020-07-18T03:29:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3800cb70 con 0x7fde44153ad0 2020-07-18T03:29:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde380123f0 con 0x7fde44153ad0 2020-07-18T03:29:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44137220 con 0x7fde44153ad0 2020-07-18T03:29:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.575+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440b5800 con 0x7fde44153ad0 2020-07-18T03:29:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.576+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4415c770 con 0x7fde44153ad0 2020-07-18T03:29:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.576+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fde380125f0 con 0x7fde44153ad0 2020-07-18T03:29:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.577+0000 7fde23fff700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde1c04fe20 0x7fde1c0522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.577+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fde380682f0 con 0x7fde44153ad0 2020-07-18T03:29:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.577+0000 7fde48f57700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde1c04fe20 0x7fde1c0522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.577+0000 7fde48f57700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde1c04fe20 0x7fde1c0522e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fde3400d000 tx=0x7fde34007b70).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.580+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fde38040900 con 0x7fde44153ad0 2020-07-18T03:29:19.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:29:19.751 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:29:19.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.796+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fde44061d20 con 0x7fde44153ad0 2020-07-18T03:29:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.800+0000 7fde23fff700 1 -- 172.21.15.47:0/3791139581 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7fde38016020 con 0x7fde44153ad0 2020-07-18T03:29:19.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde1c04fe20 msgr2=0x7fde1c0522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde1c04fe20 0x7fde1c0522e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fde3400d000 tx=0x7fde34007b70).stop 2020-07-18T03:29:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153ad0 msgr2=0x7fde44129fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153ad0 0x7fde44129fa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fde3800c000 tx=0x7fde38008040).stop 2020-07-18T03:29:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 shutdown_connections 2020-07-18T03:29:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde1c04fe20 0x7fde1c0522e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fde44080510 0x7fde44129ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 --2- 172.21.15.47:0/3791139581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44153ad0 0x7fde44129fa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 >> 172.21.15.47:0/3791139581 conn(0x7fde4405c620 msgr2=0x7fde4412a900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 shutdown_connections 2020-07-18T03:29:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.801+0000 7fde4a9ba700 1 -- 172.21.15.47:0/3791139581 wait complete. 2020-07-18T03:29:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2020-07-18T03:29:19.814 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:29:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.972+0000 7f5baf894700 1 Processor -- start 2020-07-18T03:29:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.972+0000 7f5baf894700 1 -- start start 2020-07-18T03:29:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.973+0000 7f5baf894700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5ba8157cf0 0x7f5ba8063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.973+0000 7f5baf894700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba8080bb0 con 0x7f5ba8080510 2020-07-18T03:29:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.973+0000 7f5baf894700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba8157140 con 0x7f5ba8153ad0 2020-07-18T03:29:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.973+0000 7f5baf894700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80808f0 con 0x7f5ba8157cf0 2020-07-18T03:29:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.973+0000 7f5bad630700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5ba8153ad0 0x7f5ba81575e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42016/0 (socket says 172.21.15.47:42016) 2020-07-18T03:29:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.973+0000 7f5bad630700 1 -- 172.21.15.47:0/143582145 learned_addr learned my addr 172.21.15.47:0/143582145 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001877636 0 0) 0x7f5ba8157140 con 0x7f5ba8153ad0 2020-07-18T03:29:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bae632700 1 --2- 172.21.15.47:0/143582145 >> v2:172.21.15.18:3300/0 conn(0x7f5ba8157cf0 0x7f5ba8063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b94003240 con 0x7f5ba8153ad0 2020-07-18T03:29:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2583971744 0 0) 0x7f5b94003240 con 0x7f5ba8153ad0 2020-07-18T03:29:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 >> v2:172.21.15.18:3300/0 conn(0x7f5ba8157cf0 msgr2=0x7f5ba8063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 --2- 172.21.15.47:0/143582145 >> v2:172.21.15.18:3300/0 conn(0x7f5ba8157cf0 0x7f5ba8063450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 >> v1:172.21.15.18:6789/0 conn(0x7f5ba8080510 legacy=0x7f5ba80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8080e50 con 0x7f5ba8153ad0 2020-07-18T03:29:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5b980016d0 con 0x7f5ba8153ad0 2020-07-18T03:29:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 >> v1:172.21.15.47:6789/0 conn(0x7f5ba8153ad0 legacy=0x7f5ba81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b94003780 0x7f5b94003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b940044e0 0x7f5b94004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b94004e50 0x7f5b9400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80808f0 con 0x7f5b94004e50 2020-07-18T03:29:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9404fc30 con 0x7f5b940044e0 2020-07-18T03:29:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9404fd70 con 0x7f5b94003780 2020-07-18T03:29:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b940044e0 0x7f5b94004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bade31700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b94003780 0x7f5b94003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bae632700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b94004e50 0x7f5b9400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 -- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b94003780 msgr2=0x7f5b94003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b94003780 0x7f5b94003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 -- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b94004e50 msgr2=0x7f5b9400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:19.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b94004e50 0x7f5b9400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 -- 172.21.15.47:0/143582145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94003240 con 0x7f5b940044e0 2020-07-18T03:29:19.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bae632700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b94004e50 0x7f5b9400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:19.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bade31700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5b94003780 0x7f5b94003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:19.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bae632700 1 -- 172.21.15.47:0/143582145 reap_dead start 2020-07-18T03:29:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.974+0000 7f5bad630700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b940044e0 0x7f5b94004960 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5b980013e0 tx=0x7f5b98007610).ready entity=mon.1 client_cookie=f755bfffe00d4fd7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9800ce90 con 0x7f5b940044e0 2020-07-18T03:29:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b980070d0 con 0x7f5b940044e0 2020-07-18T03:29:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5bace2f700 1 -- 172.21.15.47:0/143582145 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b98007430 con 0x7f5b940044e0 2020-07-18T03:29:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 -- 172.21.15.47:0/143582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b940044e0 msgr2=0x7f5b94004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b940044e0 0x7f5b94004960 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5b980013e0 tx=0x7f5b98007610).stop 2020-07-18T03:29:19.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 -- 172.21.15.47:0/143582145 shutdown_connections 2020-07-18T03:29:19.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 --2- 172.21.15.47:0/143582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b940044e0 0x7f5b94004960 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 -- 172.21.15.47:0/143582145 >> 172.21.15.47:0/143582145 conn(0x7f5ba805c620 msgr2=0x7f5ba80b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:19.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 -- 172.21.15.47:0/143582145 shutdown_connections 2020-07-18T03:29:19.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.975+0000 7f5baf894700 1 -- 172.21.15.47:0/143582145 wait complete. 2020-07-18T03:29:19.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.976+0000 7f5baf894700 1 Processor -- start 2020-07-18T03:29:19.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.976+0000 7f5baf894700 1 -- start start 2020-07-18T03:29:19.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5baf894700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5ba80afb80 0x7f5ba80b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5baf894700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 0x7f5ba8152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5baf894700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba8141a00 con 0x7f5ba80b22d0 2020-07-18T03:29:19.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5baf894700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80b3600 con 0x7f5ba80afb80 2020-07-18T03:29:19.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5baf894700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80b37e0 con 0x7f5ba80b2760 2020-07-18T03:29:19.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5bad630700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5ba80b22d0 0x7f5ba805f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42030/0 (socket says 172.21.15.47:42030) 2020-07-18T03:29:19.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.977+0000 7f5bad630700 1 -- 172.21.15.47:0/860168237 learned_addr learned my addr 172.21.15.47:0/860168237 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bade31700 1 --2- 172.21.15.47:0/860168237 >> v2:172.21.15.18:3300/0 conn(0x7f5ba80afb80 0x7f5ba80b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912157492 0 0) 0x7f5ba8141a00 con 0x7f5ba80b22d0 2020-07-18T03:29:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bae632700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 0x7f5ba8152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b80003240 con 0x7f5ba80b22d0 2020-07-18T03:29:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1315473549 0 0) 0x7f5b80003240 con 0x7f5ba80b22d0 2020-07-18T03:29:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ba8141a00 con 0x7f5ba80b22d0 2020-07-18T03:29:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5b980016d0 con 0x7f5ba80b22d0 2020-07-18T03:29:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 msgr2=0x7f5ba8152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 0x7f5ba8152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 >> v2:172.21.15.18:3300/0 conn(0x7f5ba80afb80 msgr2=0x7f5ba80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 --2- 172.21.15.47:0/860168237 >> v2:172.21.15.18:3300/0 conn(0x7f5ba80afb80 0x7f5ba80b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 >> v1:172.21.15.47:6789/0 conn(0x7f5ba80b22d0 legacy=0x7f5ba805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b80003800 0x7f5b80003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b800044e0 0x7f5b80004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 0x7f5b80004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:19.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80b3600 con 0x7f5b80003800 2020-07-18T03:29:19.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80b37e0 con 0x7f5b800044e0 2020-07-18T03:29:19.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80005300 con 0x7f5ba80b2760 2020-07-18T03:29:19.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bad630700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b800044e0 0x7f5b80004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bade31700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b80003800 0x7f5b80003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bae632700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 0x7f5b80004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:19.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bad630700 1 -- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 msgr2=0x7f5b80004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bad630700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5ba80b2760 0x7f5b80004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bad630700 1 -- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b80003800 msgr2=0x7f5b80003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:19.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bad630700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b80003800 0x7f5b80003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:19.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5bad630700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba805fcb0 con 0x7f5b800044e0 2020-07-18T03:29:20.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba80aff60 con 0x7f5b800044e0 2020-07-18T03:29:20.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.978+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba80b0390 con 0x7f5b800044e0 2020-07-18T03:29:20.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.979+0000 7f5bad630700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b800044e0 0x7f5b80004920 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f5b98001980 tx=0x7f5b98015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.979+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b98001a90 con 0x7f5b800044e0 2020-07-18T03:29:20.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.979+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b980011d0 con 0x7f5b800044e0 2020-07-18T03:29:20.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.979+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b98013e00 con 0x7f5b800044e0 2020-07-18T03:29:20.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.979+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8141800 con 0x7f5b800044e0 2020-07-18T03:29:20.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.980+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5b98010070 con 0x7f5b800044e0 2020-07-18T03:29:20.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.980+0000 7f5b9d7fa700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5b80057f90 0x7f5b8005a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.980+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b98019030 con 0x7f5b800044e0 2020-07-18T03:29:20.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.980+0000 7f5bade31700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5b80057f90 0x7f5b8005a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.980+0000 7f5bade31700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5b80057f90 0x7f5b8005a450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5ba400d000 tx=0x7f5ba4009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:19.983+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5b98042030 con 0x7f5b800044e0 2020-07-18T03:29:20.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.202+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5ba80b3950 con 0x7f5b800044e0 2020-07-18T03:29:20.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.206+0000 7f5b9d7fa700 1 -- 172.21.15.47:0/860168237 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5b98045610 con 0x7f5b800044e0 2020-07-18T03:29:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5b80057f90 msgr2=0x7f5b8005a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5baf894700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5b80057f90 0x7f5b8005a450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5ba400d000 tx=0x7f5ba4009040).stop 2020-07-18T03:29:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b800044e0 msgr2=0x7f5b80004920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5baf894700 1 --2- 172.21.15.47:0/860168237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b800044e0 0x7f5b80004920 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f5b98001980 tx=0x7f5b98015000).stop 2020-07-18T03:29:20.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5bae632700 1 -- 172.21.15.47:0/860168237 reap_dead start 2020-07-18T03:29:20.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 shutdown_connections 2020-07-18T03:29:20.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.207+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 >> 172.21.15.47:0/860168237 conn(0x7f5ba805c620 msgr2=0x7f5ba8156bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:20.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.208+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 shutdown_connections 2020-07-18T03:29:20.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.208+0000 7f5baf894700 1 -- 172.21.15.47:0/860168237 wait complete. 2020-07-18T03:29:20.213 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.2 2020-07-18T03:29:20.220 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:29:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.339+0000 7fbf7574f700 1 Processor -- start 2020-07-18T03:29:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.339+0000 7fbf7574f700 1 -- start start 2020-07-18T03:29:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf7574f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 0x7fbf701575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf7574f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf70080bb0 con 0x7fbf70153f80 2020-07-18T03:29:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf7574f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf70157140 con 0x7fbf70080510 2020-07-18T03:29:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf7574f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700808f0 con 0x7fbf70153ad0 2020-07-18T03:29:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf6f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbf70080510 0x7fbf700b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42042/0 (socket says 172.21.15.47:42042) 2020-07-18T03:29:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf6f7fe700 1 -- 172.21.15.47:0/1748507084 learned_addr learned my addr 172.21.15.47:0/1748507084 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf6effd700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 0x7fbf701575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805484563 0 0) 0x7fbf70157140 con 0x7fbf70080510 2020-07-18T03:29:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.340+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf58003240 con 0x7fbf70080510 2020-07-18T03:29:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3325612008 0 0) 0x7fbf58003240 con 0x7fbf70080510 2020-07-18T03:29:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 msgr2=0x7fbf701575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 0x7fbf701575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 >> v1:172.21.15.18:6789/0 conn(0x7fbf70153f80 legacy=0x7fbf70063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70080e50 con 0x7fbf70080510 2020-07-18T03:29:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fbf600016d0 con 0x7fbf70080510 2020-07-18T03:29:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 >> v1:172.21.15.47:6789/0 conn(0x7fbf70080510 legacy=0x7fbf700b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 0x7fbf58003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf58003c20 0x7fbf580044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf580049d0 0x7fbf58004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700808f0 con 0x7fbf580049d0 2020-07-18T03:29:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58004140 con 0x7fbf58003c20 2020-07-18T03:29:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58004300 con 0x7fbf70153ad0 2020-07-18T03:29:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf58003c20 0x7fbf580044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6f7fe700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 0x7fbf58003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6ffff700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf580049d0 0x7fbf58004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 -- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 msgr2=0x7fbf58003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf70153ad0 0x7fbf58003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 -- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf580049d0 msgr2=0x7fbf58004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf580049d0 0x7fbf58004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 -- 172.21.15.47:0/1748507084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58003240 con 0x7fbf58003c20 2020-07-18T03:29:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6effd700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf58003c20 0x7fbf580044e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fbf6400c000 tx=0x7fbf64008040).ready entity=mon.1 client_cookie=63993682b13037a1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6400e8a0 con 0x7fbf58003c20 2020-07-18T03:29:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6400ea00 con 0x7fbf58003c20 2020-07-18T03:29:20.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6e7fc700 1 -- 172.21.15.47:0/1748507084 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6400ed60 con 0x7fbf58003c20 2020-07-18T03:29:20.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf7574f700 1 -- 172.21.15.47:0/1748507084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf58003c20 msgr2=0x7fbf580044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf7574f700 1 --2- 172.21.15.47:0/1748507084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf58003c20 0x7fbf580044e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fbf6400c000 tx=0x7fbf64008040).stop 2020-07-18T03:29:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.341+0000 7fbf6ffff700 1 -- 172.21.15.47:0/1748507084 reap_dead start 2020-07-18T03:29:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.342+0000 7fbf7574f700 1 -- 172.21.15.47:0/1748507084 shutdown_connections 2020-07-18T03:29:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.342+0000 7fbf7574f700 1 -- 172.21.15.47:0/1748507084 >> 172.21.15.47:0/1748507084 conn(0x7fbf7005c620 msgr2=0x7fbf700b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.342+0000 7fbf7574f700 1 -- 172.21.15.47:0/1748507084 shutdown_connections 2020-07-18T03:29:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.342+0000 7fbf7574f700 1 -- 172.21.15.47:0/1748507084 wait complete. 2020-07-18T03:29:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.343+0000 7fbf7574f700 1 Processor -- start 2020-07-18T03:29:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.343+0000 7fbf7574f700 1 -- start start 2020-07-18T03:29:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf7574f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf70153610 0x7fbf70152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf7574f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbf7013f250 0x7fbf700ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf7574f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf700af470 con 0x7fbf7005f5a0 2020-07-18T03:29:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf7574f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf700adc90 con 0x7fbf7013f250 2020-07-18T03:29:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf7574f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf700ad950 con 0x7fbf70153610 2020-07-18T03:29:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf70153610 0x7fbf70152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf70153610 0x7fbf70152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60966/0 (socket says 172.21.15.47:60966) 2020-07-18T03:29:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 -- 172.21.15.47:0/2742080588 learned_addr learned my addr 172.21.15.47:0/2742080588 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6effd700 1 --2- 172.21.15.47:0/2742080588 >> v2:172.21.15.18:3300/0 conn(0x7fbf7013f250 0x7fbf700ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 -- 172.21.15.47:0/2742080588 >> v2:172.21.15.18:3300/0 conn(0x7fbf7013f250 msgr2=0x7fbf700ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 --2- 172.21.15.47:0/2742080588 >> v2:172.21.15.18:3300/0 conn(0x7fbf7013f250 0x7fbf700ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 -- 172.21.15.47:0/2742080588 >> v1:172.21.15.18:6789/0 conn(0x7fbf7005f5a0 legacy=0x7fbf700af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6f7fe700 1 -- 172.21.15.47:0/2742080588 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700adf30 con 0x7fbf70153610 2020-07-18T03:29:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.344+0000 7fbf6effd700 1 --2- 172.21.15.47:0/2742080588 >> v2:172.21.15.18:3300/0 conn(0x7fbf7013f250 0x7fbf700ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.345+0000 7fbf6f7fe700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf70153610 0x7fbf70152660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fbf700601a0 tx=0x7fbf60007750).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.345+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6000fa90 con 0x7fbf70153610 2020-07-18T03:29:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.345+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6000fbf0 con 0x7fbf70153610 2020-07-18T03:29:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.345+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7005f9f0 con 0x7fbf70153610 2020-07-18T03:29:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.345+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf60012350 con 0x7fbf70153610 2020-07-18T03:29:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.345+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf700affe0 con 0x7fbf70153610 2020-07-18T03:29:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.346+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf700ada90 con 0x7fbf70153610 2020-07-18T03:29:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.346+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbf6001b030 con 0x7fbf70153610 2020-07-18T03:29:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.346+0000 7fbf56ffd700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4c050000 0x7fbf4c0524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.346+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbf60067760 con 0x7fbf70153610 2020-07-18T03:29:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.346+0000 7fbf6effd700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4c050000 0x7fbf4c0524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.347+0000 7fbf6effd700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4c050000 0x7fbf4c0524c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbf64007000 tx=0x7fbf6401a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.350+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbf6003fef0 con 0x7fbf70153610 2020-07-18T03:29:20.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.567+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbf700af630 con 0x7fbf70153610 2020-07-18T03:29:20.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.572+0000 7fbf56ffd700 1 -- 172.21.15.47:0/2742080588 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbf700af630 con 0x7fbf70153610 2020-07-18T03:29:20.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4c050000 msgr2=0x7fbf4c0524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4c050000 0x7fbf4c0524c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbf64007000 tx=0x7fbf6401a000).stop 2020-07-18T03:29:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf70153610 msgr2=0x7fbf70152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf70153610 0x7fbf70152660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fbf700601a0 tx=0x7fbf60007750).stop 2020-07-18T03:29:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 shutdown_connections 2020-07-18T03:29:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4c050000 0x7fbf4c0524c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 --2- 172.21.15.47:0/2742080588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf70153610 0x7fbf70152660 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 --2- 172.21.15.47:0/2742080588 >> v2:172.21.15.18:3300/0 conn(0x7fbf7013f250 0x7fbf700ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 >> 172.21.15.47:0/2742080588 conn(0x7fbf7005c620 msgr2=0x7fbf700b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 shutdown_connections 2020-07-18T03:29:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.573+0000 7fbf7574f700 1 -- 172.21.15.47:0/2742080588 wait complete. 2020-07-18T03:29:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.3 2020-07-18T03:29:20.585 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:20.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.698+0000 7fd363966700 1 Processor -- start 2020-07-18T03:29:20.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.698+0000 7fd363966700 1 -- start start 2020-07-18T03:29:20.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd363966700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd35c0ad100 0x7fd35c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd363966700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd35c147a00 0x7fd35c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd363966700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c147330 con 0x7fd35c05f5a0 2020-07-18T03:29:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd363966700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35c0ae530 con 0x7fd35c147a00 2020-07-18T03:29:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd363966700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35c0b57e0 con 0x7fd35c0ad100 2020-07-18T03:29:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd35c0ad100 0x7fd35c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd35c0ad100 0x7fd35c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60974/0 (socket says 172.21.15.47:60974) 2020-07-18T03:29:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 -- 172.21.15.47:0/1457855745 learned_addr learned my addr 172.21.15.47:0/1457855745 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd360f01700 1 --2- 172.21.15.47:0/1457855745 >> v2:172.21.15.18:3300/0 conn(0x7fd35c147a00 0x7fd35c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 -- 172.21.15.47:0/1457855745 >> v2:172.21.15.18:3300/0 conn(0x7fd35c147a00 msgr2=0x7fd35c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 --2- 172.21.15.47:0/1457855745 >> v2:172.21.15.18:3300/0 conn(0x7fd35c147a00 0x7fd35c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 -- 172.21.15.47:0/1457855745 >> v1:172.21.15.18:6789/0 conn(0x7fd35c05f5a0 legacy=0x7fd35c05f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 -- 172.21.15.47:0/1457855745 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0b5a80 con 0x7fd35c0ad100 2020-07-18T03:29:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd361702700 1 --2- 172.21.15.47:0/1457855745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd35c0ad100 0x7fd35c0b4c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fd34c002f30 tx=0x7fd34c006e90).ready entity=mon.1 client_cookie=c3e9a1722f295a66 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.699+0000 7fd34bfff700 1 -- 172.21.15.47:0/1457855745 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c0099c0 con 0x7fd35c0ad100 2020-07-18T03:29:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd34bfff700 1 -- 172.21.15.47:0/1457855745 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c009b20 con 0x7fd35c0ad100 2020-07-18T03:29:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd34bfff700 1 -- 172.21.15.47:0/1457855745 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c009e80 con 0x7fd35c0ad100 2020-07-18T03:29:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 -- 172.21.15.47:0/1457855745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd35c0ad100 msgr2=0x7fd35c0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 --2- 172.21.15.47:0/1457855745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd35c0ad100 0x7fd35c0b4c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fd34c002f30 tx=0x7fd34c006e90).stop 2020-07-18T03:29:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 -- 172.21.15.47:0/1457855745 shutdown_connections 2020-07-18T03:29:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 --2- 172.21.15.47:0/1457855745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd35c0ad100 0x7fd35c0b4c10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 --2- 172.21.15.47:0/1457855745 >> v2:172.21.15.18:3300/0 conn(0x7fd35c147a00 0x7fd35c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 -- 172.21.15.47:0/1457855745 >> 172.21.15.47:0/1457855745 conn(0x7fd35c0b9340 msgr2=0x7fd35c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 -- 172.21.15.47:0/1457855745 shutdown_connections 2020-07-18T03:29:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.700+0000 7fd363966700 1 -- 172.21.15.47:0/1457855745 wait complete. 2020-07-18T03:29:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.701+0000 7fd363966700 1 Processor -- start 2020-07-18T03:29:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.701+0000 7fd363966700 1 -- start start 2020-07-18T03:29:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd363966700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd35c05f5a0 0x7fd35c0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd363966700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c136870 con 0x7fd35c147a00 2020-07-18T03:29:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd363966700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c1365d0 con 0x7fd35c0b0410 2020-07-18T03:29:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd363966700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0b2160 con 0x7fd35c05f5a0 2020-07-18T03:29:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd361f03700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd35c0b0410 0x7fd35c135b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42070/0 (socket says 172.21.15.47:42070) 2020-07-18T03:29:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd361f03700 1 -- 172.21.15.47:0/721835167 learned_addr learned my addr 172.21.15.47:0/721835167 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd361702700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd35c05f5a0 0x7fd35c0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255624740 0 0) 0x7fd35c1365d0 con 0x7fd35c0b0410 2020-07-18T03:29:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.702+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd334003240 con 0x7fd35c0b0410 2020-07-18T03:29:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647614145 0 0) 0x7fd35c136870 con 0x7fd35c147a00 2020-07-18T03:29:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c1365d0 con 0x7fd35c147a00 2020-07-18T03:29:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1753563989 0 0) 0x7fd334003240 con 0x7fd35c0b0410 2020-07-18T03:29:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd35c136870 con 0x7fd35c0b0410 2020-07-18T03:29:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fd354001cc0 con 0x7fd35c0b0410 2020-07-18T03:29:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd361702700 1 -- 172.21.15.47:0/721835167 >> v1:172.21.15.47:6789/0 conn(0x7fd35c0b0410 legacy=0x7fd35c135b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd361702700 1 -- 172.21.15.47:0/721835167 >> v1:172.21.15.18:6789/0 conn(0x7fd35c147a00 legacy=0x7fd35c138be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd361702700 1 -- 172.21.15.47:0/721835167 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0b2600 con 0x7fd35c05f5a0 2020-07-18T03:29:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd361702700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd35c05f5a0 0x7fd35c0b2f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd34c002f30 tx=0x7fd34c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c011c90 con 0x7fd35c05f5a0 2020-07-18T03:29:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c011df0 con 0x7fd35c05f5a0 2020-07-18T03:29:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c136280 con 0x7fd35c05f5a0 2020-07-18T03:29:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34c01e650 con 0x7fd35c05f5a0 2020-07-18T03:29:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.703+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c0b0ac0 con 0x7fd35c05f5a0 2020-07-18T03:29:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.704+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c057450 con 0x7fd35c05f5a0 2020-07-18T03:29:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.704+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd34c025030 con 0x7fd35c05f5a0 2020-07-18T03:29:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.704+0000 7fd3497fa700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3340506c0 0x7fd334052b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.705+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd34c068fd0 con 0x7fd35c05f5a0 2020-07-18T03:29:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.705+0000 7fd360f01700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3340506c0 0x7fd334052b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.705+0000 7fd360f01700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3340506c0 0x7fd334052b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd358007000 tx=0x7fd358008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.709+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd34c0416e0 con 0x7fd35c05f5a0 2020-07-18T03:29:20.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.921+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd35c0b22a0 con 0x7fd35c05f5a0 2020-07-18T03:29:20.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.923+0000 7fd3497fa700 1 -- 172.21.15.47:0/721835167 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fd34c044d00 con 0x7fd35c05f5a0 2020-07-18T03:29:20.924 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:20.925 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3340506c0 msgr2=0x7fd334052b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3340506c0 0x7fd334052b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd358007000 tx=0x7fd358008040).stop 2020-07-18T03:29:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd35c05f5a0 msgr2=0x7fd35c0b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd35c05f5a0 0x7fd35c0b2f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fd34c002f30 tx=0x7fd34c015000).stop 2020-07-18T03:29:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 shutdown_connections 2020-07-18T03:29:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3340506c0 0x7fd334052b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 --2- 172.21.15.47:0/721835167 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd35c05f5a0 0x7fd35c0b2f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:20.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 >> 172.21.15.47:0/721835167 conn(0x7fd35c0b9340 msgr2=0x7fd35c05f980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:20.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 shutdown_connections 2020-07-18T03:29:20.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:20.924+0000 7fd363966700 1 -- 172.21.15.47:0/721835167 wait complete. 2020-07-18T03:29:20.928 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:21.139 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:16.949357+0000', 'modified': '2020-07-18T03:29:17.950573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5530': {'gid': 5530, 'name': 'a', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/1368579801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1368579801}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1368579801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:29:21.139 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:29:21.140 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:29:21.278 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5530,"want_state":"up:active","state":"up:active","rank_uptime":4.3185467710000003,"mdsmap_epoch":17,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":9.7000315149999992} 2020-07-18T03:29:21.279 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5530, "mdsmap_epoch": 17, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 4.318546771, "state": "up:active", "uptime": 9.700031515, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:29:21.280 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.392+0000 7fa4133e8700 1 Processor -- start 2020-07-18T03:29:21.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.392+0000 7fa4133e8700 1 -- start start 2020-07-18T03:29:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa4133e8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b5370 0x7fa40c05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa4133e8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c0b41c0 0x7fa40c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa4133e8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa40c0b4620 0x7fa40c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa4133e8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0b5a10 con 0x7fa40c0b5370 2020-07-18T03:29:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa4133e8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0b0040 con 0x7fa40c0b4620 2020-07-18T03:29:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa4133e8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b3af0 con 0x7fa40c0b41c0 2020-07-18T03:29:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa411985700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa40c0b4620 0x7fa40c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa411985700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa40c0b4620 0x7fa40c0ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60988/0 (socket says 172.21.15.47:60988) 2020-07-18T03:29:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.393+0000 7fa411985700 1 -- 172.21.15.47:0/275900914 learned_addr learned my addr 172.21.15.47:0/275900914 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa410983700 1 --2- 172.21.15.47:0/275900914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c0b41c0 0x7fa40c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411184700 1 --2- 172.21.15.47:0/275900914 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b5370 0x7fa40c05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411985700 1 -- 172.21.15.47:0/275900914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c0b41c0 msgr2=0x7fa40c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411985700 1 --2- 172.21.15.47:0/275900914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c0b41c0 0x7fa40c0ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411985700 1 -- 172.21.15.47:0/275900914 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b5370 msgr2=0x7fa40c05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411985700 1 --2- 172.21.15.47:0/275900914 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b5370 0x7fa40c05fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411985700 1 -- 172.21.15.47:0/275900914 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0adcb0 con 0x7fa40c0b4620 2020-07-18T03:29:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa411985700 1 --2- 172.21.15.47:0/275900914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b4620 0x7fa40c0ad7c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa3fc002f30 tx=0x7fa3fc006e90).ready entity=mon.1 client_cookie=d77fabd29fd7cfe2 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa40bfff700 1 -- 172.21.15.47:0/275900914 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3fc009730 con 0x7fa40c0b4620 2020-07-18T03:29:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa40bfff700 1 -- 172.21.15.47:0/275900914 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc009890 con 0x7fa40c0b4620 2020-07-18T03:29:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa40bfff700 1 -- 172.21.15.47:0/275900914 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3fc009bf0 con 0x7fa40c0b4620 2020-07-18T03:29:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 -- 172.21.15.47:0/275900914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b4620 msgr2=0x7fa40c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 --2- 172.21.15.47:0/275900914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b4620 0x7fa40c0ad7c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa3fc002f30 tx=0x7fa3fc006e90).stop 2020-07-18T03:29:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 -- 172.21.15.47:0/275900914 shutdown_connections 2020-07-18T03:29:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 --2- 172.21.15.47:0/275900914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b4620 0x7fa40c0ad7c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 --2- 172.21.15.47:0/275900914 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b5370 0x7fa40c05fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 --2- 172.21.15.47:0/275900914 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c0b41c0 0x7fa40c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 -- 172.21.15.47:0/275900914 >> 172.21.15.47:0/275900914 conn(0x7fa40c0b9340 msgr2=0x7fa40c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 -- 172.21.15.47:0/275900914 shutdown_connections 2020-07-18T03:29:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.394+0000 7fa4133e8700 1 -- 172.21.15.47:0/275900914 wait complete. 2020-07-18T03:29:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.396+0000 7fa4133e8700 1 Processor -- start 2020-07-18T03:29:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.396+0000 7fa4133e8700 1 -- start start 2020-07-18T03:29:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.396+0000 7fa4133e8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b41c0 0x7fa40c138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa40c0b5370 0x7fa40c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c142210 0x7fa40c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c143420 con 0x7fa40c0b41c0 2020-07-18T03:29:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c142a40 con 0x7fa40c0b5370 2020-07-18T03:29:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c142ce0 con 0x7fa40c142210 2020-07-18T03:29:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa40c0b5370 0x7fa40c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa40c0b5370 0x7fa40c0b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60994/0 (socket says 172.21.15.47:60994) 2020-07-18T03:29:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 -- 172.21.15.47:0/2708554197 learned_addr learned my addr 172.21.15.47:0/2708554197 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa411184700 1 --2- 172.21.15.47:0/2708554197 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b41c0 0x7fa40c138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa411985700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c142210 0x7fa40c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 -- 172.21.15.47:0/2708554197 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c142210 msgr2=0x7fa40c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c142210 0x7fa40c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 -- 172.21.15.47:0/2708554197 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b41c0 msgr2=0x7fa40c138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 --2- 172.21.15.47:0/2708554197 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b41c0 0x7fa40c138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 -- 172.21.15.47:0/2708554197 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c142f80 con 0x7fa40c0b5370 2020-07-18T03:29:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa410983700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b5370 0x7fa40c0b2110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa40000c000 tx=0x7fa400006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa40000ca10 con 0x7fa40c0b5370 2020-07-18T03:29:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40000cb70 con 0x7fa40c0b5370 2020-07-18T03:29:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c139300 con 0x7fa40c0b5370 2020-07-18T03:29:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4000123f0 con 0x7fa40c0b5370 2020-07-18T03:29:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.397+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c1426c0 con 0x7fa40c0b5370 2020-07-18T03:29:21.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.398+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c057450 con 0x7fa40c0b5370 2020-07-18T03:29:21.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.398+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa400012620 con 0x7fa40c0b5370 2020-07-18T03:29:21.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.399+0000 7fa4097fa700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa3e4050000 0x7fa3e40524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.399+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa400068160 con 0x7fa40c0b5370 2020-07-18T03:29:21.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.399+0000 7fa411184700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa3e4050000 0x7fa3e40524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.399+0000 7fa411184700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa3e4050000 0x7fa3e40524c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fa40400b000 tx=0x7fa4040091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:21.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.402+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa4000407f0 con 0x7fa40c0b5370 2020-07-18T03:29:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.564+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa40c0b01a0 con 0x7fa40c0b5370 2020-07-18T03:29:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.565+0000 7fa4097fa700 1 -- 172.21.15.47:0/2708554197 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fa400043e10 con 0x7fa40c0b5370 2020-07-18T03:29:21.568 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:21.568 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:17.951029+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:14.526774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:15.768057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa3e4050000 msgr2=0x7fa3e40524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa3e4050000 0x7fa3e40524c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fa40400b000 tx=0x7fa4040091d0).stop 2020-07-18T03:29:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b5370 msgr2=0x7fa40c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b5370 0x7fa40c0b2110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa40000c000 tx=0x7fa400006dd0).stop 2020-07-18T03:29:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 shutdown_connections 2020-07-18T03:29:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa3e4050000 0x7fa3e40524c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa40c142210 0x7fa40c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 --2- 172.21.15.47:0/2708554197 >> v2:172.21.15.18:3300/0 conn(0x7fa40c0b41c0 0x7fa40c138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 --2- 172.21.15.47:0/2708554197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa40c0b5370 0x7fa40c0b2110 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 >> 172.21.15.47:0/2708554197 conn(0x7fa40c0b9340 msgr2=0x7fa40c05fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 shutdown_connections 2020-07-18T03:29:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.566+0000 7fa4133e8700 1 -- 172.21.15.47:0/2708554197 wait complete. 2020-07-18T03:29:21.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:29:14.526774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:29:15.768057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:29:21.581 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:29:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.723+0000 7f6964de2700 1 Processor -- start 2020-07-18T03:29:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.723+0000 7f6964de2700 1 -- start start 2020-07-18T03:29:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f69600b59b0 0x7f69600b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69600b6300 0x7f69600b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6960147de0 con 0x7f69600ad330 2020-07-18T03:29:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69600ae000 con 0x7f69600b59b0 2020-07-18T03:29:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6960148010 con 0x7f69600b6300 2020-07-18T03:29:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69600b6300 0x7f69600b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69600b6300 0x7f69600b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32770/0 (socket says 172.21.15.47:32770) 2020-07-18T03:29:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 -- 172.21.15.47:0/3764289477 learned_addr learned my addr 172.21.15.47:0/3764289477 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695dd9b700 1 --2- 172.21.15.47:0/3764289477 >> v2:172.21.15.18:3300/0 conn(0x7f69600b59b0 0x7f69600b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 -- 172.21.15.47:0/3764289477 >> v2:172.21.15.18:3300/0 conn(0x7f69600b59b0 msgr2=0x7f69600b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 --2- 172.21.15.47:0/3764289477 >> v2:172.21.15.18:3300/0 conn(0x7f69600b59b0 0x7f69600b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 -- 172.21.15.47:0/3764289477 >> v1:172.21.15.18:6789/0 conn(0x7f69600ad330 legacy=0x7f6960143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 -- 172.21.15.47:0/3764289477 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600ad710 con 0x7f69600b6300 2020-07-18T03:29:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695ed9d700 1 --2- 172.21.15.47:0/3764289477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69600b6300 0x7f69600b3a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f69600ae4e0 tx=0x7f6958006e90).ready entity=mon.1 client_cookie=29b89ea5088cf8ca server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695d59a700 1 -- 172.21.15.47:0/3764289477 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6958009930 con 0x7f69600b6300 2020-07-18T03:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695d59a700 1 -- 172.21.15.47:0/3764289477 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958009a90 con 0x7f69600b6300 2020-07-18T03:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f695d59a700 1 -- 172.21.15.47:0/3764289477 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6958009e30 con 0x7f69600b6300 2020-07-18T03:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 -- 172.21.15.47:0/3764289477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69600b6300 msgr2=0x7f69600b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.724+0000 7f6964de2700 1 --2- 172.21.15.47:0/3764289477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69600b6300 0x7f69600b3a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f69600ae4e0 tx=0x7f6958006e90).stop 2020-07-18T03:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.725+0000 7f6964de2700 1 -- 172.21.15.47:0/3764289477 shutdown_connections 2020-07-18T03:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.725+0000 7f6964de2700 1 --2- 172.21.15.47:0/3764289477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69600b6300 0x7f69600b3a10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.725+0000 7f6964de2700 1 --2- 172.21.15.47:0/3764289477 >> v2:172.21.15.18:3300/0 conn(0x7f69600b59b0 0x7f69600b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.725+0000 7f6964de2700 1 -- 172.21.15.47:0/3764289477 >> 172.21.15.47:0/3764289477 conn(0x7f696005f5a0 msgr2=0x7f696005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.725+0000 7f6964de2700 1 -- 172.21.15.47:0/3764289477 shutdown_connections 2020-07-18T03:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.725+0000 7f6964de2700 1 -- 172.21.15.47:0/3764289477 wait complete. 2020-07-18T03:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.726+0000 7f6964de2700 1 Processor -- start 2020-07-18T03:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.726+0000 7f6964de2700 1 -- start start 2020-07-18T03:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f6964de2700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f69600b9da0 0x7f6960146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f6964de2700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6960146d30 con 0x7f69600b59b0 2020-07-18T03:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f6964de2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696013e740 con 0x7f69600ad330 2020-07-18T03:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f6964de2700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69601467f0 con 0x7f69600b9da0 2020-07-18T03:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f695e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f69600ad330 0x7f69601452e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42094/0 (socket says 172.21.15.47:42094) 2020-07-18T03:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f695e59c700 1 -- 172.21.15.47:0/3848934035 learned_addr learned my addr 172.21.15.47:0/3848934035 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f695ed9d700 1 --2- 172.21.15.47:0/3848934035 >> v2:172.21.15.18:3300/0 conn(0x7f69600b9da0 0x7f6960146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081776317 0 0) 0x7f6954002160 con 0x7f69600ad330 2020-07-18T03:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6934003240 con 0x7f69600ad330 2020-07-18T03:29:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964141357 0 0) 0x7f6960146d30 con 0x7f69600b59b0 2020-07-18T03:29:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6954002160 con 0x7f69600b59b0 2020-07-18T03:29:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f695ed9d700 1 -- 172.21.15.47:0/3848934035 >> v1:172.21.15.47:6789/0 conn(0x7f69600ad330 legacy=0x7f69601452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f695ed9d700 1 -- 172.21.15.47:0/3848934035 >> v1:172.21.15.18:6789/0 conn(0x7f69600b59b0 legacy=0x7f696014a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.727+0000 7f695ed9d700 1 -- 172.21.15.47:0/3848934035 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960146a90 con 0x7f69600b9da0 2020-07-18T03:29:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.728+0000 7f695ed9d700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69600b9da0 0x7f6960146070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f69600ae4e0 tx=0x7f6958015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.728+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6958009930 con 0x7f69600b9da0 2020-07-18T03:29:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.728+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958009e70 con 0x7f69600b9da0 2020-07-18T03:29:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.728+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696014ab30 con 0x7f69600b9da0 2020-07-18T03:29:21.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.728+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f695801e650 con 0x7f69600b9da0 2020-07-18T03:29:21.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.728+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6960157590 con 0x7f69600b9da0 2020-07-18T03:29:21.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.729+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960147220 con 0x7f69600b9da0 2020-07-18T03:29:21.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.729+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6958025030 con 0x7f69600b9da0 2020-07-18T03:29:21.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.729+0000 7f694e7fc700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69340504a0 0x7f6934052960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:21.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.729+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f69580691d0 con 0x7f69600b9da0 2020-07-18T03:29:21.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.729+0000 7f695e59c700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69340504a0 0x7f6934052960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:21.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.730+0000 7f695e59c700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69340504a0 0x7f6934052960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f69540022f0 tx=0x7f69540027a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:21.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.733+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6958041880 con 0x7f69600b9da0 2020-07-18T03:29:21.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.893+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f696005b120 con 0x7f69600b9da0 2020-07-18T03:29:21.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.893+0000 7f694e7fc700 1 -- 172.21.15.47:0/3848934035 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7f69580230b0 con 0x7f69600b9da0 2020-07-18T03:29:21.896 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:29:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69340504a0 msgr2=0x7f6934052960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69340504a0 0x7f6934052960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f69540022f0 tx=0x7f69540027a0).stop 2020-07-18T03:29:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69600b9da0 msgr2=0x7f6960146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:21.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69600b9da0 0x7f6960146070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f69600ae4e0 tx=0x7f6958015000).stop 2020-07-18T03:29:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 shutdown_connections 2020-07-18T03:29:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69600b9da0 0x7f6960146070 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 --2- 172.21.15.47:0/3848934035 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69340504a0 0x7f6934052960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 >> 172.21.15.47:0/3848934035 conn(0x7f696005f5a0 msgr2=0x7f69600b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 shutdown_connections 2020-07-18T03:29:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:21.895+0000 7f6964de2700 1 -- 172.21.15.47:0/3848934035 wait complete. 2020-07-18T03:29:21.911 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:29:22.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.031+0000 7fa66ea68700 1 Processor -- start 2020-07-18T03:29:22.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.031+0000 7fa66ea68700 1 -- start start 2020-07-18T03:29:22.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66ea68700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66ea68700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa668061c60 0x7fa6680620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66ea68700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6680af4b0 con 0x7fa668146ac0 2020-07-18T03:29:22.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66ea68700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6680b6d80 con 0x7fa668142a20 2020-07-18T03:29:22.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66ea68700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6680af710 con 0x7fa668061c60 2020-07-18T03:29:22.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa667fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa668061c60 0x7fa6680620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66d005700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa668146ac0 0x7fa668146ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42104/0 (socket says 172.21.15.47:42104) 2020-07-18T03:29:22.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66d005700 1 -- 172.21.15.47:0/3575620771 learned_addr learned my addr 172.21.15.47:0/3575620771 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:22.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa66c804700 1 --2- 172.21.15.47:0/3575620771 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa6677fe700 1 -- 172.21.15.47:0/3575620771 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124789462 0 0) 0x7fa6680af4b0 con 0x7fa668146ac0 2020-07-18T03:29:22.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa6677fe700 1 -- 172.21.15.47:0/3575620771 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa64c003240 con 0x7fa668146ac0 2020-07-18T03:29:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa667fff700 1 -- 172.21.15.47:0/3575620771 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 msgr2=0x7fa668142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa667fff700 1 --2- 172.21.15.47:0/3575620771 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa667fff700 1 -- 172.21.15.47:0/3575620771 >> v1:172.21.15.47:6789/0 conn(0x7fa668146ac0 legacy=0x7fa668146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa667fff700 1 -- 172.21.15.47:0/3575620771 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6680af9b0 con 0x7fa668061c60 2020-07-18T03:29:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.032+0000 7fa667fff700 1 --2- 172.21.15.47:0/3575620771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668061c60 0x7fa6680620f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fa654002f30 tx=0x7fa654006dd0).ready entity=mon.1 client_cookie=62772546ee85871f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa6677fe700 1 -- 172.21.15.47:0/3575620771 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6540097b0 con 0x7fa668061c60 2020-07-18T03:29:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa6677fe700 1 -- 172.21.15.47:0/3575620771 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa654009910 con 0x7fa668061c60 2020-07-18T03:29:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa6677fe700 1 -- 172.21.15.47:0/3575620771 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa654009c70 con 0x7fa668061c60 2020-07-18T03:29:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 -- 172.21.15.47:0/3575620771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668061c60 msgr2=0x7fa6680620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3575620771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668061c60 0x7fa6680620f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fa654002f30 tx=0x7fa654006dd0).stop 2020-07-18T03:29:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 -- 172.21.15.47:0/3575620771 shutdown_connections 2020-07-18T03:29:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3575620771 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3575620771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668061c60 0x7fa6680620f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 -- 172.21.15.47:0/3575620771 >> 172.21.15.47:0/3575620771 conn(0x7fa66805f5a0 msgr2=0x7fa66805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 -- 172.21.15.47:0/3575620771 shutdown_connections 2020-07-18T03:29:22.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.033+0000 7fa66ea68700 1 -- 172.21.15.47:0/3575620771 wait complete. 2020-07-18T03:29:22.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.034+0000 7fa66ea68700 1 Processor -- start 2020-07-18T03:29:22.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.034+0000 7fa66ea68700 1 -- start start 2020-07-18T03:29:22.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa66ea68700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa66ea68700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa668064330 con 0x7fa668063e90 2020-07-18T03:29:22.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa66ea68700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6680645d0 con 0x7fa668146ac0 2020-07-18T03:29:22.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa66ea68700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa668066af0 con 0x7fa668142a20 2020-07-18T03:29:22.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa667fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa668146ac0 0x7fa668141b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42108/0 (socket says 172.21.15.47:42108) 2020-07-18T03:29:22.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa667fff700 1 -- 172.21.15.47:0/3498300186 learned_addr learned my addr 172.21.15.47:0/3498300186 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:22.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392480552 0 0) 0x7fa6680645d0 con 0x7fa668146ac0 2020-07-18T03:29:22.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa66c804700 1 --2- 172.21.15.47:0/3498300186 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.035+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa640003240 con 0x7fa668146ac0 2020-07-18T03:29:22.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 4191417118 0 0) 0x7fa640003240 con 0x7fa668146ac0 2020-07-18T03:29:22.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6680645d0 con 0x7fa668146ac0 2020-07-18T03:29:22.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fa6540097b0 con 0x7fa668146ac0 2020-07-18T03:29:22.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 msgr2=0x7fa668152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 --2- 172.21.15.47:0/3498300186 >> v2:172.21.15.18:3300/0 conn(0x7fa668142a20 0x7fa668152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 >> v1:172.21.15.47:6789/0 conn(0x7fa668146ac0 legacy=0x7fa668141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 >> v1:172.21.15.18:6789/0 conn(0x7fa668063e90 legacy=0x7fa668065cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa640003800 0x7fa640003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6400044e0 0x7fa640004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa640004e10 0x7fa64000b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668066af0 con 0x7fa6400044e0 2020-07-18T03:29:22.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400053b0 con 0x7fa640004e10 2020-07-18T03:29:22.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64000bca0 con 0x7fa640003800 2020-07-18T03:29:22.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66d005700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa640004e10 0x7fa64000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66c804700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa640003800 0x7fa640003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa667fff700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6400044e0 0x7fa640004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66c804700 1 -- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa640004e10 msgr2=0x7fa64000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66c804700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa640004e10 0x7fa64000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66c804700 1 -- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6400044e0 msgr2=0x7fa640004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66c804700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6400044e0 0x7fa640004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66d005700 1 -- 172.21.15.47:0/3498300186 reap_dead start 2020-07-18T03:29:22.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66c804700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668153200 con 0x7fa640003800 2020-07-18T03:29:22.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa668153450 con 0x7fa640003800 2020-07-18T03:29:22.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa667fff700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6400044e0 0x7fa640004920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:22.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.036+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa668152f50 con 0x7fa640003800 2020-07-18T03:29:22.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.037+0000 7fa66c804700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa640003800 0x7fa640003c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fa65c00c000 tx=0x7fa65c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.037+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65c00d480 con 0x7fa640003800 2020-07-18T03:29:22.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.037+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c00e970 con 0x7fa640003800 2020-07-18T03:29:22.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.037+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65c00ecd0 con 0x7fa640003800 2020-07-18T03:29:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.037+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66805b120 con 0x7fa640003800 2020-07-18T03:29:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.038+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa65c024070 con 0x7fa640003800 2020-07-18T03:29:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.038+0000 7fa664ff9700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa640053f60 0x7fa640056420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.038+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa65c068770 con 0x7fa640003800 2020-07-18T03:29:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.038+0000 7fa667fff700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa640053f60 0x7fa640056420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.038+0000 7fa667fff700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa640053f60 0x7fa640056420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fa65401b000 tx=0x7fa654015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.042+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa65c040e60 con 0x7fa640003800 2020-07-18T03:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.202+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6680690d0 con 0x7fa640003800 2020-07-18T03:29:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa664ff9700 1 -- 172.21.15.47:0/3498300186 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa65c0180a0 con 0x7fa640003800 2020-07-18T03:29:22.205 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa640053f60 msgr2=0x7fa640056420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa640053f60 0x7fa640056420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fa65401b000 tx=0x7fa654015000).stop 2020-07-18T03:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa640003800 msgr2=0x7fa640003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa640003800 0x7fa640003c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fa65c00c000 tx=0x7fa65c008040).stop 2020-07-18T03:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 shutdown_connections 2020-07-18T03:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa640053f60 0x7fa640056420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 --2- 172.21.15.47:0/3498300186 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa640003800 0x7fa640003c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 >> 172.21.15.47:0/3498300186 conn(0x7fa66805f5a0 msgr2=0x7fa66814e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 shutdown_connections 2020-07-18T03:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.204+0000 7fa66ea68700 1 -- 172.21.15.47:0/3498300186 wait complete. 2020-07-18T03:29:22.217 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:29:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 Processor -- start 2020-07-18T03:29:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 -- start start 2020-07-18T03:29:22.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8ec8063e70 0x7f8ec8066620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec8064250 0x7f8ec80b8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec8066fa0 con 0x7f8ec8066b10 2020-07-18T03:29:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec8067240 con 0x7f8ec8063e70 2020-07-18T03:29:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecee97700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80b7df0 con 0x7f8ec8064250 2020-07-18T03:29:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ec7fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec8064250 0x7f8ec80b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8eccc33700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8ec8063e70 0x7f8ec8066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecd434700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f8ec8066b10 0x7f8ec80b8780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60892/0 (socket says 172.21.15.47:60892) 2020-07-18T03:29:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ec7fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec8064250 0x7f8ec80b8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:53634/0 (socket says 172.21.15.47:53634) 2020-07-18T03:29:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8ecd434700 1 -- 172.21.15.47:0/1921950121 learned_addr learned my addr 172.21.15.47:0/1921950121 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:22.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8eccc33700 1 -- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec8064250 msgr2=0x7f8ec80b8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:22.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8eccc33700 1 --2- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec8064250 0x7f8ec80b8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8eccc33700 1 -- 172.21.15.47:0/1921950121 >> v1:172.21.15.18:6789/0 conn(0x7f8ec8066b10 legacy=0x7f8ec80b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8eccc33700 1 -- 172.21.15.47:0/1921950121 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec80612f0 con 0x7f8ec8063e70 2020-07-18T03:29:22.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.334+0000 7f8eccc33700 1 --2- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ec8063e70 0x7f8ec8066620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f8eb0002f30 tx=0x7f8eb0006dd0).ready entity=mon.0 client_cookie=eccaa28423c04b2e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ec77fe700 1 -- 172.21.15.47:0/1921950121 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb0009760 con 0x7f8ec8063e70 2020-07-18T03:29:22.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ec77fe700 1 -- 172.21.15.47:0/1921950121 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb00098c0 con 0x7f8ec8063e70 2020-07-18T03:29:22.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ec77fe700 1 -- 172.21.15.47:0/1921950121 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb0009c20 con 0x7f8ec8063e70 2020-07-18T03:29:22.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 -- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ec8063e70 msgr2=0x7f8ec8066620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 --2- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ec8063e70 0x7f8ec8066620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f8eb0002f30 tx=0x7f8eb0006dd0).stop 2020-07-18T03:29:22.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 -- 172.21.15.47:0/1921950121 shutdown_connections 2020-07-18T03:29:22.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 --2- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ec8063e70 0x7f8ec8066620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 --2- 172.21.15.47:0/1921950121 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec8064250 0x7f8ec80b8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 -- 172.21.15.47:0/1921950121 >> 172.21.15.47:0/1921950121 conn(0x7f8ec805f5a0 msgr2=0x7f8ec805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 -- 172.21.15.47:0/1921950121 shutdown_connections 2020-07-18T03:29:22.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.335+0000 7f8ecee97700 1 -- 172.21.15.47:0/1921950121 wait complete. 2020-07-18T03:29:22.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.336+0000 7f8ecee97700 1 Processor -- start 2020-07-18T03:29:22.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.336+0000 7f8ecee97700 1 -- start start 2020-07-18T03:29:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8ecee97700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 0x7f8ec80b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8ecee97700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8ec8129cc0 0x7f8ec812a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8ecee97700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec812a660 con 0x7f8ec8064250 2020-07-18T03:29:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8ecee97700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec812e640 con 0x7f8ec8129cc0 2020-07-18T03:29:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8ecee97700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80b1e80 con 0x7f8ec812e1e0 2020-07-18T03:29:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8eccc33700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f8ec8064250 0x7f8ec80b5f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60894/0 (socket says 172.21.15.47:60894) 2020-07-18T03:29:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8eccc33700 1 -- 172.21.15.47:0/1953732215 learned_addr learned my addr 172.21.15.47:0/1953732215 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.337+0000 7f8ecd434700 1 --2- 172.21.15.47:0/1953732215 >> v2:172.21.15.18:3300/0 conn(0x7f8ec8129cc0 0x7f8ec812a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec7fff700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 0x7f8ec80b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510579702 0 0) 0x7f8ec812a660 con 0x7f8ec8064250 2020-07-18T03:29:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea4003240 con 0x7f8ec8064250 2020-07-18T03:29:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1122236424 0 0) 0x7f8ea4003240 con 0x7f8ec8064250 2020-07-18T03:29:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 --> v1:172.21.15.18:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ec812a660 con 0x7f8ec8064250 2020-07-18T03:29:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v1:172.21.15.18:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8eb0009760 con 0x7f8ec8064250 2020-07-18T03:29:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 msgr2=0x7f8ec80b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 0x7f8ec80b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 >> v2:172.21.15.18:3300/0 conn(0x7f8ec8129cc0 msgr2=0x7f8ec812a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 --2- 172.21.15.47:0/1953732215 >> v2:172.21.15.18:3300/0 conn(0x7f8ec8129cc0 0x7f8ec812a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 >> v1:172.21.15.18:6789/0 conn(0x7f8ec8064250 legacy=0x7f8ec80b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4003880 0x7f8ea4003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 0x7f8ea40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40049d0 0x7f8ea4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec812e640 con 0x7f8ea4003880 2020-07-18T03:29:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80b1e80 con 0x7f8ea40049d0 2020-07-18T03:29:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40042e0 con 0x7f8ec812e1e0 2020-07-18T03:29:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8ecd434700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40049d0 0x7f8ea4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.338+0000 7f8eccc33700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4003880 0x7f8ea4003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8ec7fff700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 0x7f8ea40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8eccc33700 1 -- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 msgr2=0x7f8ea40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8eccc33700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8ec812e1e0 0x7f8ea40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8eccc33700 1 -- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40049d0 msgr2=0x7f8ea4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8eccc33700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40049d0 0x7f8ea4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8eccc33700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec80b2120 con 0x7f8ea4003880 2020-07-18T03:29:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec80b23c0 con 0x7f8ea4003880 2020-07-18T03:29:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec80b3fe0 con 0x7f8ea4003880 2020-07-18T03:29:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8eccc33700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4003880 0x7f8ea4003ca0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f8ea40054b0 tx=0x7f8eb0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb0027460 con 0x7f8ea4003880 2020-07-18T03:29:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb00275c0 con 0x7f8ea4003880 2020-07-18T03:29:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.339+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb0027920 con 0x7f8ea4003880 2020-07-18T03:29:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.340+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec80b2550 con 0x7f8ea4003880 2020-07-18T03:29:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.340+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8eb000f070 con 0x7f8ea4003880 2020-07-18T03:29:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.341+0000 7f8ec4ff9700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8ea4058170 0x7f8ea405a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.341+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8eb006a710 con 0x7f8ea4003880 2020-07-18T03:29:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.341+0000 7f8ec7fff700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8ea4058170 0x7f8ea405a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.341+0000 7f8ec7fff700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8ea4058170 0x7f8ea405a630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8eb800b000 tx=0x7f8eb8006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.345+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8eb0042f50 con 0x7f8ea4003880 2020-07-18T03:29:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.500+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ec805b120 con 0x7f8ea4003880 2020-07-18T03:29:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.502+0000 7f8ec4ff9700 1 -- 172.21.15.47:0/1953732215 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8eb0046570 con 0x7f8ea4003880 2020-07-18T03:29:22.503 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8ea4058170 msgr2=0x7f8ea405a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8ea4058170 0x7f8ea405a630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8eb800b000 tx=0x7f8eb8006dd0).stop 2020-07-18T03:29:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4003880 msgr2=0x7f8ea4003ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 --2- 172.21.15.47:0/1953732215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4003880 0x7f8ea4003ca0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f8ea40054b0 tx=0x7f8eb0015000).stop 2020-07-18T03:29:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecd434700 1 -- 172.21.15.47:0/1953732215 reap_dead start 2020-07-18T03:29:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 shutdown_connections 2020-07-18T03:29:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 >> 172.21.15.47:0/1953732215 conn(0x7f8ec805f5a0 msgr2=0x7f8ec80b81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 shutdown_connections 2020-07-18T03:29:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.503+0000 7f8ecee97700 1 -- 172.21.15.47:0/1953732215 wait complete. 2020-07-18T03:29:22.515 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:29:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.632+0000 7f55e395d700 1 Processor -- start 2020-07-18T03:29:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.632+0000 7f55e395d700 1 -- start start 2020-07-18T03:29:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55dc0b59b0 0x7f55dc0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc147de0 con 0x7f55dc0b6300 2020-07-18T03:29:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc0ae000 con 0x7f55dc0ad330 2020-07-18T03:29:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc148010 con 0x7f55dc0b59b0 2020-07-18T03:29:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e0ef8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55dc0b59b0 0x7f55dc0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e16f9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55dc0ad330 0x7f55dc143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42142/0 (socket says 172.21.15.47:42142) 2020-07-18T03:29:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e16f9700 1 -- 172.21.15.47:0/2273299989 learned_addr learned my addr 172.21.15.47:0/2273299989 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e0ef8700 1 -- 172.21.15.47:0/2273299989 >> v1:172.21.15.47:6789/0 conn(0x7f55dc0ad330 legacy=0x7f55dc143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e0ef8700 1 -- 172.21.15.47:0/2273299989 >> v1:172.21.15.18:6789/0 conn(0x7f55dc0b6300 legacy=0x7f55dc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e0ef8700 1 -- 172.21.15.47:0/2273299989 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0ad710 con 0x7f55dc0b59b0 2020-07-18T03:29:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55cbfff700 1 -- 172.21.15.47:0/2273299989 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144485065 0 0) 0x7f55dc0ae000 con 0x7f55dc0ad330 2020-07-18T03:29:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e0ef8700 1 --2- 172.21.15.47:0/2273299989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b59b0 0x7f55dc0b5e10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f55dc0ae4e0 tx=0x7f55d8006dd0).ready entity=mon.1 client_cookie=1792b07c60b71352 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55cbfff700 1 -- 172.21.15.47:0/2273299989 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d80097b0 con 0x7f55dc0b59b0 2020-07-18T03:29:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55cbfff700 1 -- 172.21.15.47:0/2273299989 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8009910 con 0x7f55dc0b59b0 2020-07-18T03:29:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55cbfff700 1 -- 172.21.15.47:0/2273299989 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d8009c70 con 0x7f55dc0b59b0 2020-07-18T03:29:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- 172.21.15.47:0/2273299989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b59b0 msgr2=0x7f55dc0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 --2- 172.21.15.47:0/2273299989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b59b0 0x7f55dc0b5e10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f55dc0ae4e0 tx=0x7f55d8006dd0).stop 2020-07-18T03:29:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- 172.21.15.47:0/2273299989 shutdown_connections 2020-07-18T03:29:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 --2- 172.21.15.47:0/2273299989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b59b0 0x7f55dc0b5e10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- 172.21.15.47:0/2273299989 >> 172.21.15.47:0/2273299989 conn(0x7f55dc05f5a0 msgr2=0x7f55dc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- 172.21.15.47:0/2273299989 shutdown_connections 2020-07-18T03:29:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.633+0000 7f55e395d700 1 -- 172.21.15.47:0/2273299989 wait complete. 2020-07-18T03:29:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.635+0000 7f55e395d700 1 Processor -- start 2020-07-18T03:29:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 -- start start 2020-07-18T03:29:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55dc0b6300 0x7f55dc063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55dc0b0e90 0x7f55dc066490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc062f20 con 0x7f55dc0ad330 2020-07-18T03:29:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc0b3480 con 0x7f55dc0b6300 2020-07-18T03:29:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0631c0 con 0x7f55dc0b0e90 2020-07-18T03:29:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e16f9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55dc0ad330 0x7f55dc061d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42148/0 (socket says 172.21.15.47:42148) 2020-07-18T03:29:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e16f9700 1 -- 172.21.15.47:0/119906869 learned_addr learned my addr 172.21.15.47:0/119906869 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e0ef8700 1 --2- 172.21.15.47:0/119906869 >> v2:172.21.15.47:3300/0 conn(0x7f55dc0b6300 0x7f55dc063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e1efa700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55dc0b0e90 0x7f55dc066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e0ef8700 1 -- 172.21.15.47:0/119906869 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55dc0b0e90 msgr2=0x7f55dc066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e0ef8700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55dc0b0e90 0x7f55dc066490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e0ef8700 1 -- 172.21.15.47:0/119906869 >> v1:172.21.15.47:6789/0 conn(0x7f55dc0ad330 legacy=0x7f55dc061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561764691 0 0) 0x7f55dc062f20 con 0x7f55dc0ad330 2020-07-18T03:29:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e0ef8700 1 -- 172.21.15.47:0/119906869 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc063460 con 0x7f55dc0b6300 2020-07-18T03:29:22.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e0ef8700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b6300 0x7f55dc063900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f55dc0ae4e0 tx=0x7f55d8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d801d5d0 con 0x7f55dc0b6300 2020-07-18T03:29:22.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d801d730 con 0x7f55dc0b6300 2020-07-18T03:29:22.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc063df0 con 0x7f55dc0b6300 2020-07-18T03:29:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d801da90 con 0x7f55dc0b6300 2020-07-18T03:29:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.636+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc1565c0 con 0x7f55dc0b6300 2020-07-18T03:29:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.638+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc063f80 con 0x7f55dc0b6300 2020-07-18T03:29:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.638+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f55d8025030 con 0x7f55dc0b6300 2020-07-18T03:29:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.638+0000 7f55c97fa700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f55b404fec0 0x7f55b4052380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.639+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f55d80694d0 con 0x7f55dc0b6300 2020-07-18T03:29:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.639+0000 7f55e16f9700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f55b404fec0 0x7f55b4052380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.639+0000 7f55e16f9700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f55b404fec0 0x7f55b4052380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f55cc005510 tx=0x7f55cc0078e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.642+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f55d8041c60 con 0x7f55dc0b6300 2020-07-18T03:29:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.798+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55dc0635f0 con 0x7f55dc0b6300 2020-07-18T03:29:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.799+0000 7f55c97fa700 1 -- 172.21.15.47:0/119906869 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+10 (secure 0 0 0) 0x7f55d8045280 con 0x7f55dc0b6300 2020-07-18T03:29:22.800 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f55b404fec0 msgr2=0x7f55b4052380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f55b404fec0 0x7f55b4052380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f55cc005510 tx=0x7f55cc0078e0).stop 2020-07-18T03:29:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b6300 msgr2=0x7f55dc063900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b6300 0x7f55dc063900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f55dc0ae4e0 tx=0x7f55d8015000).stop 2020-07-18T03:29:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 shutdown_connections 2020-07-18T03:29:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55dc0b0e90 0x7f55dc066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f55b404fec0 0x7f55b4052380 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 --2- 172.21.15.47:0/119906869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55dc0b6300 0x7f55dc063900 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 >> 172.21.15.47:0/119906869 conn(0x7f55dc05f5a0 msgr2=0x7f55dc147460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 shutdown_connections 2020-07-18T03:29:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.800+0000 7f55e395d700 1 -- 172.21.15.47:0/119906869 wait complete. 2020-07-18T03:29:22.812 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:29:22.813 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:22.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.928+0000 7f6354db4700 1 Processor -- start 2020-07-18T03:29:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.928+0000 7f6354db4700 1 -- start start 2020-07-18T03:29:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6354db4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f63500b31f0 0x7f63500b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6354db4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2620 0x7f63501454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6354db4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63500afbd0 con 0x7f63500b2e10 2020-07-18T03:29:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6354db4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6350144e30 con 0x7f63500b31f0 2020-07-18T03:29:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6354db4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63501491c0 con 0x7f63500b2620 2020-07-18T03:29:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f634ed9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2620 0x7f63501454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6347fff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f63500b31f0 0x7f63500b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f634ed9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2620 0x7f63501454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:53668/0 (socket says 172.21.15.47:53668) 2020-07-18T03:29:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6347fff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f63500b31f0 0x7f63500b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:46750/0 (socket says 172.21.15.47:46750) 2020-07-18T03:29:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f634e59c700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f63500b2e10 0x7f63500b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:60922/0 (socket says 172.21.15.47:60922) 2020-07-18T03:29:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f634ed9d700 1 -- 172.21.15.47:0/4083731477 learned_addr learned my addr 172.21.15.47:0/4083731477 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6347fff700 1 -- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2620 msgr2=0x7f63501454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6347fff700 1 --2- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2620 0x7f63501454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6347fff700 1 -- 172.21.15.47:0/4083731477 >> v1:172.21.15.18:6789/0 conn(0x7f63500b2e10 legacy=0x7f63500b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f634dd9b700 1 -- 172.21.15.47:0/4083731477 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567635456 0 0) 0x7f63400010d0 con 0x7f63500b2e10 2020-07-18T03:29:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.929+0000 7f6347fff700 1 -- 172.21.15.47:0/4083731477 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350145050 con 0x7f63500b31f0 2020-07-18T03:29:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6347fff700 1 --2- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63500b31f0 0x7f63500b2130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f63500ba340 tx=0x7f6338006dd0).ready entity=mon.0 client_cookie=317a1f09cb2fccf1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f634dd9b700 1 -- 172.21.15.47:0/4083731477 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63380097b0 con 0x7f63500b31f0 2020-07-18T03:29:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f634dd9b700 1 -- 172.21.15.47:0/4083731477 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6338009910 con 0x7f63500b31f0 2020-07-18T03:29:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f634dd9b700 1 -- 172.21.15.47:0/4083731477 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6338009c70 con 0x7f63500b31f0 2020-07-18T03:29:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 -- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63500b31f0 msgr2=0x7f63500b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 --2- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63500b31f0 0x7f63500b2130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f63500ba340 tx=0x7f6338006dd0).stop 2020-07-18T03:29:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 -- 172.21.15.47:0/4083731477 shutdown_connections 2020-07-18T03:29:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 --2- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2620 0x7f63501454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 --2- 172.21.15.47:0/4083731477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63500b31f0 0x7f63500b2130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 -- 172.21.15.47:0/4083731477 >> 172.21.15.47:0/4083731477 conn(0x7f63500b9340 msgr2=0x7f63500b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 -- 172.21.15.47:0/4083731477 shutdown_connections 2020-07-18T03:29:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.930+0000 7f6354db4700 1 -- 172.21.15.47:0/4083731477 wait complete. 2020-07-18T03:29:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.931+0000 7f6354db4700 1 Processor -- start 2020-07-18T03:29:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.931+0000 7f6354db4700 1 -- start start 2020-07-18T03:29:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f6354db4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2e10 0x7f6350140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f6354db4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63501531e0 0x7f6350062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f6354db4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63501536f0 con 0x7f63500b2620 2020-07-18T03:29:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f6354db4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6350153990 con 0x7f63501531e0 2020-07-18T03:29:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f6354db4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6350153c30 con 0x7f63500b2e10 2020-07-18T03:29:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f634ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63501531e0 0x7f6350062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f634ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63501531e0 0x7f6350062e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32846/0 (socket says 172.21.15.47:32846) 2020-07-18T03:29:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f634ed9d700 1 -- 172.21.15.47:0/2415758866 learned_addr learned my addr 172.21.15.47:0/2415758866 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.932+0000 7f6347fff700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2e10 0x7f6350140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f634ed9d700 1 -- 172.21.15.47:0/2415758866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2e10 msgr2=0x7f6350140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f634ed9d700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2e10 0x7f6350140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f634ed9d700 1 -- 172.21.15.47:0/2415758866 >> v1:172.21.15.18:6789/0 conn(0x7f63500b2620 legacy=0x7f6350156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f634ed9d700 1 -- 172.21.15.47:0/2415758866 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63501540d0 con 0x7f63501531e0 2020-07-18T03:29:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f6347fff700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2e10 0x7f6350140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f634ed9d700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63501531e0 0x7f6350062e10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f6350146420 tx=0x7f6348009270).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f634800cbb0 con 0x7f63501531e0 2020-07-18T03:29:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634800cd10 con 0x7f63501531e0 2020-07-18T03:29:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63480125e0 con 0x7f63501531e0 2020-07-18T03:29:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6350157620 con 0x7f63501531e0 2020-07-18T03:29:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.933+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63501413b0 con 0x7f63501531e0 2020-07-18T03:29:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.934+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6350057450 con 0x7f63501531e0 2020-07-18T03:29:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.934+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6348022070 con 0x7f63501531e0 2020-07-18T03:29:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.934+0000 7f63467fc700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f632404ffa0 0x7f6324052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.934+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6348068360 con 0x7f63501531e0 2020-07-18T03:29:22.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.934+0000 7f634e59c700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f632404ffa0 0x7f6324052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:22.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.935+0000 7f634e59c700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f632404ffa0 0x7f6324052460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f6340000b80 tx=0x7f6340001260).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:22.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:22.938+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6348040af0 con 0x7f63501531e0 2020-07-18T03:29:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.151+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6350153da0 con 0x7f63501531e0 2020-07-18T03:29:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.151+0000 7f63467fc700 1 -- 172.21.15.47:0/2415758866 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f6348016090 con 0x7f63501531e0 2020-07-18T03:29:23.153 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:23.153 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f632404ffa0 msgr2=0x7f6324052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f632404ffa0 0x7f6324052460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f6340000b80 tx=0x7f6340001260).stop 2020-07-18T03:29:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63501531e0 msgr2=0x7f6350062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63501531e0 0x7f6350062e10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f6350146420 tx=0x7f6348009270).stop 2020-07-18T03:29:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 shutdown_connections 2020-07-18T03:29:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f632404ffa0 0x7f6324052460 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63501531e0 0x7f6350062e10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 --2- 172.21.15.47:0/2415758866 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f63500b2e10 0x7f6350140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 >> 172.21.15.47:0/2415758866 conn(0x7f63500b9340 msgr2=0x7f63500b8810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 shutdown_connections 2020-07-18T03:29:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:23.153+0000 7f6354db4700 1 -- 172.21.15.47:0/2415758866 wait complete. 2020-07-18T03:29:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:23.366 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:16.949357+0000', 'modified': '2020-07-18T03:29:17.950573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5530': {'gid': 5530, 'name': 'a', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/1368579801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1368579801}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1368579801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:29:23.367 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:29:23.367 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:29:23.508 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5530,"want_state":"up:active","state":"up:active","rank_uptime":6.5480568200000002,"mdsmap_epoch":17,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":11.929541754000001} 2020-07-18T03:29:23.509 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5530, "mdsmap_epoch": 17, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 6.54805682, "state": "up:active", "uptime": 11.929541754, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:29:23.509 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:29:23.510 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:29:23.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.511 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:23.527 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:23.527 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:23.527 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:23.528 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:23.528 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:23.528 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:23.528 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:23.528 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:23.529 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:23.529 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:23.529 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:23.529 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42627sec preferred_lft 42627sec 2020-07-18T03:29:23.529 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:23.530 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:23.530 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:23.530 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:23.530 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:23.530 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:23.531 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:29:23.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.532 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:29:23.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.601 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:29:23.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.676 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:29:23.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.749 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:29:23.820 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:29:23.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.822 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:29:23.877 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:29:23.878 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:29:23.878 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:29:23.878 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:29:23.879 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:29:23.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.880 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:29:23.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:23.951 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:29:24.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.024 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:29:24.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.096 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:29:24.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.151 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:29:24.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.206 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:29:24.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.284 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-07-18T03:29:24.356 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:29:24.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.357 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-07-18T03:29:24.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.429 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:29:24.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.523 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:29:24.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.585 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:29:24.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.650 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:29:24.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.700 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:29:24.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:24.741 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:29:24.851 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:29:24.852 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:29:24.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:24.853 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:24.868 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:29:24.940 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:29:24.941 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:29:25.016 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:29:25.018 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:29:25.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:25.019 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:29:25.073 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:29:25.074 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:25.117 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:29:25.149 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:29:25.149 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:29:25.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:29:25.213+0000 7fc78134a1c0 -1 init, newargv = 0x55b8cb696b30 newargc=15 2020-07-18T03:29:25.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[37571]: starting ceph client 2020-07-18T03:29:25.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[37571]: starting fuse 2020-07-18T03:29:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:29:25.222 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:29:25.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:25.223 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:29:25.276 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:29:25.276 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:29:25.277 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:29:25.278 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:29:25.278 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:29:25.279 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:29:25.279 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:29:25.279 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:25.280 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:29:25.280 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:29:25.281 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:29:25.281 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:29:25.282 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:25.283 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:29:25.283 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:29:25.283 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:29:25.284 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:29:25.284 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:25.285 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:29:25.285 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:29:25.286 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:29:25.286 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:29:25.287 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:29:25.287 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:29:25.287 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:29:25.288 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:29:25.288 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:25.289 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:29:25.289 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:29:25.408 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.37571.asok 2020-07-18T03:29:25.589 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37571.asok 2020-07-18T03:29:25.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:25.590 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37571.asok status 2020-07-18T03:29:25.743 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:25.743 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:29:25.743 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:29:25.743 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:29:25.744 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:29:25.744 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:29:25.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:29:25.744 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "37571", 2020-07-18T03:29:25.744 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:29:25.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:25.745 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:29:25.745 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:29:25.745 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5719, 2020-07-18T03:29:25.745 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:29:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:29:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:29:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5719 2020-07-18T03:29:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:25.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:29:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 111275234 2020-07-18T03:29:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:25.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:29:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 111275234 2020-07-18T03:29:25.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5719 192.168.0.1:0/111275234", 2020-07-18T03:29:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/111275234", 2020-07-18T03:29:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:29:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 17, 2020-07-18T03:29:25.749 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 43, 2020-07-18T03:29:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:29:25.750 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:29:25.750 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:25.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:25.755 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:29:25.755 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:25.771 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:29:25.772 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:29:25.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:25.773 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:25.845 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:25.998+0000 7f442beb6700 1 Processor -- start 2020-07-18T03:29:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:25.999+0000 7f442beb6700 1 -- start start 2020-07-18T03:29:26.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:25.999+0000 7f442beb6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44240ad9e0 0x7f44241428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:25.999+0000 7f442beb6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240b3ac0 con 0x7f44240b5370 2020-07-18T03:29:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:25.999+0000 7f442beb6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240b3d20 con 0x7f44240b41c0 2020-07-18T03:29:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:25.999+0000 7f442beb6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44240b5a10 con 0x7f44240ad9e0 2020-07-18T03:29:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442a453700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44240ad9e0 0x7f44241428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f4429451700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f44240b41c0 0x7f44240ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42184/0 (socket says 172.21.15.47:42184) 2020-07-18T03:29:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f4429451700 1 -- 172.21.15.47:0/1089492694 learned_addr learned my addr 172.21.15.47:0/1089492694 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442a453700 1 -- 172.21.15.47:0/1089492694 >> v1:172.21.15.47:6789/0 conn(0x7f44240b41c0 legacy=0x7f44240ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442a453700 1 -- 172.21.15.47:0/1089492694 >> v1:172.21.15.18:6789/0 conn(0x7f44240b5370 legacy=0x7f442405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f4428c50700 1 -- 172.21.15.47:0/1089492694 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150301025 0 0) 0x7f44240b3d20 con 0x7f44240b41c0 2020-07-18T03:29:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442a453700 1 -- 172.21.15.47:0/1089492694 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240b5750 con 0x7f44240ad9e0 2020-07-18T03:29:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442a453700 1 --2- 172.21.15.47:0/1089492694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240ad9e0 0x7f44241428f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f44240af270 tx=0x7f441c0091d0).ready entity=mon.1 client_cookie=9e000db7330b4b28 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f4428c50700 1 -- 172.21.15.47:0/1089492694 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f441c009e80 con 0x7f44240ad9e0 2020-07-18T03:29:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f4428c50700 1 -- 172.21.15.47:0/1089492694 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c00bb80 con 0x7f44240ad9e0 2020-07-18T03:29:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f4428c50700 1 -- 172.21.15.47:0/1089492694 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f441c00bcf0 con 0x7f44240ad9e0 2020-07-18T03:29:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 -- 172.21.15.47:0/1089492694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240ad9e0 msgr2=0x7f44241428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 --2- 172.21.15.47:0/1089492694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240ad9e0 0x7f44241428f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f44240af270 tx=0x7f441c0091d0).stop 2020-07-18T03:29:26.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 -- 172.21.15.47:0/1089492694 shutdown_connections 2020-07-18T03:29:26.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 --2- 172.21.15.47:0/1089492694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240ad9e0 0x7f44241428f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 -- 172.21.15.47:0/1089492694 >> 172.21.15.47:0/1089492694 conn(0x7f44240b9340 msgr2=0x7f44240b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 -- 172.21.15.47:0/1089492694 shutdown_connections 2020-07-18T03:29:26.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.000+0000 7f442beb6700 1 -- 172.21.15.47:0/1089492694 wait complete. 2020-07-18T03:29:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.002+0000 7f442beb6700 1 Processor -- start 2020-07-18T03:29:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.002+0000 7f442beb6700 1 -- start start 2020-07-18T03:29:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.002+0000 7f442beb6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44240b5370 0x7f442414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.002+0000 7f442beb6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4424149e80 con 0x7f4424146d00 2020-07-18T03:29:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.002+0000 7f442beb6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442414b780 con 0x7f44240b41c0 2020-07-18T03:29:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.002+0000 7f442beb6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442414a120 con 0x7f44240b5370 2020-07-18T03:29:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429c52700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f44240b41c0 0x7f4424140210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42190/0 (socket says 172.21.15.47:42190) 2020-07-18T03:29:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429451700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44240b5370 0x7f442414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429c52700 1 -- 172.21.15.47:0/1074880501 learned_addr learned my addr 172.21.15.47:0/1074880501 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429451700 1 -- 172.21.15.47:0/1074880501 >> v1:172.21.15.47:6789/0 conn(0x7f44240b41c0 legacy=0x7f4424140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144599090 0 0) 0x7f442414b780 con 0x7f44240b41c0 2020-07-18T03:29:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429451700 1 -- 172.21.15.47:0/1074880501 >> v1:172.21.15.18:6789/0 conn(0x7f4424146d00 legacy=0x7f4424064aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429451700 1 -- 172.21.15.47:0/1074880501 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424147a30 con 0x7f44240b5370 2020-07-18T03:29:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4429451700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240b5370 0x7f442414b290 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f4424148100 tx=0x7f4420007920).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44200072e0 con 0x7f44240b5370 2020-07-18T03:29:26.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420007440 con 0x7f44240b5370 2020-07-18T03:29:26.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4424147c80 con 0x7f44240b5370 2020-07-18T03:29:26.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44200057e0 con 0x7f44240b5370 2020-07-18T03:29:26.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.003+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4424065370 con 0x7f44240b5370 2020-07-18T03:29:26.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.004+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442414afa0 con 0x7f44240b5370 2020-07-18T03:29:26.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.004+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4420010070 con 0x7f44240b5370 2020-07-18T03:29:26.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.005+0000 7f4415ffb700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f43fc04fef0 0x7f43fc0523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.005+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4420067230 con 0x7f44240b5370 2020-07-18T03:29:26.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.005+0000 7f4429c52700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f43fc04fef0 0x7f43fc0523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.005+0000 7f4429c52700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f43fc04fef0 0x7f43fc0523b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f44180022f0 tx=0x7f44180026c0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.008+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f442003f9c0 con 0x7f44240b5370 2020-07-18T03:29:26.166 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.165+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4424057450 con 0x7f44240b5370 2020-07-18T03:29:26.167 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.166+0000 7f4415ffb700 1 -- 172.21.15.47:0/1074880501 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f4420042fe0 con 0x7f44240b5370 2020-07-18T03:29:26.168 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:26.168 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:17.951029+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:14.526774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:15.768057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:26.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f43fc04fef0 msgr2=0x7f43fc0523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f43fc04fef0 0x7f43fc0523b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f44180022f0 tx=0x7f44180026c0).stop 2020-07-18T03:29:26.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240b5370 msgr2=0x7f442414b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240b5370 0x7f442414b290 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f4424148100 tx=0x7f4420007920).stop 2020-07-18T03:29:26.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 shutdown_connections 2020-07-18T03:29:26.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f43fc04fef0 0x7f43fc0523b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 --2- 172.21.15.47:0/1074880501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44240b5370 0x7f442414b290 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 >> 172.21.15.47:0/1074880501 conn(0x7f44240b9340 msgr2=0x7f442405f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 shutdown_connections 2020-07-18T03:29:26.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.167+0000 7f442beb6700 1 -- 172.21.15.47:0/1074880501 wait complete. 2020-07-18T03:29:26.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:29:14.526774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:29:15.768057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:29:26.180 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:29:26.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.300+0000 7fd3ff875700 1 Processor -- start 2020-07-18T03:29:26.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.300+0000 7fd3ff875700 1 -- start start 2020-07-18T03:29:26.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3ff875700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 0x7fd3f8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3ff875700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd3f80b3a10 0x7fd3f80b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3ff875700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f8147de0 con 0x7fd3f80b59b0 2020-07-18T03:29:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3ff875700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80ae000 con 0x7fd3f80b3a10 2020-07-18T03:29:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3ff875700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8148010 con 0x7fd3f80ad330 2020-07-18T03:29:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3fce10700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd3f80b59b0 0x7fd3f80b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42198/0 (socket says 172.21.15.47:42198) 2020-07-18T03:29:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3fce10700 1 -- 172.21.15.47:0/3924578953 learned_addr learned my addr 172.21.15.47:0/3924578953 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3fd611700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 0x7fd3f8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3fde12700 1 --2- 172.21.15.47:0/3924578953 >> v2:172.21.15.18:3300/0 conn(0x7fd3f80b3a10 0x7fd3f80b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046168060 0 0) 0x7fd3f8147de0 con 0x7fd3f80b59b0 2020-07-18T03:29:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.301+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e0003240 con 0x7fd3f80b59b0 2020-07-18T03:29:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3493080403 0 0) 0x7fd3e0003240 con 0x7fd3f80b59b0 2020-07-18T03:29:26.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 msgr2=0x7fd3f8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 0x7fd3f8143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 >> v2:172.21.15.18:3300/0 conn(0x7fd3f80b3a10 msgr2=0x7fd3f80b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 --2- 172.21.15.47:0/3924578953 >> v2:172.21.15.18:3300/0 conn(0x7fd3f80b3a10 0x7fd3f80b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80ad710 con 0x7fd3f80b59b0 2020-07-18T03:29:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fd3f40016d0 con 0x7fd3f80b59b0 2020-07-18T03:29:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 >> v1:172.21.15.47:6789/0 conn(0x7fd3f80b59b0 legacy=0x7fd3f80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3e0003730 0x7fd3e0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 0x7fd3e00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3e00049d0 0x7fd3e0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80ae000 con 0x7fd3e0003730 2020-07-18T03:29:26.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8148010 con 0x7fd3e00049d0 2020-07-18T03:29:26.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00041d0 con 0x7fd3f80ad330 2020-07-18T03:29:26.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3e00049d0 0x7fd3e0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fd611700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3e0003730 0x7fd3e0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fce10700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 0x7fd3e00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 -- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 msgr2=0x7fd3e00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd3f80ad330 0x7fd3e00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 -- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3e0003730 msgr2=0x7fd3e0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3e0003730 0x7fd3e0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 -- 172.21.15.47:0/3924578953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0003240 con 0x7fd3e00049d0 2020-07-18T03:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.302+0000 7fd3fde12700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3e00049d0 0x7fd3e0004e50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fd3f000c000 tx=0x7fd3f00091d0).ready entity=mon.1 client_cookie=dea7ba0230ae811a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3f000ee20 con 0x7fd3e00049d0 2020-07-18T03:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f001a470 con 0x7fd3e00049d0 2020-07-18T03:29:26.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3e7fff700 1 -- 172.21.15.47:0/3924578953 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3f001a7d0 con 0x7fd3e00049d0 2020-07-18T03:29:26.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3ff875700 1 -- 172.21.15.47:0/3924578953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3e00049d0 msgr2=0x7fd3e0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3ff875700 1 --2- 172.21.15.47:0/3924578953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3e00049d0 0x7fd3e0004e50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fd3f000c000 tx=0x7fd3f00091d0).stop 2020-07-18T03:29:26.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3fde12700 1 -- 172.21.15.47:0/3924578953 reap_dead start 2020-07-18T03:29:26.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3ff875700 1 -- 172.21.15.47:0/3924578953 shutdown_connections 2020-07-18T03:29:26.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3ff875700 1 -- 172.21.15.47:0/3924578953 >> 172.21.15.47:0/3924578953 conn(0x7fd3f805f5a0 msgr2=0x7fd3f805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3ff875700 1 -- 172.21.15.47:0/3924578953 shutdown_connections 2020-07-18T03:29:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.303+0000 7fd3ff875700 1 -- 172.21.15.47:0/3924578953 wait complete. 2020-07-18T03:29:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.304+0000 7fd3ff875700 1 Processor -- start 2020-07-18T03:29:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.304+0000 7fd3ff875700 1 -- start start 2020-07-18T03:29:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3ff875700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3f8136520 0x7fd3f8137f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3ff875700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f80b0860 con 0x7fd3f80b3110 2020-07-18T03:29:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3ff875700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f80b0b00 con 0x7fd3f8138480 2020-07-18T03:29:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3ff875700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f8137af0 con 0x7fd3f8136520 2020-07-18T03:29:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fd611700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3f8136520 0x7fd3f8137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fce10700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd3f8138480 0x7fd3f8136c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42210/0 (socket says 172.21.15.47:42210) 2020-07-18T03:29:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fd611700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3f8136520 0x7fd3f8137f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32888/0 (socket says 172.21.15.47:32888) 2020-07-18T03:29:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fce10700 1 -- 172.21.15.47:0/2135876938 learned_addr learned my addr 172.21.15.47:0/2135876938 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fd611700 1 -- 172.21.15.47:0/2135876938 >> v1:172.21.15.47:6789/0 conn(0x7fd3f8138480 legacy=0x7fd3f8136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fd611700 1 -- 172.21.15.47:0/2135876938 >> v1:172.21.15.18:6789/0 conn(0x7fd3f80b3110 legacy=0x7fd3f8137350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529246390 0 0) 0x7fd3f80b0b00 con 0x7fd3f8138480 2020-07-18T03:29:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.305+0000 7fd3fd611700 1 -- 172.21.15.47:0/2135876938 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80b3690 con 0x7fd3f8136520 2020-07-18T03:29:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.306+0000 7fd3fd611700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f8136520 0x7fd3f8137f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fd3f813d5e0 tx=0x7fd3e8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.306+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e800f910 con 0x7fd3f8136520 2020-07-18T03:29:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.306+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f813f870 con 0x7fd3f8136520 2020-07-18T03:29:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.306+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e800fa70 con 0x7fd3f8136520 2020-07-18T03:29:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.306+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e800fdd0 con 0x7fd3f8136520 2020-07-18T03:29:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.306+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f80b1fb0 con 0x7fd3f8136520 2020-07-18T03:29:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.307+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f80b0c90 con 0x7fd3f8136520 2020-07-18T03:29:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.307+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd3e8014440 con 0x7fd3f8136520 2020-07-18T03:29:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.307+0000 7fd3e57fa700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3d004fde0 0x7fd3d00522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.307+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd3e8068500 con 0x7fd3f8136520 2020-07-18T03:29:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.307+0000 7fd3fce10700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3d004fde0 0x7fd3d00522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.307+0000 7fd3fce10700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3d004fde0 0x7fd3d00522a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fd3f40013e0 tx=0x7fd3f40077d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.311+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd3e8040bb0 con 0x7fd3f8136520 2020-07-18T03:29:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.467+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd3f805b120 con 0x7fd3f8136520 2020-07-18T03:29:26.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.467+0000 7fd3e57fa700 1 -- 172.21.15.47:0/2135876938 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd3e80180b0 con 0x7fd3f8136520 2020-07-18T03:29:26.469 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:29:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3d004fde0 msgr2=0x7fd3d00522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3d004fde0 0x7fd3d00522a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fd3f40013e0 tx=0x7fd3f40077d0).stop 2020-07-18T03:29:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f8136520 msgr2=0x7fd3f8137f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f8136520 0x7fd3f8137f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fd3f813d5e0 tx=0x7fd3e8008040).stop 2020-07-18T03:29:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 shutdown_connections 2020-07-18T03:29:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f8136520 0x7fd3f8137f90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 --2- 172.21.15.47:0/2135876938 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd3d004fde0 0x7fd3d00522a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 >> 172.21.15.47:0/2135876938 conn(0x7fd3f805f5a0 msgr2=0x7fd3f80af360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 shutdown_connections 2020-07-18T03:29:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.469+0000 7fd3ff875700 1 -- 172.21.15.47:0/2135876938 wait complete. 2020-07-18T03:29:26.480 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.598+0000 7fe5922f7700 1 Processor -- start 2020-07-18T03:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.599+0000 7fe5922f7700 1 -- start start 2020-07-18T03:29:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.599+0000 7fe5922f7700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe58c143c50 0x7fe58c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.599+0000 7fe5922f7700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58c0ad570 con 0x7fe58c141290 2020-07-18T03:29:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.599+0000 7fe5922f7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58c0ad810 con 0x7fe58c0ad0d0 2020-07-18T03:29:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.599+0000 7fe5922f7700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c0adab0 con 0x7fe58c143c50 2020-07-18T03:29:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.599+0000 7fe5912f5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe58c0ad0d0 0x7fe58c05f760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42216/0 (socket says 172.21.15.47:42216) 2020-07-18T03:29:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe5912f5700 1 -- 172.21.15.47:0/3125337479 learned_addr learned my addr 172.21.15.47:0/3125337479 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe590af4700 1 --2- 172.21.15.47:0/3125337479 >> v2:172.21.15.18:3300/0 conn(0x7fe58c143c50 0x7fe58c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227957847 0 0) 0x7fe58c0ad810 con 0x7fe58c0ad0d0 2020-07-18T03:29:26.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe574003240 con 0x7fe58c0ad0d0 2020-07-18T03:29:26.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927122714 0 0) 0x7fe58c0ad570 con 0x7fe58c141290 2020-07-18T03:29:26.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe58c0ad810 con 0x7fe58c141290 2020-07-18T03:29:26.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3592689899 0 0) 0x7fe574003240 con 0x7fe58c0ad0d0 2020-07-18T03:29:26.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 >> v2:172.21.15.18:3300/0 conn(0x7fe58c143c50 msgr2=0x7fe58c140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 --2- 172.21.15.47:0/3125337479 >> v2:172.21.15.18:3300/0 conn(0x7fe58c143c50 0x7fe58c140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 >> v1:172.21.15.18:6789/0 conn(0x7fe58c141290 legacy=0x7fe58c144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c141690 con 0x7fe58c0ad0d0 2020-07-18T03:29:26.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fe57c0016d0 con 0x7fe58c0ad0d0 2020-07-18T03:29:26.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 >> v1:172.21.15.47:6789/0 conn(0x7fe58c0ad0d0 legacy=0x7fe58c05f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5740038c0 0x7fe574003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe5740044e0 0x7fe574004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe574004e50 0x7fe57400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c0adab0 con 0x7fe5740038c0 2020-07-18T03:29:26.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57400bc20 con 0x7fe574004e50 2020-07-18T03:29:26.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57400bde0 con 0x7fe5740044e0 2020-07-18T03:29:26.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe591af6700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe574004e50 0x7fe57400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe5912f5700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5740038c0 0x7fe574003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.600+0000 7fe590af4700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe5740044e0 0x7fe574004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 -- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe5740044e0 msgr2=0x7fe574004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe5740044e0 0x7fe574004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 -- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5740038c0 msgr2=0x7fe574003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:26.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5740038c0 0x7fe574003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 -- 172.21.15.47:0/3125337479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574003240 con 0x7fe574004e50 2020-07-18T03:29:26.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 -- 172.21.15.47:0/3125337479 reap_dead start 2020-07-18T03:29:26.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe591af6700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe574004e50 0x7fe57400b730 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fe5740054b0 tx=0x7fe584007e60).ready entity=mon.1 client_cookie=bdd6b9f1413b243a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58400cbc0 con 0x7fe574004e50 2020-07-18T03:29:26.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58400cd20 con 0x7fe574004e50 2020-07-18T03:29:26.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe57bfff700 1 -- 172.21.15.47:0/3125337479 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5840125e0 con 0x7fe574004e50 2020-07-18T03:29:26.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 -- 172.21.15.47:0/3125337479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe574004e50 msgr2=0x7fe57400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe574004e50 0x7fe57400b730 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fe5740054b0 tx=0x7fe584007e60).stop 2020-07-18T03:29:26.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 -- 172.21.15.47:0/3125337479 shutdown_connections 2020-07-18T03:29:26.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 --2- 172.21.15.47:0/3125337479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe574004e50 0x7fe57400b730 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 -- 172.21.15.47:0/3125337479 >> 172.21.15.47:0/3125337479 conn(0x7fe58c0b91d0 msgr2=0x7fe58c0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 -- 172.21.15.47:0/3125337479 shutdown_connections 2020-07-18T03:29:26.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.601+0000 7fe5922f7700 1 -- 172.21.15.47:0/3125337479 wait complete. 2020-07-18T03:29:26.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 Processor -- start 2020-07-18T03:29:26.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 -- start start 2020-07-18T03:29:26.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe58c0b4e20 0x7fe58c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe58c0aefe0 0x7fe58c0629f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58c0af760 con 0x7fe58c0b4a40 2020-07-18T03:29:26.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c0b01d0 con 0x7fe58c0aefe0 2020-07-18T03:29:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe5922f7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c0afa00 con 0x7fe58c0b4e20 2020-07-18T03:29:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.603+0000 7fe591af6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe58c0aefe0 0x7fe58c0629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe58c0aefe0 0x7fe58c0629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32912/0 (socket says 172.21.15.47:32912) 2020-07-18T03:29:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 -- 172.21.15.47:0/2877351209 learned_addr learned my addr 172.21.15.47:0/2877351209 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe590af4700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe58c0b4e20 0x7fe58c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 -- 172.21.15.47:0/2877351209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe58c0b4e20 msgr2=0x7fe58c151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe58c0b4e20 0x7fe58c151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 -- 172.21.15.47:0/2877351209 >> v1:172.21.15.18:6789/0 conn(0x7fe58c0b4a40 legacy=0x7fe58c12bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 -- 172.21.15.47:0/2877351209 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c0afca0 con 0x7fe58c0aefe0 2020-07-18T03:29:26.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe591af6700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c0aefe0 0x7fe58c0629f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fe5740054b0 tx=0x7fe584016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58401f070 con 0x7fe58c0aefe0 2020-07-18T03:29:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584012e10 con 0x7fe58c0aefe0 2020-07-18T03:29:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c0af470 con 0x7fe58c0aefe0 2020-07-18T03:29:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58400cbc0 con 0x7fe58c0aefe0 2020-07-18T03:29:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.604+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c062f60 con 0x7fe58c0aefe0 2020-07-18T03:29:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.605+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe58402b070 con 0x7fe58c0aefe0 2020-07-18T03:29:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.605+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c0afe30 con 0x7fe58c0aefe0 2020-07-18T03:29:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.606+0000 7fe5797fa700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe564050010 0x7fe5640524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.606+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe58406af60 con 0x7fe58c0aefe0 2020-07-18T03:29:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.606+0000 7fe5912f5700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe564050010 0x7fe5640524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.606+0000 7fe5912f5700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe564050010 0x7fe5640524d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fe58c064950 tx=0x7fe58c0b9fd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.609+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe5840436f0 con 0x7fe58c0aefe0 2020-07-18T03:29:26.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.769+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe58c12a820 con 0x7fe58c0aefe0 2020-07-18T03:29:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.770+0000 7fe5797fa700 1 -- 172.21.15.47:0/2877351209 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe584046d10 con 0x7fe58c0aefe0 2020-07-18T03:29:26.771 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:26.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe564050010 msgr2=0x7fe5640524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe564050010 0x7fe5640524d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fe58c064950 tx=0x7fe58c0b9fd0).stop 2020-07-18T03:29:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c0aefe0 msgr2=0x7fe58c0629f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c0aefe0 0x7fe58c0629f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fe5740054b0 tx=0x7fe584016000).stop 2020-07-18T03:29:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 shutdown_connections 2020-07-18T03:29:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe564050010 0x7fe5640524d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c0aefe0 0x7fe58c0629f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 --2- 172.21.15.47:0/2877351209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe58c0b4e20 0x7fe58c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 >> 172.21.15.47:0/2877351209 conn(0x7fe58c0b91d0 msgr2=0x7fe58c0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 shutdown_connections 2020-07-18T03:29:26.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.771+0000 7fe5922f7700 1 -- 172.21.15.47:0/2877351209 wait complete. 2020-07-18T03:29:26.782 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:29:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.901+0000 7f9ee8fbb700 1 Processor -- start 2020-07-18T03:29:26.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.901+0000 7f9ee8fbb700 1 -- start start 2020-07-18T03:29:26.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee8fbb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ee40ad330 0x7f9ee4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee8fbb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee4147de0 con 0x7f9ee40b3a10 2020-07-18T03:29:26.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee8fbb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee40ae000 con 0x7f9ee40b59b0 2020-07-18T03:29:26.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee8fbb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee4148010 con 0x7f9ee40ad330 2020-07-18T03:29:26.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ee40ad330 0x7f9ee4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee1d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ee40b59b0 0x7f9ee40b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42236/0 (socket says 172.21.15.47:42236) 2020-07-18T03:29:26.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee1d9b700 1 -- 172.21.15.47:0/1042933178 learned_addr learned my addr 172.21.15.47:0/1042933178 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ee40ad330 0x7f9ee4143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32916/0 (socket says 172.21.15.47:32916) 2020-07-18T03:29:26.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.902+0000 7f9ee259c700 1 -- 172.21.15.47:0/1042933178 >> v1:172.21.15.47:6789/0 conn(0x7f9ee40b59b0 legacy=0x7f9ee40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee159a700 1 -- 172.21.15.47:0/1042933178 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685461472 0 0) 0x7f9ee40ae000 con 0x7f9ee40b59b0 2020-07-18T03:29:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee259c700 1 -- 172.21.15.47:0/1042933178 >> v1:172.21.15.18:6789/0 conn(0x7f9ee40b3a10 legacy=0x7f9ee40b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee259c700 1 -- 172.21.15.47:0/1042933178 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee40ad710 con 0x7f9ee40ad330 2020-07-18T03:29:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee259c700 1 --2- 172.21.15.47:0/1042933178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40ad330 0x7f9ee4143640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9ed8002f30 tx=0x7f9ed8006e60).ready entity=mon.1 client_cookie=607653146eb8eaf8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee159a700 1 -- 172.21.15.47:0/1042933178 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed8010470 con 0x7f9ee40ad330 2020-07-18T03:29:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee159a700 1 -- 172.21.15.47:0/1042933178 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed80105d0 con 0x7f9ee40ad330 2020-07-18T03:29:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee159a700 1 -- 172.21.15.47:0/1042933178 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed8010930 con 0x7f9ee40ad330 2020-07-18T03:29:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/1042933178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40ad330 msgr2=0x7f9ee4143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 --2- 172.21.15.47:0/1042933178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40ad330 0x7f9ee4143640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9ed8002f30 tx=0x7f9ed8006e60).stop 2020-07-18T03:29:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/1042933178 shutdown_connections 2020-07-18T03:29:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 --2- 172.21.15.47:0/1042933178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40ad330 0x7f9ee4143640 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/1042933178 >> 172.21.15.47:0/1042933178 conn(0x7f9ee405f5a0 msgr2=0x7f9ee405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/1042933178 shutdown_connections 2020-07-18T03:29:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.903+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/1042933178 wait complete. 2020-07-18T03:29:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.904+0000 7f9ee8fbb700 1 Processor -- start 2020-07-18T03:29:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.904+0000 7f9ee8fbb700 1 -- start start 2020-07-18T03:29:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee8fbb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ee40b59b0 0x7f9ee4151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee8fbb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee413d830 con 0x7f9ee4144a20 2020-07-18T03:29:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee8fbb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee4064200 con 0x7f9ee40b3a10 2020-07-18T03:29:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee8fbb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee40644a0 con 0x7f9ee40b59b0 2020-07-18T03:29:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ee40b3a10 0x7f9ee40664b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42242/0 (socket says 172.21.15.47:42242) 2020-07-18T03:29:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee259c700 1 -- 172.21.15.47:0/2603938450 learned_addr learned my addr 172.21.15.47:0/2603938450 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ee1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ee40b59b0 0x7f9ee4151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689046151 0 0) 0x7f9ee4064200 con 0x7f9ee40b3a10 2020-07-18T03:29:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb8003240 con 0x7f9ee40b3a10 2020-07-18T03:29:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.905+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103176064 0 0) 0x7f9ee413d830 con 0x7f9ee4144a20 2020-07-18T03:29:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ee1d9b700 1 -- 172.21.15.47:0/2603938450 >> v1:172.21.15.47:6789/0 conn(0x7f9ee40b3a10 legacy=0x7f9ee40664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ee1d9b700 1 -- 172.21.15.47:0/2603938450 >> v1:172.21.15.18:6789/0 conn(0x7f9ee4144a20 legacy=0x7f9ee4064940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ee1d9b700 1 -- 172.21.15.47:0/2603938450 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4066bc0 con 0x7f9ee40b59b0 2020-07-18T03:29:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ee1d9b700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40b59b0 0x7f9ee4151010 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f9ed4000b80 tx=0x7f9ed40079c0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed40072e0 con 0x7f9ee40b59b0 2020-07-18T03:29:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4007440 con 0x7f9ee40b59b0 2020-07-18T03:29:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee413e370 con 0x7f9ee40b59b0 2020-07-18T03:29:26.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed40057e0 con 0x7f9ee40b59b0 2020-07-18T03:29:26.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.906+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee4066240 con 0x7f9ee40b59b0 2020-07-18T03:29:26.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.907+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee40adc80 con 0x7f9ee40b59b0 2020-07-18T03:29:26.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.907+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9ed4010070 con 0x7f9ee40b59b0 2020-07-18T03:29:26.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.907+0000 7f9ed27fc700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9eb80502e0 0x7f9eb80527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:26.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.907+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9ed4067110 con 0x7f9ee40b59b0 2020-07-18T03:29:26.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.907+0000 7f9ee259c700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9eb80502e0 0x7f9eb80527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:26.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.908+0000 7f9ee259c700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9eb80502e0 0x7f9eb80527a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f9ed800f000 tx=0x7f9ed8006e60).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:26.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:26.911+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9ed403f8a0 con 0x7f9ee40b59b0 2020-07-18T03:29:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.070+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ee40b50e0 con 0x7f9ee40b59b0 2020-07-18T03:29:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.070+0000 7f9ed27fc700 1 -- 172.21.15.47:0/2603938450 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9ed4042ec0 con 0x7f9ee40b59b0 2020-07-18T03:29:27.071 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9eb80502e0 msgr2=0x7f9eb80527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9eb80502e0 0x7f9eb80527a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f9ed800f000 tx=0x7f9ed8006e60).stop 2020-07-18T03:29:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40b59b0 msgr2=0x7f9ee4151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40b59b0 0x7f9ee4151010 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f9ed4000b80 tx=0x7f9ed40079c0).stop 2020-07-18T03:29:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 shutdown_connections 2020-07-18T03:29:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9eb80502e0 0x7f9eb80527a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 --2- 172.21.15.47:0/2603938450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ee40b59b0 0x7f9ee4151010 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 >> 172.21.15.47:0/2603938450 conn(0x7f9ee405f5a0 msgr2=0x7f9ee40b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 shutdown_connections 2020-07-18T03:29:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.071+0000 7f9ee8fbb700 1 -- 172.21.15.47:0/2603938450 wait complete. 2020-07-18T03:29:27.083 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:29:27.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.204+0000 7f55a0b16700 1 Processor -- start 2020-07-18T03:29:27.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.204+0000 7f55a0b16700 1 -- start start 2020-07-18T03:29:27.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f55a0b16700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c148b20 0x7f559c148f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f55a0b16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f559c0650b0 0x7f559c065510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f55a0b16700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c145550 con 0x7f559c144e40 2020-07-18T03:29:27.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f55a0b16700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c1457f0 con 0x7f559c0650b0 2020-07-18T03:29:27.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f55a0b16700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c145a90 con 0x7f559c148b20 2020-07-18T03:29:27.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f559ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f559c0650b0 0x7f559c065510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f559ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f559c0650b0 0x7f559c065510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32930/0 (socket says 172.21.15.47:32930) 2020-07-18T03:29:27.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.205+0000 7f559ad9d700 1 -- 172.21.15.47:0/1618434212 learned_addr learned my addr 172.21.15.47:0/1618434212 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f5599d9b700 1 --2- 172.21.15.47:0/1618434212 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c148b20 0x7f559c148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559ad9d700 1 -- 172.21.15.47:0/1618434212 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c148b20 msgr2=0x7f559c148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559ad9d700 1 --2- 172.21.15.47:0/1618434212 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c148b20 0x7f559c148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559ad9d700 1 -- 172.21.15.47:0/1618434212 >> v1:172.21.15.18:6789/0 conn(0x7f559c144e40 legacy=0x7f559c063e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559ad9d700 1 -- 172.21.15.47:0/1618434212 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c145220 con 0x7f559c0650b0 2020-07-18T03:29:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559ad9d700 1 --2- 172.21.15.47:0/1618434212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c0650b0 0x7f559c065510 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f559c05d130 tx=0x7f5594009270).ready entity=mon.1 client_cookie=46f0188a5df70540 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559959a700 1 -- 172.21.15.47:0/1618434212 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f559400bbc0 con 0x7f559c0650b0 2020-07-18T03:29:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559959a700 1 -- 172.21.15.47:0/1618434212 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559400bd20 con 0x7f559c0650b0 2020-07-18T03:29:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f559959a700 1 -- 172.21.15.47:0/1618434212 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5594011400 con 0x7f559c0650b0 2020-07-18T03:29:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 -- 172.21.15.47:0/1618434212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c0650b0 msgr2=0x7f559c065510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 --2- 172.21.15.47:0/1618434212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c0650b0 0x7f559c065510 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f559c05d130 tx=0x7f5594009270).stop 2020-07-18T03:29:27.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 -- 172.21.15.47:0/1618434212 shutdown_connections 2020-07-18T03:29:27.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 --2- 172.21.15.47:0/1618434212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c0650b0 0x7f559c065510 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 --2- 172.21.15.47:0/1618434212 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c148b20 0x7f559c148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 -- 172.21.15.47:0/1618434212 >> 172.21.15.47:0/1618434212 conn(0x7f559c05f5a0 msgr2=0x7f559c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 -- 172.21.15.47:0/1618434212 shutdown_connections 2020-07-18T03:29:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.206+0000 7f55a0b16700 1 -- 172.21.15.47:0/1618434212 wait complete. 2020-07-18T03:29:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 Processor -- start 2020-07-18T03:29:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 -- start start 2020-07-18T03:29:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f559c148b20 0x7f559c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c13ad20 0x7f559c13bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c13b2e0 con 0x7f559c144e40 2020-07-18T03:29:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c13eeb0 con 0x7f559c148b20 2020-07-18T03:29:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f55a0b16700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c13b580 con 0x7f559c13ad20 2020-07-18T03:29:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f559a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f559c144e40 0x7f559c1415c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42256/0 (socket says 172.21.15.47:42256) 2020-07-18T03:29:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f5599d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f559c148b20 0x7f559c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f559a59c700 1 -- 172.21.15.47:0/4172364773 learned_addr learned my addr 172.21.15.47:0/4172364773 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.208+0000 7f559ad9d700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c13ad20 0x7f559c13bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f5599d9b700 1 -- 172.21.15.47:0/4172364773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c13ad20 msgr2=0x7f559c13bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f5599d9b700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c13ad20 0x7f559c13bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f5599d9b700 1 -- 172.21.15.47:0/4172364773 >> v1:172.21.15.47:6789/0 conn(0x7f559c144e40 legacy=0x7f559c1415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040604547 0 0) 0x7f559c13b2e0 con 0x7f559c144e40 2020-07-18T03:29:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f5599d9b700 1 -- 172.21.15.47:0/4172364773 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c13b820 con 0x7f559c148b20 2020-07-18T03:29:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f5599d9b700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c148b20 0x7f559c153d30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f558c007000 tx=0x7f558c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558c00e510 con 0x7f559c148b20 2020-07-18T03:29:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c00c030 con 0x7f559c148b20 2020-07-18T03:29:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c13c1b0 con 0x7f559c148b20 2020-07-18T03:29:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558c00e860 con 0x7f559c148b20 2020-07-18T03:29:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.209+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c13d000 con 0x7f559c148b20 2020-07-18T03:29:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.210+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c13b9b0 con 0x7f559c148b20 2020-07-18T03:29:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.210+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f558c019030 con 0x7f559c148b20 2020-07-18T03:29:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.210+0000 7f558a7fc700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f557004fe90 0x7f5570052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.210+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f558c067360 con 0x7f559c148b20 2020-07-18T03:29:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.210+0000 7f559a59c700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f557004fe90 0x7f5570052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.211+0000 7f559a59c700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f557004fe90 0x7f5570052350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f55900022f0 tx=0x7f55900026c0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.214+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f558c03fba0 con 0x7f559c148b20 2020-07-18T03:29:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.371+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f559c066da0 con 0x7f559c148b20 2020-07-18T03:29:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.371+0000 7f558a7fc700 1 -- 172.21.15.47:0/4172364773 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+10 (secure 0 0 0) 0x7f558c0431c0 con 0x7f559c148b20 2020-07-18T03:29:27.373 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f557004fe90 msgr2=0x7f5570052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f557004fe90 0x7f5570052350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f55900022f0 tx=0x7f55900026c0).stop 2020-07-18T03:29:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c148b20 msgr2=0x7f559c153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c148b20 0x7f559c153d30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f558c007000 tx=0x7f558c008040).stop 2020-07-18T03:29:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 shutdown_connections 2020-07-18T03:29:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f559c13ad20 0x7f559c13bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f557004fe90 0x7f5570052350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 --2- 172.21.15.47:0/4172364773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c148b20 0x7f559c153d30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 >> 172.21.15.47:0/4172364773 conn(0x7f559c05f5a0 msgr2=0x7f559c0654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 shutdown_connections 2020-07-18T03:29:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.372+0000 7f55a0b16700 1 -- 172.21.15.47:0/4172364773 wait complete. 2020-07-18T03:29:27.384 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:29:27.385 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:27.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.499+0000 7faf1d914700 1 Processor -- start 2020-07-18T03:29:27.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.499+0000 7faf1d914700 1 -- start start 2020-07-18T03:29:27.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf180b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf181443f0 0x7faf18147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf18147330 con 0x7faf18147a00 2020-07-18T03:29:27.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf180ae530 con 0x7faf180ad100 2020-07-18T03:29:27.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf180b57e0 con 0x7faf181443f0 2020-07-18T03:29:27.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf181443f0 0x7faf18147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf181443f0 0x7faf18147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32948/0 (socket says 172.21.15.47:32948) 2020-07-18T03:29:27.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 -- 172.21.15.47:0/1678130910 learned_addr learned my addr 172.21.15.47:0/1678130910 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:27.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf16ffd700 1 --2- 172.21.15.47:0/1678130910 >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf180b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 -- 172.21.15.47:0/1678130910 >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 msgr2=0x7faf180b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 --2- 172.21.15.47:0/1678130910 >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf180b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 -- 172.21.15.47:0/1678130910 >> v1:172.21.15.18:6789/0 conn(0x7faf18147a00 legacy=0x7faf1805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 -- 172.21.15.47:0/1678130910 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf180b5a80 con 0x7faf181443f0 2020-07-18T03:29:27.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf177fe700 1 --2- 172.21.15.47:0/1678130910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf181443f0 0x7faf18147e40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7faf1805ca30 tx=0x7faf08006e90).ready entity=mon.1 client_cookie=e3af6d15958b4466 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf15ffb700 1 -- 172.21.15.47:0/1678130910 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf08009930 con 0x7faf181443f0 2020-07-18T03:29:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf15ffb700 1 -- 172.21.15.47:0/1678130910 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf08009a90 con 0x7faf181443f0 2020-07-18T03:29:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf15ffb700 1 -- 172.21.15.47:0/1678130910 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf08009e30 con 0x7faf181443f0 2020-07-18T03:29:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- 172.21.15.47:0/1678130910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf181443f0 msgr2=0x7faf18147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 --2- 172.21.15.47:0/1678130910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf181443f0 0x7faf18147e40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7faf1805ca30 tx=0x7faf08006e90).stop 2020-07-18T03:29:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- 172.21.15.47:0/1678130910 shutdown_connections 2020-07-18T03:29:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 --2- 172.21.15.47:0/1678130910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf181443f0 0x7faf18147e40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 --2- 172.21.15.47:0/1678130910 >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf180b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- 172.21.15.47:0/1678130910 >> 172.21.15.47:0/1678130910 conn(0x7faf180b9340 msgr2=0x7faf180b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- 172.21.15.47:0/1678130910 shutdown_connections 2020-07-18T03:29:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.500+0000 7faf1d914700 1 -- 172.21.15.47:0/1678130910 wait complete. 2020-07-18T03:29:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.502+0000 7faf1d914700 1 Processor -- start 2020-07-18T03:29:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.502+0000 7faf1d914700 1 -- start start 2020-07-18T03:29:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf1d914700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf18142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf1d914700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faf18147a00 0x7faf18143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf1d914700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf180b1f30 con 0x7faf181405f0 2020-07-18T03:29:27.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf1d914700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf18142620 con 0x7faf180ad100 2020-07-18T03:29:27.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf1d914700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf18158b40 con 0x7faf18147a00 2020-07-18T03:29:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf16ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf18142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf167fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faf18147a00 0x7faf18143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf177fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7faf181405f0 0x7faf18158fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:32808/0 (socket says 172.21.15.47:32808) 2020-07-18T03:29:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf167fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faf18147a00 0x7faf18143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:53782/0 (socket says 172.21.15.47:53782) 2020-07-18T03:29:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf177fe700 1 -- 172.21.15.47:0/4065958019 learned_addr learned my addr 172.21.15.47:0/4065958019 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf16ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faf180ad100 0x7faf18142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:46862/0 (socket says 172.21.15.47:46862) 2020-07-18T03:29:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995319661 0 0) 0x7faf180b1f30 con 0x7faf181405f0 2020-07-18T03:29:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef4003240 con 0x7faf181405f0 2020-07-18T03:29:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf16ffd700 1 -- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faf18147a00 msgr2=0x7faf18143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf16ffd700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faf18147a00 0x7faf18143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.503+0000 7faf16ffd700 1 -- 172.21.15.47:0/4065958019 >> v1:172.21.15.18:6789/0 conn(0x7faf181405f0 legacy=0x7faf18158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf16ffd700 1 -- 172.21.15.47:0/4065958019 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18143a60 con 0x7faf180ad100 2020-07-18T03:29:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf16ffd700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf180ad100 0x7faf18142030 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7faf10007000 tx=0x7faf10008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf10007750 con 0x7faf180ad100 2020-07-18T03:29:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf181596f0 con 0x7faf180ad100 2020-07-18T03:29:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf100078b0 con 0x7faf180ad100 2020-07-18T03:29:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf10007c10 con 0x7faf180ad100 2020-07-18T03:29:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.504+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf18152650 con 0x7faf180ad100 2020-07-18T03:29:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.505+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf18142760 con 0x7faf180ad100 2020-07-18T03:29:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.505+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7faf1001b460 con 0x7faf180ad100 2020-07-18T03:29:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.505+0000 7faf0effd700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7faef4050330 0x7faef40527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.506+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faf10068040 con 0x7faf180ad100 2020-07-18T03:29:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.506+0000 7faf167fc700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7faef4050330 0x7faef40527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.506+0000 7faf167fc700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7faef4050330 0x7faef40527f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7faf0000c000 tx=0x7faf00008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.509+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7faf100406f0 con 0x7faf180ad100 2020-07-18T03:29:27.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.719+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf18057450 con 0x7faf180ad100 2020-07-18T03:29:27.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.720+0000 7faf0effd700 1 -- 172.21.15.47:0/4065958019 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7faf10043d10 con 0x7faf180ad100 2020-07-18T03:29:27.722 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:27.722 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:27.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7faef4050330 msgr2=0x7faef40527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7faef4050330 0x7faef40527f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7faf0000c000 tx=0x7faf00008040).stop 2020-07-18T03:29:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf180ad100 msgr2=0x7faf18142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf180ad100 0x7faf18142030 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7faf10007000 tx=0x7faf10008040).stop 2020-07-18T03:29:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 shutdown_connections 2020-07-18T03:29:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7faef4050330 0x7faef40527f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf180ad100 0x7faf18142030 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 --2- 172.21.15.47:0/4065958019 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faf18147a00 0x7faf18143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 >> 172.21.15.47:0/4065958019 conn(0x7faf180b9340 msgr2=0x7faf181469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 shutdown_connections 2020-07-18T03:29:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:27.722+0000 7faf1d914700 1 -- 172.21.15.47:0/4065958019 wait complete. 2020-07-18T03:29:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:27.935 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:16.949357+0000', 'modified': '2020-07-18T03:29:17.950573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5530': {'gid': 5530, 'name': 'a', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/1368579801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1368579801}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1368579801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:29:27.935 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:29:27.936 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:29:28.072 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5530,"want_state":"up:active","state":"up:active","rank_uptime":11.112667318,"mdsmap_epoch":17,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":16.494153207} 2020-07-18T03:29:28.073 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5530, "mdsmap_epoch": 17, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 11.112667318, "state": "up:active", "uptime": 16.494153207, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:29:28.074 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:29:28.074 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:29:28.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.075 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:28.090 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:28.090 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:28.091 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:28.091 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.091 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:28.091 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.092 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:28.092 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:28.092 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:28.092 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:28.092 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:28.092 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42623sec preferred_lft 42623sec 2020-07-18T03:29:28.093 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:28.093 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.093 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:28.093 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:28.093 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:28.094 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:28.094 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:28.094 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 92:83:11:8a:a5:bd brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:28.094 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:29:28.094 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.095 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::43e:8aff:fe02:cd77/64 scope link 2020-07-18T03:29:28.095 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.095 INFO:teuthology.orchestra.run.smithi047.stdout:13: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:29:28.095 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 92:83:11:8a:a5:bd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:29:28.095 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::9083:11ff:fe8a:a5bd/64 scope link 2020-07-18T03:29:28.095 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.096 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:29:28.166 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:29:28.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.167 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:29:28.221 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:29:28.221 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:29:28.222 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:29:28.222 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:29:28.222 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:29:28.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.223 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:29:28.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.295 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:29:28.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.367 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:29:28.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.439 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-07-18T03:29:28.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.495 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:29:28.548 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:29:28.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.549 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:29:28.603 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:29:28.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.604 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:29:28.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.685 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1' 2020-07-18T03:29:28.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.758 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:29:28.841 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:28.842 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:28.842 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:28.843 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.843 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:28.844 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.844 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if13: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:28.844 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 26:e2:74:4e:d5:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:29:28.844 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:29:28.845 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.845 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::24e2:74ff:fe4e:d57c/64 scope link 2020-07-18T03:29:28.845 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.851 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:29:28.892 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:28.893 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:28.893 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:28.893 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.894 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:28.894 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.894 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if13: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:28.894 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 26:e2:74:4e:d5:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:29:28.895 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:29:28.895 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.895 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::24e2:74ff:fe4e:d57c/64 scope link 2020-07-18T03:29:28.895 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:28.901 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-07-18T03:29:28.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.903 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1' 2020-07-18T03:29:28.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:28.974 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:29:29.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:29.062 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:29:29.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:29.126 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:29:29.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:29.197 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:29:29.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:29.251 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 up' 2020-07-18T03:29:29.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:29.291 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.1 master ceph-brx' 2020-07-18T03:29:29.397 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:29:29.398 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:29:29.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:29.399 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:29.414 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:29:29.486 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:29:29.487 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:29:29.557 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:29:29.558 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:29:29.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:29.559 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:29:29.613 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:29:29.614 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:29:29.615 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:29.656 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:29:29.690 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:29:29.690 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:29:29.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:29:29.755+0000 7f4a3724a1c0 -1 init, newargv = 0x55ee2b8449a0 newargc=15 2020-07-18T03:29:29.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[38415]: starting ceph client 2020-07-18T03:29:29.763 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:29:29.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[38415]: starting fuse 2020-07-18T03:29:29.764 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:29:29.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:29.765 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:29:29.819 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:29:29.820 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:29:29.820 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:29:29.821 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:29:29.821 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:29:29.822 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:29:29.822 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:29:29.823 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:29:29.823 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:29.823 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:29:29.824 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:29:29.824 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:29:29.825 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:29:29.825 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:29.826 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:29:29.826 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:29:29.826 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:29:29.827 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:29:29.827 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:29.828 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:29:29.828 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:29:29.830 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:29:29.830 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:29:29.831 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:29:29.831 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:29:29.832 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:29:29.832 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:29:29.833 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:29.833 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:29:29.833 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:29:29.951 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.38415.asok 2020-07-18T03:29:30.132 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38415.asok 2020-07-18T03:29:30.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:30.133 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38415.asok status 2020-07-18T03:29:30.286 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:30.286 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:29:30.286 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:29:30.286 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:29:30.287 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:29:30.287 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:29:30.287 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:29:30.287 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "38415", 2020-07-18T03:29:30.287 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:29:30.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:30.288 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:29:30.288 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:29:30.288 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5776, 2020-07-18T03:29:30.288 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5776 2020-07-18T03:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:30.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:30.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:29:30.290 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1096649958 2020-07-18T03:29:30.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:30.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:30.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:30.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:30.291 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:29:30.291 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1096649958 2020-07-18T03:29:30.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:30.291 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5776 192.168.0.2:0/1096649958", 2020-07-18T03:29:30.292 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/1096649958", 2020-07-18T03:29:30.292 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:29:30.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 17, 2020-07-18T03:29:30.292 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 43, 2020-07-18T03:29:30.292 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:29:30.292 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:29:30.293 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:30.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:30.295 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:29:30.295 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:30.310 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:29:30.311 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:29:30.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:30.312 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:30.384 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2020-07-18T03:29:30.553 INFO:teuthology.orchestra.run.smithi047.stdout:{"mds_reconnect_timeout":"45.000000"} 2020-07-18T03:29:30.553 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2020-07-18T03:29:30.554 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2020-07-18T03:29:30.683 INFO:teuthology.orchestra.run.smithi047.stdout:{"ms_max_backoff":"15.000000"} 2020-07-18T03:29:30.684 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2020-07-18T03:29:30.685 INFO:tasks.cephfs.mount:Creating file a 2020-07-18T03:29:30.685 INFO:teuthology.orchestra.run.smithi047:> sudo touch /home/ubuntu/cephtest/mnt.0/a 2020-07-18T03:29:30.724 INFO:tasks.cephfs.mount:Creating file b 2020-07-18T03:29:30.725 INFO:teuthology.orchestra.run.smithi047:> sudo touch /home/ubuntu/cephtest/mnt.0/b 2020-07-18T03:29:30.798 INFO:tasks.cephfs.mount:Creating file c 2020-07-18T03:29:30.798 INFO:teuthology.orchestra.run.smithi047:> sudo touch /home/ubuntu/cephtest/mnt.0/c 2020-07-18T03:29:30.870 INFO:tasks.cephfs.mount:Checking file a 2020-07-18T03:29:30.871 INFO:teuthology.orchestra.run.smithi047:> sudo ls /home/ubuntu/cephtest/mnt.0/a 2020-07-18T03:29:30.941 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.0/a 2020-07-18T03:29:30.943 INFO:tasks.cephfs.mount:Checking file b 2020-07-18T03:29:30.943 INFO:teuthology.orchestra.run.smithi047:> sudo ls /home/ubuntu/cephtest/mnt.0/b 2020-07-18T03:29:31.013 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.0/b 2020-07-18T03:29:31.015 INFO:tasks.cephfs.mount:Checking file c 2020-07-18T03:29:31.016 INFO:teuthology.orchestra.run.smithi047:> sudo ls /home/ubuntu/cephtest/mnt.0/c 2020-07-18T03:29:31.086 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.0/c 2020-07-18T03:29:31.088 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:29:31.089 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:29:31.089 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:31.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[37571]: fuse finished with error 0 and tester_r 0 2020-07-18T03:29:31.168 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:29:37.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:37.272 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:37.291 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:29:37.292 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:29:37.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:37.293 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:29:37.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:37.378 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:29:37.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:37.446 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:29:37.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:37.492 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:37.548 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:37.549 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:37.549 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:37.549 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:37.550 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:37.550 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:37.550 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:37.551 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.552 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:37.552 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.553 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:37.553 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42613sec preferred_lft 42613sec 2020-07-18T03:29:37.554 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:37.554 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:37.554 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:37.555 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.555 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:37.555 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.556 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:37.556 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.556 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:29:37.557 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:37.557 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::43e:8aff:fe02:cd77/64 scope link 2020-07-18T03:29:37.557 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:37.558 INFO:teuthology.orchestra.run.smithi047.stdout:14: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:29:37.558 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:29:37.558 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::640f:ddff:fe6f:26c/64 scope link 2020-07-18T03:29:37.558 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:37.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:37.560 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:29:37.620 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.620 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:37.621 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.621 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.621 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.621 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.622 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.622 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.622 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.622 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.622 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.623 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:37.623 INFO:teuthology.orchestra.run.smithi047.stdout:14: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:37.623 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:29:37.624 INFO:tasks.cephfs.mount:Checking file a 2020-07-18T03:29:37.625 INFO:teuthology.orchestra.run.smithi047:> sudo ls /home/ubuntu/cephtest/mnt.1/a 2020-07-18T03:29:37.696 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.1/a 2020-07-18T03:29:37.698 INFO:tasks.cephfs.mount:Checking file b 2020-07-18T03:29:37.698 INFO:teuthology.orchestra.run.smithi047:> sudo ls /home/ubuntu/cephtest/mnt.1/b 2020-07-18T03:29:37.769 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.1/b 2020-07-18T03:29:37.770 INFO:tasks.cephfs.mount:Checking file c 2020-07-18T03:29:37.770 INFO:teuthology.orchestra.run.smithi047:> sudo ls /home/ubuntu/cephtest/mnt.1/c 2020-07-18T03:29:37.841 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.1/c 2020-07-18T03:29:37.843 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.996+0000 7fc2b5d88700 1 Processor -- start 2020-07-18T03:29:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.996+0000 7fc2b5d88700 1 -- start start 2020-07-18T03:29:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2b5d88700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc2b0147150 0x7fc2b01445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2b5d88700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2b5d88700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2b014a720 0x7fc2b014abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2b5d88700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b0147530 con 0x7fc2b0063d20 2020-07-18T03:29:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2b5d88700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b0060ff0 con 0x7fc2b014a720 2020-07-18T03:29:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2b5d88700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b0147740 con 0x7fc2b0147150 2020-07-18T03:29:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2b014a720 0x7fc2b014abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2b014a720 0x7fc2b014abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32976/0 (socket says 172.21.15.47:32976) 2020-07-18T03:29:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 -- 172.21.15.47:0/2226427140 learned_addr learned my addr 172.21.15.47:0/2226427140 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2af7fe700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc2b0147150 0x7fc2b01445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2aeffd700 1 --2- 172.21.15.47:0/2226427140 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 -- 172.21.15.47:0/2226427140 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc2b0147150 msgr2=0x7fc2b01445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc2b0147150 0x7fc2b01445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 -- 172.21.15.47:0/2226427140 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 msgr2=0x7fc2b00641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 --2- 172.21.15.47:0/2226427140 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 -- 172.21.15.47:0/2226427140 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b01479e0 con 0x7fc2b014a720 2020-07-18T03:29:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2af7fe700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc2b0147150 0x7fc2b01445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2aeffd700 1 --2- 172.21.15.47:0/2226427140 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2affff700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b014a720 0x7fc2b014abb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fc2a80032b0 tx=0x7fc2a8009270).ready entity=mon.1 client_cookie=fbb3923425bcf4ff server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.997+0000 7fc2ae7fc700 1 -- 172.21.15.47:0/2226427140 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a800bc00 con 0x7fc2b014a720 2020-07-18T03:29:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2ae7fc700 1 -- 172.21.15.47:0/2226427140 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a800bd60 con 0x7fc2b014a720 2020-07-18T03:29:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2ae7fc700 1 -- 172.21.15.47:0/2226427140 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a8011400 con 0x7fc2b014a720 2020-07-18T03:29:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2226427140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b014a720 msgr2=0x7fc2b014abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b014a720 0x7fc2b014abb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fc2a80032b0 tx=0x7fc2a8009270).stop 2020-07-18T03:29:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2226427140 shutdown_connections 2020-07-18T03:29:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b014a720 0x7fc2b014abb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2226427140 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc2b0147150 0x7fc2b01445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2226427140 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2226427140 >> 172.21.15.47:0/2226427140 conn(0x7fc2b005f5a0 msgr2=0x7fc2b005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2226427140 shutdown_connections 2020-07-18T03:29:38.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.998+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2226427140 wait complete. 2020-07-18T03:29:38.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.999+0000 7fc2b5d88700 1 Processor -- start 2020-07-18T03:29:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:37.999+0000 7fc2b5d88700 1 -- start start 2020-07-18T03:29:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2b5d88700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00ba720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2b5d88700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2b00b9110 0x7fc2b0142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2b5d88700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b0158ce0 con 0x7fc2b0147150 2020-07-18T03:29:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2b5d88700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b0158b00 con 0x7fc2b0063d20 2020-07-18T03:29:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2b5d88700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b0156100 con 0x7fc2b00b9110 2020-07-18T03:29:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2b00b9110 0x7fc2b0142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2b00b9110 0x7fc2b0142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32980/0 (socket says 172.21.15.47:32980) 2020-07-18T03:29:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2affff700 1 -- 172.21.15.47:0/2044953566 learned_addr learned my addr 172.21.15.47:0/2044953566 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:38.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2af7fe700 1 --2- 172.21.15.47:0/2044953566 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.000+0000 7fc2affff700 1 -- 172.21.15.47:0/2044953566 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 msgr2=0x7fc2b00ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc2affff700 1 --2- 172.21.15.47:0/2044953566 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00ba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc2affff700 1 -- 172.21.15.47:0/2044953566 >> v1:172.21.15.18:6789/0 conn(0x7fc2b0147150 legacy=0x7fc2b0066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc2affff700 1 -- 172.21.15.47:0/2044953566 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0158f80 con 0x7fc2b00b9110 2020-07-18T03:29:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc2affff700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b00b9110 0x7fc2b0142e00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fc2b00b5e10 tx=0x7fc2a8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a80109c0 con 0x7fc2b00b9110 2020-07-18T03:29:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a8010b20 con 0x7fc2b00b9110 2020-07-18T03:29:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a8010e80 con 0x7fc2b00b9110 2020-07-18T03:29:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b0158720 con 0x7fc2b00b9110 2020-07-18T03:29:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.001+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b01432f0 con 0x7fc2b00b9110 2020-07-18T03:29:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.002+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b0057450 con 0x7fc2b00b9110 2020-07-18T03:29:38.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.002+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc2a800f070 con 0x7fc2b00b9110 2020-07-18T03:29:38.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.002+0000 7fc28f7fe700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc288050090 0x7fc288052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.002+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc2a8069320 con 0x7fc2b00b9110 2020-07-18T03:29:38.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.002+0000 7fc2af7fe700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc288050090 0x7fc288052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.003+0000 7fc2af7fe700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc288050090 0x7fc288052550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc2a0008000 tx=0x7fc2a0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.006+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc2a8041b60 con 0x7fc2b00b9110 2020-07-18T03:29:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.163+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2b006a100 con 0x7fc2b00b9110 2020-07-18T03:29:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.163+0000 7fc28f7fe700 1 -- 172.21.15.47:0/2044953566 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fc2a8015020 con 0x7fc2b00b9110 2020-07-18T03:29:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc288050090 msgr2=0x7fc288052550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc288050090 0x7fc288052550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc2a0008000 tx=0x7fc2a0008040).stop 2020-07-18T03:29:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b00b9110 msgr2=0x7fc2b0142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b00b9110 0x7fc2b0142e00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fc2b00b5e10 tx=0x7fc2a8016000).stop 2020-07-18T03:29:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 shutdown_connections 2020-07-18T03:29:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc288050090 0x7fc288052550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2044953566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2b00b9110 0x7fc2b0142e00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 --2- 172.21.15.47:0/2044953566 >> v2:172.21.15.18:3300/0 conn(0x7fc2b0063d20 0x7fc2b00ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 >> 172.21.15.47:0/2044953566 conn(0x7fc2b005f5a0 msgr2=0x7fc2b00652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 shutdown_connections 2020-07-18T03:29:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.165+0000 7fc2b5d88700 1 -- 172.21.15.47:0/2044953566 wait complete. 2020-07-18T03:29:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:38.183 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:17.951029+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:14.526774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:15.768057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:38.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:29:14.526774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:29:15.768057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:29:38.185 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:29:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.302+0000 7f25be99d700 1 Processor -- start 2020-07-18T03:29:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.302+0000 7f25be99d700 1 -- start start 2020-07-18T03:29:38.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25be99d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25b80b3a10 0x7f25b80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25be99d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b8147de0 con 0x7f25b80ad330 2020-07-18T03:29:38.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25be99d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b80ae000 con 0x7f25b80b59b0 2020-07-18T03:29:38.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25be99d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8148010 con 0x7f25b80b3a10 2020-07-18T03:29:38.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25affff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f25b80b59b0 0x7f25b80b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42310/0 (socket says 172.21.15.47:42310) 2020-07-18T03:29:38.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25affff700 1 -- 172.21.15.47:0/3622136069 learned_addr learned my addr 172.21.15.47:0/3622136069 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:38.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25bcf3a700 1 --2- 172.21.15.47:0/3622136069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25b80b3a10 0x7f25b80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245703544 0 0) 0x7f25b80ae000 con 0x7f25b80b59b0 2020-07-18T03:29:38.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.303+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2598003240 con 0x7f25b80b59b0 2020-07-18T03:29:38.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874464791 0 0) 0x7f25b8147de0 con 0x7f25b80ad330 2020-07-18T03:29:38.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b80ae000 con 0x7f25b80ad330 2020-07-18T03:29:38.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25bcf3a700 1 -- 172.21.15.47:0/3622136069 >> v1:172.21.15.47:6789/0 conn(0x7f25b80b59b0 legacy=0x7f25b80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25bcf3a700 1 -- 172.21.15.47:0/3622136069 >> v1:172.21.15.18:6789/0 conn(0x7f25b80ad330 legacy=0x7f25b8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25bcf3a700 1 -- 172.21.15.47:0/3622136069 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80ad710 con 0x7f25b80b3a10 2020-07-18T03:29:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25bcf3a700 1 --2- 172.21.15.47:0/3622136069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25b80b3a10 0x7f25b80b3e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f25b00032b0 tx=0x7f25b00091d0).ready entity=mon.2 client_cookie=93e320e0f97d0747 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25b000b920 con 0x7f25b80b3a10 2020-07-18T03:29:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b000ba80 con 0x7f25b80b3a10 2020-07-18T03:29:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25b77fe700 1 -- 172.21.15.47:0/3622136069 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25b000bbf0 con 0x7f25b80b3a10 2020-07-18T03:29:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 -- 172.21.15.47:0/3622136069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25b80b3a10 msgr2=0x7f25b80b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 --2- 172.21.15.47:0/3622136069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25b80b3a10 0x7f25b80b3e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f25b00032b0 tx=0x7f25b00091d0).stop 2020-07-18T03:29:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 -- 172.21.15.47:0/3622136069 shutdown_connections 2020-07-18T03:29:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 --2- 172.21.15.47:0/3622136069 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f25b80b3a10 0x7f25b80b3e50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 -- 172.21.15.47:0/3622136069 >> 172.21.15.47:0/3622136069 conn(0x7f25b805f5a0 msgr2=0x7f25b805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 -- 172.21.15.47:0/3622136069 shutdown_connections 2020-07-18T03:29:38.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.304+0000 7f25be99d700 1 -- 172.21.15.47:0/3622136069 wait complete. 2020-07-18T03:29:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25be99d700 1 Processor -- start 2020-07-18T03:29:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25be99d700 1 -- start start 2020-07-18T03:29:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25be99d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25b80b59b0 0x7f25b81424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25be99d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b8143fb0 con 0x7f25b80ad330 2020-07-18T03:29:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25be99d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b814ec80 con 0x7f25b81429a0 2020-07-18T03:29:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25be99d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b8142e30 con 0x7f25b80b59b0 2020-07-18T03:29:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25b80b59b0 0x7f25b81424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25bcf3a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f25b81429a0 0x7f25b814d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42316/0 (socket says 172.21.15.47:42316) 2020-07-18T03:29:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.306+0000 7f25bcf3a700 1 -- 172.21.15.47:0/2473181211 learned_addr learned my addr 172.21.15.47:0/2473181211 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25affff700 1 -- 172.21.15.47:0/2473181211 >> v1:172.21.15.47:6789/0 conn(0x7f25b81429a0 legacy=0x7f25b814d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25affff700 1 -- 172.21.15.47:0/2473181211 >> v1:172.21.15.18:6789/0 conn(0x7f25b80ad330 legacy=0x7f25b8144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298547146 0 0) 0x7f25b814ec80 con 0x7f25b81429a0 2020-07-18T03:29:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25affff700 1 -- 172.21.15.47:0/2473181211 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8144d40 con 0x7f25b80b59b0 2020-07-18T03:29:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25affff700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b80b59b0 0x7f25b81424b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f25a00010d0 tx=0x7f25a0007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25a00072e0 con 0x7f25b80b59b0 2020-07-18T03:29:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a0007440 con 0x7f25b80b59b0 2020-07-18T03:29:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b8153520 con 0x7f25b80b59b0 2020-07-18T03:29:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25a00057e0 con 0x7f25b80b59b0 2020-07-18T03:29:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.307+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b814fbd0 con 0x7f25b80b59b0 2020-07-18T03:29:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.308+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b814ee10 con 0x7f25b80b59b0 2020-07-18T03:29:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.308+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f25a0010070 con 0x7f25b80b59b0 2020-07-18T03:29:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.308+0000 7f25b4ff9700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f259404fd30 0x7f25940521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.308+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f25a0067390 con 0x7f25b80b59b0 2020-07-18T03:29:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.309+0000 7f25b7fff700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f259404fd30 0x7f25940521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.309+0000 7f25b7fff700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f259404fd30 0x7f25940521f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f25b814e410 tx=0x7f25a80076a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.312+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f25a003faa0 con 0x7f25b80b59b0 2020-07-18T03:29:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.470+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f25b80b3750 con 0x7f25b80b59b0 2020-07-18T03:29:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.470+0000 7f25b4ff9700 1 -- 172.21.15.47:0/2473181211 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7f25a00430c0 con 0x7f25b80b59b0 2020-07-18T03:29:38.471 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:29:38.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f259404fd30 msgr2=0x7f25940521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f259404fd30 0x7f25940521f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f25b814e410 tx=0x7f25a80076a0).stop 2020-07-18T03:29:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b80b59b0 msgr2=0x7f25b81424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b80b59b0 0x7f25b81424b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f25a00010d0 tx=0x7f25a0007a20).stop 2020-07-18T03:29:38.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 shutdown_connections 2020-07-18T03:29:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f259404fd30 0x7f25940521f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 --2- 172.21.15.47:0/2473181211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25b80b59b0 0x7f25b81424b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 >> 172.21.15.47:0/2473181211 conn(0x7f25b805f5a0 msgr2=0x7f25b8150520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 shutdown_connections 2020-07-18T03:29:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.471+0000 7f25be99d700 1 -- 172.21.15.47:0/2473181211 wait complete. 2020-07-18T03:29:38.483 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:29:38.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.599+0000 7f949116e700 1 Processor -- start 2020-07-18T03:29:38.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.599+0000 7f949116e700 1 -- start start 2020-07-18T03:29:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.600+0000 7f949116e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f948c0b59b0 0x7f948c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.600+0000 7f949116e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 0x7f948c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.600+0000 7f949116e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c147de0 con 0x7f948c0ad330 2020-07-18T03:29:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.600+0000 7f949116e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0ae000 con 0x7f948c0b59b0 2020-07-18T03:29:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.600+0000 7f949116e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c148010 con 0x7f948c0b6300 2020-07-18T03:29:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948b59e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 0x7f948c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948a59c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f948c0b59b0 0x7f948c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948b59e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 0x7f948c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:53834/0 (socket says 172.21.15.47:53834) 2020-07-18T03:29:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948b59e700 1 -- 172.21.15.47:0/2035976373 learned_addr learned my addr 172.21.15.47:0/2035976373 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f9489d9b700 1 -- 172.21.15.47:0/2035976373 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247818127 0 0) 0x7f948c147de0 con 0x7f948c0ad330 2020-07-18T03:29:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f9489d9b700 1 -- 172.21.15.47:0/2035976373 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946c003240 con 0x7f948c0ad330 2020-07-18T03:29:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948a59c700 1 -- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 msgr2=0x7f948c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948a59c700 1 --2- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 0x7f948c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948a59c700 1 -- 172.21.15.47:0/2035976373 >> v1:172.21.15.18:6789/0 conn(0x7f948c0ad330 legacy=0x7f948c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948a59c700 1 -- 172.21.15.47:0/2035976373 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0ad710 con 0x7f948c0b59b0 2020-07-18T03:29:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948b59e700 1 --2- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 0x7f948c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f948a59c700 1 --2- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f948c0b59b0 0x7f948c0b5e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f947c002f30 tx=0x7f947c006dd0).ready entity=mon.0 client_cookie=68ae29fc61d60c1c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f9489d9b700 1 -- 172.21.15.47:0/2035976373 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f947c0096a0 con 0x7f948c0b59b0 2020-07-18T03:29:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f9489d9b700 1 -- 172.21.15.47:0/2035976373 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c009800 con 0x7f948c0b59b0 2020-07-18T03:29:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f9489d9b700 1 -- 172.21.15.47:0/2035976373 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f947c009b60 con 0x7f948c0b59b0 2020-07-18T03:29:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f949116e700 1 -- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f948c0b59b0 msgr2=0x7f948c0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.601+0000 7f949116e700 1 --2- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f948c0b59b0 0x7f948c0b5e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f947c002f30 tx=0x7f947c006dd0).stop 2020-07-18T03:29:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.602+0000 7f949116e700 1 -- 172.21.15.47:0/2035976373 shutdown_connections 2020-07-18T03:29:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.602+0000 7f949116e700 1 --2- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f948c0b6300 0x7f948c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.602+0000 7f949116e700 1 --2- 172.21.15.47:0/2035976373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f948c0b59b0 0x7f948c0b5e10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.602+0000 7f949116e700 1 -- 172.21.15.47:0/2035976373 >> 172.21.15.47:0/2035976373 conn(0x7f948c05f5a0 msgr2=0x7f948c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.602+0000 7f949116e700 1 -- 172.21.15.47:0/2035976373 shutdown_connections 2020-07-18T03:29:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.602+0000 7f949116e700 1 -- 172.21.15.47:0/2035976373 wait complete. 2020-07-18T03:29:38.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.603+0000 7f949116e700 1 Processor -- start 2020-07-18T03:29:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.603+0000 7f949116e700 1 -- start start 2020-07-18T03:29:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f949116e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f948c0ad330 0x7f948c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f949116e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f948c138a70 0x7f948c138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f949116e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c064500 con 0x7f948c0b6300 2020-07-18T03:29:38.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f949116e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0647a0 con 0x7f948c0ad330 2020-07-18T03:29:38.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f949116e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c064a40 con 0x7f948c138a70 2020-07-18T03:29:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f948c138a70 0x7f948c138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f948c138a70 0x7f948c138ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33008/0 (socket says 172.21.15.47:33008) 2020-07-18T03:29:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 -- 172.21.15.47:0/75858105 learned_addr learned my addr 172.21.15.47:0/75858105 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948ad9d700 1 --2- 172.21.15.47:0/75858105 >> v2:172.21.15.18:3300/0 conn(0x7f948c0ad330 0x7f948c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 -- 172.21.15.47:0/75858105 >> v2:172.21.15.18:3300/0 conn(0x7f948c0ad330 msgr2=0x7f948c14d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 --2- 172.21.15.47:0/75858105 >> v2:172.21.15.18:3300/0 conn(0x7f948c0ad330 0x7f948c14d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 -- 172.21.15.47:0/75858105 >> v1:172.21.15.18:6789/0 conn(0x7f948c0b6300 legacy=0x7f948c1463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 -- 172.21.15.47:0/75858105 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c14dec0 con 0x7f948c138a70 2020-07-18T03:29:38.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.604+0000 7f948b59e700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948c138a70 0x7f948c138ee0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f948c143f30 tx=0x7f9484009270).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.605+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f948400cce0 con 0x7f948c138a70 2020-07-18T03:29:38.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.605+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948400ce40 con 0x7f948c138a70 2020-07-18T03:29:38.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.605+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c144300 con 0x7f948c138a70 2020-07-18T03:29:38.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.605+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94840125e0 con 0x7f948c138a70 2020-07-18T03:29:38.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.605+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c064ea0 con 0x7f948c138a70 2020-07-18T03:29:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.606+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c14e0b0 con 0x7f948c138a70 2020-07-18T03:29:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.606+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9484012790 con 0x7f948c138a70 2020-07-18T03:29:38.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.606+0000 7f9482ffd700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f946804fec0 0x7f9468052380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.606+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9484068280 con 0x7f948c138a70 2020-07-18T03:29:38.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.606+0000 7f948ad9d700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f946804fec0 0x7f9468052380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.606+0000 7f948ad9d700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f946804fec0 0x7f9468052380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f947400d000 tx=0x7f9474007a00).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.609+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9484040a10 con 0x7f948c138a70 2020-07-18T03:29:38.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.766+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f948c05b120 con 0x7f948c138a70 2020-07-18T03:29:38.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.766+0000 7f9482ffd700 1 -- 172.21.15.47:0/75858105 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f94840160a0 con 0x7f948c138a70 2020-07-18T03:29:38.768 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:38.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.767+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f946804fec0 msgr2=0x7f9468052380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.767+0000 7f949116e700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f946804fec0 0x7f9468052380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f947400d000 tx=0x7f9474007a00).stop 2020-07-18T03:29:38.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.767+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948c138a70 msgr2=0x7f948c138ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.767+0000 7f949116e700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948c138a70 0x7f948c138ee0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f948c143f30 tx=0x7f9484009270).stop 2020-07-18T03:29:38.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 shutdown_connections 2020-07-18T03:29:38.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f946804fec0 0x7f9468052380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 --2- 172.21.15.47:0/75858105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f948c138a70 0x7f948c138ee0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 --2- 172.21.15.47:0/75858105 >> v2:172.21.15.18:3300/0 conn(0x7f948c0ad330 0x7f948c14d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 >> 172.21.15.47:0/75858105 conn(0x7f948c05f5a0 msgr2=0x7f948c0618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 shutdown_connections 2020-07-18T03:29:38.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.768+0000 7f949116e700 1 -- 172.21.15.47:0/75858105 wait complete. 2020-07-18T03:29:38.779 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:29:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.896+0000 7fa1de822700 1 Processor -- start 2020-07-18T03:29:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.896+0000 7fa1de822700 1 -- start start 2020-07-18T03:29:38.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1d80ad330 0x7fa1d8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b3a10 0x7fa1d80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d8147de0 con 0x7fa1d80b59b0 2020-07-18T03:29:38.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d80ae000 con 0x7fa1d80ad330 2020-07-18T03:29:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8148010 con 0x7fa1d80b3a10 2020-07-18T03:29:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1d80ad330 0x7fa1d8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1d80ad330 0x7fa1d8143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33014/0 (socket says 172.21.15.47:33014) 2020-07-18T03:29:38.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 -- 172.21.15.47:0/94553897 learned_addr learned my addr 172.21.15.47:0/94553897 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1dcdbf700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b3a10 0x7fa1d80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 -- 172.21.15.47:0/94553897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b3a10 msgr2=0x7fa1d80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b3a10 0x7fa1d80b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 -- 172.21.15.47:0/94553897 >> v1:172.21.15.18:6789/0 conn(0x7fa1d80b59b0 legacy=0x7fa1d80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:38.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 -- 172.21.15.47:0/94553897 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80ad710 con 0x7fa1d80ad330 2020-07-18T03:29:38.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1dcdbf700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b3a10 0x7fa1d80b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:38.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d7fff700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d80ad330 0x7fa1d8143640 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fa1d8067970 tx=0x7fa1c8006e90).ready entity=mon.1 client_cookie=53bd7eb52c9e9bd0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d6ffd700 1 -- 172.21.15.47:0/94553897 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c80098e0 con 0x7fa1d80ad330 2020-07-18T03:29:38.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d6ffd700 1 -- 172.21.15.47:0/94553897 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c8009a40 con 0x7fa1d80ad330 2020-07-18T03:29:38.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1d6ffd700 1 -- 172.21.15.47:0/94553897 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1c8009de0 con 0x7fa1d80ad330 2020-07-18T03:29:38.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 -- 172.21.15.47:0/94553897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d80ad330 msgr2=0x7fa1d8143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.897+0000 7fa1de822700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d80ad330 0x7fa1d8143640 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fa1d8067970 tx=0x7fa1c8006e90).stop 2020-07-18T03:29:38.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.898+0000 7fa1de822700 1 -- 172.21.15.47:0/94553897 shutdown_connections 2020-07-18T03:29:38.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.898+0000 7fa1de822700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b3a10 0x7fa1d80b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.898+0000 7fa1de822700 1 --2- 172.21.15.47:0/94553897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d80ad330 0x7fa1d8143640 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.898+0000 7fa1de822700 1 -- 172.21.15.47:0/94553897 >> 172.21.15.47:0/94553897 conn(0x7fa1d805f5a0 msgr2=0x7fa1d805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:38.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.898+0000 7fa1de822700 1 -- 172.21.15.47:0/94553897 shutdown_connections 2020-07-18T03:29:38.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.898+0000 7fa1de822700 1 -- 172.21.15.47:0/94553897 wait complete. 2020-07-18T03:29:38.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.899+0000 7fa1de822700 1 Processor -- start 2020-07-18T03:29:38.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.899+0000 7fa1de822700 1 -- start start 2020-07-18T03:29:38.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1de822700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b59b0 0x7fa1d81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1de822700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1d813d770 0x7fa1d81504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1de822700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d813fa70 con 0x7fa1d80b3a10 2020-07-18T03:29:38.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1de822700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d813f860 con 0x7fa1d813d770 2020-07-18T03:29:38.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1de822700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d813f4f0 con 0x7fa1d80b59b0 2020-07-18T03:29:38.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1d7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1d80b3a10 0x7fa1d8065700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42340/0 (socket says 172.21.15.47:42340) 2020-07-18T03:29:38.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1dcdbf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1d813d770 0x7fa1d81504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1d7fff700 1 -- 172.21.15.47:0/588086367 learned_addr learned my addr 172.21.15.47:0/588086367 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:38.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1d77fe700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b59b0 0x7fa1d81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1dcdbf700 1 -- 172.21.15.47:0/588086367 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b59b0 msgr2=0x7fa1d81546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1dcdbf700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b59b0 0x7fa1d81546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:38.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1dcdbf700 1 -- 172.21.15.47:0/588086367 >> v1:172.21.15.47:6789/0 conn(0x7fa1d80b3a10 legacy=0x7fa1d8065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:38.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780768488 0 0) 0x7fa1d813fa70 con 0x7fa1d80b3a10 2020-07-18T03:29:38.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1dcdbf700 1 -- 172.21.15.47:0/588086367 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8155090 con 0x7fa1d813d770 2020-07-18T03:29:38.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.900+0000 7fa1dcdbf700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d813d770 0x7fa1d81504c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fa1d0007000 tx=0x7fa1d0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.901+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1d0015070 con 0x7fa1d813d770 2020-07-18T03:29:38.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.901+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0007750 con 0x7fa1d813d770 2020-07-18T03:29:38.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.901+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d8154bc0 con 0x7fa1d813d770 2020-07-18T03:29:38.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.901+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1d0007ab0 con 0x7fa1d813d770 2020-07-18T03:29:38.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.901+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d8150b70 con 0x7fa1d813d770 2020-07-18T03:29:38.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.902+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa1d0007cb0 con 0x7fa1d813d770 2020-07-18T03:29:38.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.902+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d813f630 con 0x7fa1d813d770 2020-07-18T03:29:38.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.902+0000 7fa1b7fff700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa1b004fe40 0x7fa1b0052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:38.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.902+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa1d0067ef0 con 0x7fa1d813d770 2020-07-18T03:29:38.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.902+0000 7fa1d7fff700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa1b004fe40 0x7fa1b0052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:38.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.902+0000 7fa1d7fff700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa1b004fe40 0x7fa1b0052300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fa1c8000b80 tx=0x7fa1c8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:38.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:38.906+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa1d813f630 con 0x7fa1d813d770 2020-07-18T03:29:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.064+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1d813f630 con 0x7fa1d813d770 2020-07-18T03:29:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.064+0000 7fa1b7fff700 1 -- 172.21.15.47:0/588086367 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa1d813f630 con 0x7fa1d813d770 2020-07-18T03:29:39.066 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa1b004fe40 msgr2=0x7fa1b0052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa1b004fe40 0x7fa1b0052300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fa1c8000b80 tx=0x7fa1c8015000).stop 2020-07-18T03:29:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d813d770 msgr2=0x7fa1d81504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d813d770 0x7fa1d81504c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fa1d0007000 tx=0x7fa1d0008040).stop 2020-07-18T03:29:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 shutdown_connections 2020-07-18T03:29:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d813d770 0x7fa1d81504c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa1b004fe40 0x7fa1b0052300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 --2- 172.21.15.47:0/588086367 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1d80b59b0 0x7fa1d81546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 >> 172.21.15.47:0/588086367 conn(0x7fa1d805f5a0 msgr2=0x7fa1d80b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.065+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 shutdown_connections 2020-07-18T03:29:39.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.066+0000 7fa1de822700 1 -- 172.21.15.47:0/588086367 wait complete. 2020-07-18T03:29:39.077 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:29:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.193+0000 7f76ca776700 1 Processor -- start 2020-07-18T03:29:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76ca776700 1 -- start start 2020-07-18T03:29:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76ca776700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 0x7f76c40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76ca776700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c4147de0 con 0x7f76c40b59b0 2020-07-18T03:29:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76ca776700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c40ae000 con 0x7f76c40ad330 2020-07-18T03:29:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76ca776700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4148010 con 0x7f76c40b3a10 2020-07-18T03:29:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76c3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f76c40ad330 0x7f76c4143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42348/0 (socket says 172.21.15.47:42348) 2020-07-18T03:29:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.194+0000 7f76c3fff700 1 -- 172.21.15.47:0/460569798 learned_addr learned my addr 172.21.15.47:0/460569798 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808247485 0 0) 0x7f76c40ae000 con 0x7f76c40ad330 2020-07-18T03:29:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c8d13700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 0x7f76c40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b0003240 con 0x7f76c40ad330 2020-07-18T03:29:39.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2300997904 0 0) 0x7f76b0003240 con 0x7f76c40ad330 2020-07-18T03:29:39.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 msgr2=0x7f76c40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:39.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 0x7f76c40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 >> v1:172.21.15.18:6789/0 conn(0x7f76c40b59b0 legacy=0x7f76c40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40ad710 con 0x7f76c40ad330 2020-07-18T03:29:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f76b40016d0 con 0x7f76c40ad330 2020-07-18T03:29:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 >> v1:172.21.15.47:6789/0 conn(0x7f76c40ad330 legacy=0x7f76c4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 0x7f76b0003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b0003c20 0x7f76b00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76b00049d0 0x7f76b0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4148010 con 0x7f76b0003c20 2020-07-18T03:29:39.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0004140 con 0x7f76b00049d0 2020-07-18T03:29:39.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0004300 con 0x7f76c40b3a10 2020-07-18T03:29:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c3fff700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 0x7f76b0003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c37fe700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b0003c20 0x7f76b00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.195+0000 7f76c8d13700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76b00049d0 0x7f76b0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 -- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 msgr2=0x7f76b0003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c40b3a10 0x7f76b0003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 -- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b0003c20 msgr2=0x7f76b00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b0003c20 0x7f76b00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 -- 172.21.15.47:0/460569798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0003240 con 0x7f76b00049d0 2020-07-18T03:29:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c37fe700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b0003c20 0x7f76b00044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76b00049d0 0x7f76b0004e50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f76bc00c000 tx=0x7f76bc0091d0).ready entity=mon.1 client_cookie=c2fea4c52befc873 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76bc00eb80 con 0x7f76b00049d0 2020-07-18T03:29:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc00ece0 con 0x7f76b00049d0 2020-07-18T03:29:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c2ffd700 1 -- 172.21.15.47:0/460569798 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76bc0125e0 con 0x7f76b00049d0 2020-07-18T03:29:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76ca776700 1 -- 172.21.15.47:0/460569798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76b00049d0 msgr2=0x7f76b0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76ca776700 1 --2- 172.21.15.47:0/460569798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76b00049d0 0x7f76b0004e50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f76bc00c000 tx=0x7f76bc0091d0).stop 2020-07-18T03:29:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76c8d13700 1 -- 172.21.15.47:0/460569798 reap_dead start 2020-07-18T03:29:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76ca776700 1 -- 172.21.15.47:0/460569798 shutdown_connections 2020-07-18T03:29:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76ca776700 1 -- 172.21.15.47:0/460569798 >> 172.21.15.47:0/460569798 conn(0x7f76c405f5a0 msgr2=0x7f76c405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76ca776700 1 -- 172.21.15.47:0/460569798 shutdown_connections 2020-07-18T03:29:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.196+0000 7f76ca776700 1 -- 172.21.15.47:0/460569798 wait complete. 2020-07-18T03:29:39.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 Processor -- start 2020-07-18T03:29:39.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 -- start start 2020-07-18T03:29:39.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76c40b30b0 0x7f76c40b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c413c110 0x7f76c413c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c413ca80 con 0x7f76c40b3490 2020-07-18T03:29:39.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c41446b0 con 0x7f76c40b30b0 2020-07-18T03:29:39.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76ca776700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4144950 con 0x7f76c413c110 2020-07-18T03:29:39.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76c3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76c40b30b0 0x7f76c40b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76c37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f76c40b3490 0x7f76c41415b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42362/0 (socket says 172.21.15.47:42362) 2020-07-18T03:29:39.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.198+0000 7f76c37fe700 1 -- 172.21.15.47:0/3973717380 learned_addr learned my addr 172.21.15.47:0/3973717380 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:39.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c8d13700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c413c110 0x7f76c413c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c3fff700 1 -- 172.21.15.47:0/3973717380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c413c110 msgr2=0x7f76c413c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c3fff700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c413c110 0x7f76c413c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c3fff700 1 -- 172.21.15.47:0/3973717380 >> v1:172.21.15.47:6789/0 conn(0x7f76c40b3490 legacy=0x7f76c41415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441507243 0 0) 0x7f76c413ca80 con 0x7f76c40b3490 2020-07-18T03:29:39.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c3fff700 1 -- 172.21.15.47:0/3973717380 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4144bf0 con 0x7f76c40b30b0 2020-07-18T03:29:39.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c8d13700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c413c110 0x7f76c413c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:39.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76c3fff700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76c40b30b0 0x7f76c40b1000 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f76b400d000 tx=0x7f76b4007780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:39.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76b400fb50 con 0x7f76c40b30b0 2020-07-18T03:29:39.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400fcb0 con 0x7f76c40b30b0 2020-07-18T03:29:39.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c4142010 con 0x7f76c40b30b0 2020-07-18T03:29:39.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76b4012540 con 0x7f76c40b30b0 2020-07-18T03:29:39.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.199+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c4145250 con 0x7f76c40b30b0 2020-07-18T03:29:39.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.200+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f76b401b030 con 0x7f76c40b30b0 2020-07-18T03:29:39.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.200+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c405b120 con 0x7f76c40b30b0 2020-07-18T03:29:39.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.200+0000 7f76a3fff700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f769c04fe00 0x7f769c0522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.200+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f76b4067640 con 0x7f76c40b30b0 2020-07-18T03:29:39.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.201+0000 7f76c37fe700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f769c04fe00 0x7f769c0522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.201+0000 7f76c37fe700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f769c04fe00 0x7f769c0522c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f76b8008000 tx=0x7f76b8009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:39.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.204+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f76b403fdd0 con 0x7f76c40b30b0 2020-07-18T03:29:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.362+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76c40690d0 con 0x7f76c40b30b0 2020-07-18T03:29:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.362+0000 7f76a3fff700 1 -- 172.21.15.47:0/3973717380 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+10 (secure 0 0 0) 0x7f76b4019020 con 0x7f76c40b30b0 2020-07-18T03:29:39.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:39.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f769c04fe00 msgr2=0x7f769c0522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f769c04fe00 0x7f769c0522c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f76b8008000 tx=0x7f76b8009040).stop 2020-07-18T03:29:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76c40b30b0 msgr2=0x7f76c40b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76c40b30b0 0x7f76c40b1000 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f76b400d000 tx=0x7f76b4007780).stop 2020-07-18T03:29:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 shutdown_connections 2020-07-18T03:29:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f76c413c110 0x7f76c413c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76c40b30b0 0x7f76c40b1000 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 --2- 172.21.15.47:0/3973717380 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f769c04fe00 0x7f769c0522c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 >> 172.21.15.47:0/3973717380 conn(0x7f76c405f5a0 msgr2=0x7f76c40b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 shutdown_connections 2020-07-18T03:29:39.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.364+0000 7f76ca776700 1 -- 172.21.15.47:0/3973717380 wait complete. 2020-07-18T03:29:39.375 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:29:39.376 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:39.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.489+0000 7f33e6c88700 1 Processor -- start 2020-07-18T03:29:39.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.489+0000 7f33e6c88700 1 -- start start 2020-07-18T03:29:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e6c88700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f33e00ad100 0x7f33e00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e6c88700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e01443f0 0x7f33e0147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e6c88700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e0147330 con 0x7f33e0147a00 2020-07-18T03:29:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e6c88700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e00ae530 con 0x7f33e00ad100 2020-07-18T03:29:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e6c88700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b57e0 con 0x7f33e01443f0 2020-07-18T03:29:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e4a24700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f33e00ad100 0x7f33e00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33e5225700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e01443f0 0x7f33e0147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33dffff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f33e0147a00 0x7f33e005f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:32902/0 (socket says 172.21.15.47:32902) 2020-07-18T03:29:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.490+0000 7f33dffff700 1 -- 172.21.15.47:0/3709909698 learned_addr learned my addr 172.21.15.47:0/3709909698 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e5225700 1 -- 172.21.15.47:0/3709909698 >> v2:172.21.15.18:3300/0 conn(0x7f33e00ad100 msgr2=0x7f33e00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e5225700 1 --2- 172.21.15.47:0/3709909698 >> v2:172.21.15.18:3300/0 conn(0x7f33e00ad100 0x7f33e00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e5225700 1 -- 172.21.15.47:0/3709909698 >> v1:172.21.15.18:6789/0 conn(0x7f33e0147a00 legacy=0x7f33e005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e5225700 1 -- 172.21.15.47:0/3709909698 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00b5a80 con 0x7f33e01443f0 2020-07-18T03:29:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e4a24700 1 --2- 172.21.15.47:0/3709909698 >> v2:172.21.15.18:3300/0 conn(0x7f33e00ad100 0x7f33e00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e5225700 1 --2- 172.21.15.47:0/3709909698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e01443f0 0x7f33e0147e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f33d80032b0 tx=0x7f33d80091d0).ready entity=mon.2 client_cookie=1eec6b3beb38e05 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33df7fe700 1 -- 172.21.15.47:0/3709909698 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d800b9a0 con 0x7f33e01443f0 2020-07-18T03:29:39.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33df7fe700 1 -- 172.21.15.47:0/3709909698 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800bb00 con 0x7f33e01443f0 2020-07-18T03:29:39.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33df7fe700 1 -- 172.21.15.47:0/3709909698 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d800bc70 con 0x7f33e01443f0 2020-07-18T03:29:39.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 -- 172.21.15.47:0/3709909698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e01443f0 msgr2=0x7f33e0147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3709909698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e01443f0 0x7f33e0147e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f33d80032b0 tx=0x7f33d80091d0).stop 2020-07-18T03:29:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 -- 172.21.15.47:0/3709909698 shutdown_connections 2020-07-18T03:29:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3709909698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e01443f0 0x7f33e0147e40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3709909698 >> v2:172.21.15.18:3300/0 conn(0x7f33e00ad100 0x7f33e00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 -- 172.21.15.47:0/3709909698 >> 172.21.15.47:0/3709909698 conn(0x7f33e00b9340 msgr2=0x7f33e00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 -- 172.21.15.47:0/3709909698 shutdown_connections 2020-07-18T03:29:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.491+0000 7f33e6c88700 1 -- 172.21.15.47:0/3709909698 wait complete. 2020-07-18T03:29:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.493+0000 7f33e6c88700 1 Processor -- start 2020-07-18T03:29:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.493+0000 7f33e6c88700 1 -- start start 2020-07-18T03:29:39.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e6c88700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e00ad100 0x7f33e00b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e6c88700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e013cc90 con 0x7f33e0147a00 2020-07-18T03:29:39.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e6c88700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e013cf30 con 0x7f33e013c750 2020-07-18T03:29:39.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e6c88700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e013d1d0 con 0x7f33e00ad100 2020-07-18T03:29:39.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e5225700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f33e013c750 0x7f33e00b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42378/0 (socket says 172.21.15.47:42378) 2020-07-18T03:29:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e5225700 1 -- 172.21.15.47:0/3445647292 learned_addr learned my addr 172.21.15.47:0/3445647292 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e4a24700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e00ad100 0x7f33e00b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533330503 0 0) 0x7f33e013cf30 con 0x7f33e013c750 2020-07-18T03:29:39.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b8003240 con 0x7f33e013c750 2020-07-18T03:29:39.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899074832 0 0) 0x7f33e013cc90 con 0x7f33e0147a00 2020-07-18T03:29:39.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e013cf30 con 0x7f33e0147a00 2020-07-18T03:29:39.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e4a24700 1 -- 172.21.15.47:0/3445647292 >> v1:172.21.15.47:6789/0 conn(0x7f33e013c750 legacy=0x7f33e00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e4a24700 1 -- 172.21.15.47:0/3445647292 >> v1:172.21.15.18:6789/0 conn(0x7f33e0147a00 legacy=0x7f33e0152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e4a24700 1 -- 172.21.15.47:0/3445647292 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00b4050 con 0x7f33e00ad100 2020-07-18T03:29:39.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33e4a24700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e00ad100 0x7f33e00b1420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f33d000c000 tx=0x7f33d0006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:39.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.494+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d000cae0 con 0x7f33e00ad100 2020-07-18T03:29:39.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.495+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d000cc40 con 0x7f33e00ad100 2020-07-18T03:29:39.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.495+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d00123f0 con 0x7f33e00ad100 2020-07-18T03:29:39.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.495+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e0066a50 con 0x7f33e00ad100 2020-07-18T03:29:39.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.495+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e0066e60 con 0x7f33e00ad100 2020-07-18T03:29:39.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.495+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e00adf10 con 0x7f33e00ad100 2020-07-18T03:29:39.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.496+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f33d00125a0 con 0x7f33e00ad100 2020-07-18T03:29:39.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.496+0000 7f33dcff9700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33b80504b0 0x7f33b8052970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:39.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.496+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f33d0068170 con 0x7f33e00ad100 2020-07-18T03:29:39.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.496+0000 7f33dffff700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33b80504b0 0x7f33b8052970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:39.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.496+0000 7f33dffff700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33b80504b0 0x7f33b8052970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f33d4000fe0 tx=0x7f33d40078a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:39.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.500+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f33d00408a0 con 0x7f33e00ad100 2020-07-18T03:29:39.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.709+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33e013d310 con 0x7f33e00ad100 2020-07-18T03:29:39.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.710+0000 7f33dcff9700 1 -- 172.21.15.47:0/3445647292 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f33d0043ec0 con 0x7f33e00ad100 2020-07-18T03:29:39.711 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:39.711 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33b80504b0 msgr2=0x7f33b8052970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33b80504b0 0x7f33b8052970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f33d4000fe0 tx=0x7f33d40078a0).stop 2020-07-18T03:29:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e00ad100 msgr2=0x7f33e00b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e00ad100 0x7f33e00b1420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f33d000c000 tx=0x7f33d0006dd0).stop 2020-07-18T03:29:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 shutdown_connections 2020-07-18T03:29:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f33b80504b0 0x7f33b8052970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 --2- 172.21.15.47:0/3445647292 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f33e00ad100 0x7f33e00b1420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:39.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 >> 172.21.15.47:0/3445647292 conn(0x7f33e00b9340 msgr2=0x7f33e01469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:39.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 shutdown_connections 2020-07-18T03:29:39.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:39.711+0000 7f33e6c88700 1 -- 172.21.15.47:0/3445647292 wait complete. 2020-07-18T03:29:39.715 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:39.924 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:16.949357+0000', 'modified': '2020-07-18T03:29:17.950573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5530': {'gid': 5530, 'name': 'a', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/1368579801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1368579801}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1368579801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:29:39.925 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:29:39.925 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:29:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":5530,"want_state":"up:active","state":"up:active","rank_uptime":23.103245167000001,"mdsmap_epoch":17,"osdmap_epoch":43,"osdmap_epoch_barrier":42,"uptime":28.484730959} 2020-07-18T03:29:40.064 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 5530, "mdsmap_epoch": 17, "osdmap_epoch": 43, "osdmap_epoch_barrier": 42, "rank_uptime": 23.103245167, "state": "up:active", "uptime": 28.484730959, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:29:40.064 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:29:40.065 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:29:40.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.066 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42611sec preferred_lft 42611sec 2020-07-18T03:29:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:40.084 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:40.084 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:40.084 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:29:40.084 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.085 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::43e:8aff:fe02:cd77/64 scope link 2020-07-18T03:29:40.085 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:14: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:29:40.085 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:29:40.085 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::640f:ddff:fe6f:26c/64 scope link 2020-07-18T03:29:40.086 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.087 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:29:40.150 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:29:40.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.152 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:29:40.207 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:29:40.208 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:29:40.208 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:29:40.208 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:29:40.208 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:29:40.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.209 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:29:40.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.289 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:29:40.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.360 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:29:40.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.433 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:29:40.486 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-07-18T03:29:40.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.487 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:29:40.541 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2020-07-18T03:29:40.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.542 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:29:40.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.615 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-07-18T03:29:40.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.687 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr' 2020-07-18T03:29:40.767 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:40.768 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:40.768 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:40.768 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.769 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:40.769 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.769 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if14: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:40.769 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 12:0d:61:e6:4d:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:29:40.769 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.2/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:29:40.770 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.770 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::100d:61ff:fee6:4d21/64 scope link 2020-07-18T03:29:40.770 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:40.784 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:29:40.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.785 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-07-18T03:29:40.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.857 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:29:40.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:40.955 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:29:41.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:41.012 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:29:41.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:41.083 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:29:41.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:41.136 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:29:41.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:41.183 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:29:41.292 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:29:41.293 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:29:41.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:41.294 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:41.308 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:29:41.380 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:29:41.381 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:29:41.464 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:29:41.465 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:29:41.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:41.466 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:29:41.521 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:29:41.521 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [48] 2020-07-18T03:29:41.522 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:41.564 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:29:41.598 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:29:41.599 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:29:41.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:29:41.663+0000 7fb4064be1c0 -1 init, newargv = 0x559020328dd0 newargc=15 2020-07-18T03:29:41.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[39552]: starting ceph client 2020-07-18T03:29:41.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[39552]: starting fuse 2020-07-18T03:29:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:29:41.672 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:29:41.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:41.680 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:29:41.727 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:29:41.727 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:29:41.728 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:29:41.728 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:29:41.729 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:29:41.730 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:29:41.730 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:29:41.731 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:29:41.731 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:41.732 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:29:41.732 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:29:41.733 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:29:41.733 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:29:41.734 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:41.734 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:29:41.735 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:29:41.735 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:29:41.736 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:29:41.736 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:41.737 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:29:41.737 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:29:41.738 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:29:41.738 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:29:41.739 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:29:41.739 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:29:41.740 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:29:41.741 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:29:41.741 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:41.742 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:29:41.742 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:29:41.859 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.39552.asok 2020-07-18T03:29:42.040 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39552.asok 2020-07-18T03:29:42.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:42.041 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39552.asok status 2020-07-18T03:29:42.192 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:42.193 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:29:42.193 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:29:42.193 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:29:42.193 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:29:42.194 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:29:42.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:29:42.194 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "39552", 2020-07-18T03:29:42.194 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:29:42.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:42.195 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:29:42.195 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:29:42.195 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4928, 2020-07-18T03:29:42.195 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:29:42.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:29:42.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:29:42.196 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4928 2020-07-18T03:29:42.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:42.196 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:42.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:42.196 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:29:42.197 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2758356787 2020-07-18T03:29:42.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:42.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:42.197 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:42.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:42.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:29:42.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2758356787 2020-07-18T03:29:42.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:42.198 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4928 192.168.0.1:0/2758356787", 2020-07-18T03:29:42.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2758356787", 2020-07-18T03:29:42.199 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:29:42.199 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 17, 2020-07-18T03:29:42.199 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 43, 2020-07-18T03:29:42.199 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:29:42.199 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:29:42.200 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:42.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:42.201 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:29:42.202 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:42.218 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:29:42.218 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:29:42.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:42.220 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:42.296 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.450+0000 7f2d12a23700 1 Processor -- start 2020-07-18T03:29:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.450+0000 7f2d12a23700 1 -- start start 2020-07-18T03:29:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d12a23700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d0c0ad100 0x7f2d0c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d12a23700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d0c1443f0 0x7f2d0c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d12a23700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c147330 con 0x7f2d0c147a00 2020-07-18T03:29:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d12a23700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0ae530 con 0x7f2d0c1443f0 2020-07-18T03:29:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d12a23700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b57e0 con 0x7f2d0c0ad100 2020-07-18T03:29:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d10fc0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d0c1443f0 0x7f2d0c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d0b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d0c147a00 0x7f2d0c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42396/0 (socket says 172.21.15.47:42396) 2020-07-18T03:29:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d0b7fe700 1 -- 172.21.15.47:0/2819891567 learned_addr learned my addr 172.21.15.47:0/2819891567 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d0bfff700 1 --2- 172.21.15.47:0/2819891567 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d0c0ad100 0x7f2d0c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d10fc0700 1 -- 172.21.15.47:0/2819891567 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d0c0ad100 msgr2=0x7f2d0c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d10fc0700 1 --2- 172.21.15.47:0/2819891567 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d0c0ad100 0x7f2d0c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.451+0000 7f2d10fc0700 1 -- 172.21.15.47:0/2819891567 >> v1:172.21.15.47:6789/0 conn(0x7f2d0c147a00 legacy=0x7f2d0c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d0affd700 1 -- 172.21.15.47:0/2819891567 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286689178 0 0) 0x7f2d0c147330 con 0x7f2d0c147a00 2020-07-18T03:29:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d10fc0700 1 -- 172.21.15.47:0/2819891567 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0b5a80 con 0x7f2d0c1443f0 2020-07-18T03:29:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d10fc0700 1 --2- 172.21.15.47:0/2819891567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c1443f0 0x7f2d0c147e40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f2d040032b0 tx=0x7f2d040091d0).ready entity=mon.1 client_cookie=38fad58a53c85748 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d0affd700 1 -- 172.21.15.47:0/2819891567 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0400ba50 con 0x7f2d0c1443f0 2020-07-18T03:29:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d0affd700 1 -- 172.21.15.47:0/2819891567 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0400bbb0 con 0x7f2d0c1443f0 2020-07-18T03:29:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d0affd700 1 -- 172.21.15.47:0/2819891567 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0400bd20 con 0x7f2d0c1443f0 2020-07-18T03:29:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 -- 172.21.15.47:0/2819891567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c1443f0 msgr2=0x7f2d0c147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 --2- 172.21.15.47:0/2819891567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c1443f0 0x7f2d0c147e40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f2d040032b0 tx=0x7f2d040091d0).stop 2020-07-18T03:29:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 -- 172.21.15.47:0/2819891567 shutdown_connections 2020-07-18T03:29:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 --2- 172.21.15.47:0/2819891567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c1443f0 0x7f2d0c147e40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 --2- 172.21.15.47:0/2819891567 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2d0c0ad100 0x7f2d0c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 -- 172.21.15.47:0/2819891567 >> 172.21.15.47:0/2819891567 conn(0x7f2d0c0b9340 msgr2=0x7f2d0c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 -- 172.21.15.47:0/2819891567 shutdown_connections 2020-07-18T03:29:42.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.452+0000 7f2d12a23700 1 -- 172.21.15.47:0/2819891567 wait complete. 2020-07-18T03:29:42.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.453+0000 7f2d12a23700 1 Processor -- start 2020-07-18T03:29:42.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d12a23700 1 -- start start 2020-07-18T03:29:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d12a23700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2d0c0b2c40 0x7f2d0c063330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d12a23700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c062950 con 0x7f2d0c147a00 2020-07-18T03:29:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d12a23700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c062bf0 con 0x7f2d0c0ad100 2020-07-18T03:29:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d12a23700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b30f0 con 0x7f2d0c0b2c40 2020-07-18T03:29:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d0bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d0c0ad100 0x7f2d0c0b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42402/0 (socket says 172.21.15.47:42402) 2020-07-18T03:29:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2d0bfff700 1 -- 172.21.15.47:0/3182654319 learned_addr learned my addr 172.21.15.47:0/3182654319 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.454+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794438224 0 0) 0x7f2d0c062bf0 con 0x7f2d0c0ad100 2020-07-18T03:29:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce4003240 con 0x7f2d0c0ad100 2020-07-18T03:29:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d10fc0700 1 --2- 172.21.15.47:0/3182654319 >> v2:172.21.15.18:3300/0 conn(0x7f2d0c0b2c40 0x7f2d0c063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 502553024 0 0) 0x7f2ce4003240 con 0x7f2d0c0ad100 2020-07-18T03:29:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2d0c062bf0 con 0x7f2d0c0ad100 2020-07-18T03:29:42.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f2d00006e60 con 0x7f2d0c0ad100 2020-07-18T03:29:42.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 >> v2:172.21.15.18:3300/0 conn(0x7f2d0c0b2c40 msgr2=0x7f2d0c063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 --2- 172.21.15.47:0/3182654319 >> v2:172.21.15.18:3300/0 conn(0x7f2d0c0b2c40 0x7f2d0c063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 >> v1:172.21.15.47:6789/0 conn(0x7f2d0c0ad100 legacy=0x7f2d0c0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 >> v1:172.21.15.18:6789/0 conn(0x7f2d0c147a00 legacy=0x7f2d0c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce4003800 0x7f2ce4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2ce40044e0 0x7f2ce4004960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ce4004e50 0x7f2ce400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b30f0 con 0x7f2ce4004e50 2020-07-18T03:29:42.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce400bc20 con 0x7f2ce4003800 2020-07-18T03:29:42.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce400bd60 con 0x7f2ce40044e0 2020-07-18T03:29:42.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce4003800 0x7f2ce4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d10fc0700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ce4004e50 0x7f2ce400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0b7fe700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2ce40044e0 0x7f2ce4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 -- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2ce40044e0 msgr2=0x7f2ce4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2ce40044e0 0x7f2ce4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 -- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ce4004e50 msgr2=0x7f2ce400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ce4004e50 0x7f2ce400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c062e90 con 0x7f2ce4003800 2020-07-18T03:29:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d10fc0700 1 -- 172.21.15.47:0/3182654319 reap_dead start 2020-07-18T03:29:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1429d0 con 0x7f2ce4003800 2020-07-18T03:29:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c0634f0 con 0x7f2ce4003800 2020-07-18T03:29:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.455+0000 7f2d0bfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce4003800 0x7f2ce4003c20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f2d00002f30 tx=0x7f2d00001890).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.456+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0000cc60 con 0x7f2ce4003800 2020-07-18T03:29:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.456+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000cdc0 con 0x7f2ce4003800 2020-07-18T03:29:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.456+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d00006210 con 0x7f2ce4003800 2020-07-18T03:29:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.456+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c142b60 con 0x7f2ce4003800 2020-07-18T03:29:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.457+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2d00006370 con 0x7f2ce4003800 2020-07-18T03:29:42.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.457+0000 7f2cebfff700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2ce4053ec0 0x7f2ce4056380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:42.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.457+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2d00067eb0 con 0x7f2ce4003800 2020-07-18T03:29:42.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.457+0000 7f2d0b7fe700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2ce4053ec0 0x7f2ce4056380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:42.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.457+0000 7f2d0b7fe700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2ce4053ec0 0x7f2ce4056380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f2cfc00d000 tx=0x7f2cfc009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:42.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.460+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2d0c142b60 con 0x7f2ce4003800 2020-07-18T03:29:42.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.666+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d0c142b60 con 0x7f2ce4003800 2020-07-18T03:29:42.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.667+0000 7f2cebfff700 1 -- 172.21.15.47:0/3182654319 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f2d0c142b60 con 0x7f2ce4003800 2020-07-18T03:29:42.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:42.669 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2ce4053ec0 msgr2=0x7f2ce4056380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2ce4053ec0 0x7f2ce4056380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f2cfc00d000 tx=0x7f2cfc009040).stop 2020-07-18T03:29:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce4003800 msgr2=0x7f2ce4003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce4003800 0x7f2ce4003c20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f2d00002f30 tx=0x7f2d00001890).stop 2020-07-18T03:29:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 shutdown_connections 2020-07-18T03:29:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2ce4053ec0 0x7f2ce4056380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 --2- 172.21.15.47:0/3182654319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce4003800 0x7f2ce4003c20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.668+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 >> 172.21.15.47:0/3182654319 conn(0x7f2d0c0b9340 msgr2=0x7f2d0c0b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.669+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 shutdown_connections 2020-07-18T03:29:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:42.669+0000 7f2d12a23700 1 -- 172.21.15.47:0/3182654319 wait complete. 2020-07-18T03:29:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:42.682 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2020-07-18T03:29:42.818 INFO:teuthology.orchestra.run.smithi047.stdout:[{"id":4928,"entity":{"name":{"type":"client","num":4928},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2758356787}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":1,"uptime":1.1439760809999999,"requests_in_flight":1,"completed_requests":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99736956146669797,"halflife":300},"inst":"client.4928 192.168.0.1:0/2758356787","completed_requests":[],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"0","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"39552","root":"/"}},{"id":5776,"entity":{"name":{"type":"client","num":5776},"addr":{"type":"any","addr":"192.168.0.2:0","nonce":1096649958}},"state":"open","num_leases":3,"num_caps":4,"request_load_avg":5,"uptime":13.052726792,"requests_in_flight":0,"completed_requests":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":10.84331623862683,"halflife":300},"inst":"client.5776 192.168.0.2:0/1096649958","completed_requests":[],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"1","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"38415","root":"/"}}] 2020-07-18T03:29:42.819 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "0", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "39552", "root": "/" }, "completed_requests": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2758356787, "type": "any" }, "name": { "num": 4928, "type": "client" } }, "id": 4928, "inst": "client.4928 192.168.0.1:0/2758356787", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 1, "session_cache_liveness": { "halflife": 300, "value": 0.997369561466698 }, "state": "open", "uptime": 1.143976081, "used_inos": [] }, { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "1", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "38415", "root": "/" }, "completed_requests": [], "entity": { "addr": { "addr": "192.168.0.2:0", "nonce": 1096649958, "type": "any" }, "name": { "num": 5776, "type": "client" } }, "id": 5776, "inst": "client.5776 192.168.0.2:0/1096649958", "num_caps": 4, "num_leases": 3, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 10.84331623862683 }, "state": "open", "uptime": 13.052726792, "used_inos": [] } ] 2020-07-18T03:29:42.820 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:29:42.821 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:29:42.821 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:29:42.822 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:29:42.822 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:29:42.823 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:42.823 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:29:42.824 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:29:42.825 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:29:42.825 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:29:42.825 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:42.826 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:29:42.826 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:29:42.827 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:29:42.829 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:29:42.829 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:42.831 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:29:42.831 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:29:42.832 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:29:42.833 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:29:42.833 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:29:42.834 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:29:42.834 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:29:42.836 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:29:42.836 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:42.836 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:29:42.837 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:29:42.930 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.39552.asok 2020-07-18T03:29:43.110 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39552.asok 2020-07-18T03:29:43.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:43.111 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39552.asok mds_sessions 2020-07-18T03:29:43.261 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:43.262 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4928, 2020-07-18T03:29:43.262 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:29:43.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:29:43.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:29:43.263 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4928 2020-07-18T03:29:43.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:43.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:43.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:29:43.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2758356787 2020-07-18T03:29:43.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:43.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.264 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4928 v1:192.168.0.1:0/2758356787", 2020-07-18T03:29:43.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "v1:192.168.0.1:0/2758356787", 2020-07-18T03:29:43.265 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions": [ 2020-07-18T03:29:43.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:43.266 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": 0, 2020-07-18T03:29:43.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addrs": { 2020-07-18T03:29:43.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:43.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:43.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:43.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6834", 2020-07-18T03:29:43.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1368579801 2020-07-18T03:29:43.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:43.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:43.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6835", 2020-07-18T03:29:43.268 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1368579801 2020-07-18T03:29:43.268 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:43.268 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:43.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.269 INFO:teuthology.orchestra.run.smithi047.stdout: "seq": 0, 2020-07-18T03:29:43.269 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_gen": 0, 2020-07-18T03:29:43.270 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_ttl": "2020-07-18T03:30:42.665954+0000", 2020-07-18T03:29:43.270 INFO:teuthology.orchestra.run.smithi047.stdout: "last_cap_renew_request": "2020-07-18T03:29:42.665954+0000", 2020-07-18T03:29:43.270 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_renew_seq": 2, 2020-07-18T03:29:43.271 INFO:teuthology.orchestra.run.smithi047.stdout: "num_caps": 1, 2020-07-18T03:29:43.271 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "open" 2020-07-18T03:29:43.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:43.271 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:43.271 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17 2020-07-18T03:29:43.272 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:43.273 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:29:43.273 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:29:43.274 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:29:43.274 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:29:43.275 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:29:43.276 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:43.277 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:29:43.278 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:29:43.278 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:29:43.279 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:29:43.279 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:43.280 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:29:43.280 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:29:43.281 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:29:43.281 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:29:43.282 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:43.282 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:29:43.282 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:29:43.283 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:29:43.283 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:29:43.284 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:29:43.284 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:29:43.285 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:29:43.285 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:29:43.286 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:29:43.287 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:29:43.287 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:29:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.38415.asok 2020-07-18T03:29:43.560 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38415.asok 2020-07-18T03:29:43.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:29:43.561 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38415.asok mds_sessions 2020-07-18T03:29:43.712 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:43.712 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5776, 2020-07-18T03:29:43.712 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:29:43.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:29:43.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:29:43.713 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5776 2020-07-18T03:29:43.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.713 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:29:43.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:43.714 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:29:43.714 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1096649958 2020-07-18T03:29:43.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:43.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.721 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5776 v1:192.168.0.2:0/1096649958", 2020-07-18T03:29:43.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "v1:192.168.0.2:0/1096649958", 2020-07-18T03:29:43.721 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions": [ 2020-07-18T03:29:43.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:43.722 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": 0, 2020-07-18T03:29:43.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addrs": { 2020-07-18T03:29:43.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:43.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:43.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:43.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6834", 2020-07-18T03:29:43.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1368579801 2020-07-18T03:29:43.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:43.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:43.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6835", 2020-07-18T03:29:43.724 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1368579801 2020-07-18T03:29:43.724 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:43.724 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:43.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:43.724 INFO:teuthology.orchestra.run.smithi047.stdout: "seq": 2, 2020-07-18T03:29:43.725 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_gen": 0, 2020-07-18T03:29:43.725 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_ttl": "2020-07-18T03:30:30.757080+0000", 2020-07-18T03:29:43.725 INFO:teuthology.orchestra.run.smithi047.stdout: "last_cap_renew_request": "2020-07-18T03:29:30.757080+0000", 2020-07-18T03:29:43.725 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_renew_seq": 2, 2020-07-18T03:29:43.725 INFO:teuthology.orchestra.run.smithi047.stdout: "num_caps": 4, 2020-07-18T03:29:43.725 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "open" 2020-07-18T03:29:43.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:43.726 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:43.726 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17 2020-07-18T03:29:43.726 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:43.728 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:29:43.728 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:43.789 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:43.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:29:43.846 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:29:43.847 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:29:43.847 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:43.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[39552]: fuse finished with error 0 and tester_r 0 2020-07-18T03:29:43.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:43.889 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:43.944 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:29:43.944 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:29:43.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:43.945 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:29:44.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.023 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:29:44.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.174 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:29:44.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.217 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:44.270 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:44.271 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:44.271 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:44.271 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:44.271 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:44.272 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:44.272 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:44.272 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.272 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:44.272 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.273 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:44.273 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42607sec preferred_lft 42607sec 2020-07-18T03:29:44.273 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:44.273 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:44.273 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:44.274 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.274 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:44.274 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.274 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:44.274 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.275 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:29:44.275 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:44.275 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::43e:8aff:fe02:cd77/64 scope link 2020-07-18T03:29:44.275 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:44.275 INFO:teuthology.orchestra.run.smithi047.stdout:14: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:29:44.275 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:29:44.276 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::640f:ddff:fe6f:26c/64 scope link 2020-07-18T03:29:44.276 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:44.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.277 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:29:44.341 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.342 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:44.342 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.342 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.342 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.343 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.343 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.343 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.344 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:44.344 INFO:teuthology.orchestra.run.smithi047.stdout:14: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:44.344 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 66:0f:dd:6f:02:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:29:44.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.346 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:29:44.400 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:29:44.401 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:29:44.402 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:44.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[38415]: fuse finished with error 0 and tester_r 0 2020-07-18T03:29:44.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.644 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:44.717 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:29:44.717 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:29:44.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.718 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:29:44.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.803 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.1' 2020-07-18T03:29:44.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.901 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:29:44.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:44.949 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:29:45.005 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:45.005 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.006 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.007 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42606sec preferred_lft 42606sec 2020-07-18T03:29:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:29:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.008 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.008 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.009 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:29:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:29:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.010 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::43e:8aff:fe02:cd77/64 scope link 2020-07-18T03:29:45.011 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.012 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:29:45.076 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:45.077 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:45.077 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:45.077 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.077 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:29:45.078 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.078 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:45.078 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.078 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:45.078 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.079 INFO:teuthology.orchestra.run.smithi047.stdout:12: ceph-brx: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:29:45.079 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.080 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:29:45.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.081 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:29:45.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.155 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:29:45.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.246 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:29:45.262 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:29:45.263 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:29:45.263 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:29:45.263 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:29:45.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.264 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:29:45.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.345 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:29:45.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.427 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:29:45.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.508 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:29:45.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.567 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:29:45.585 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-07-18T03:29:45.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:29:45.586 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-07-18T03:29:45.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.587 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:29:45.643 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:45.643 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:45.644 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:45.644 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.644 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:45.644 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.644 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.645 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.645 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:45.645 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.645 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:45.645 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42605sec preferred_lft 42605sec 2020-07-18T03:29:45.646 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:29:45.646 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.646 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.646 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.646 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.646 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.648 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:29:45.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.701 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:29:45.758 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2020-07-18T03:29:45.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:29:45.759 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-07-18T03:29:45.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.760 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:29:45.816 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:29:45.816 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:29:45.816 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:29:45.817 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.817 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:29:45.817 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.817 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.817 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.818 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:29:45.818 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.818 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:29:45.818 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42605sec preferred_lft 42605sec 2020-07-18T03:29:45.818 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:29:45.819 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:29:45.819 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.819 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.819 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:29:45.819 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:29:45.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:29:45.820 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:29:45.873 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic' 2020-07-18T03:29:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.989+0000 7fbf78c81700 1 Processor -- start 2020-07-18T03:29:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.989+0000 7fbf78c81700 1 -- start start 2020-07-18T03:29:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf78c81700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbf740ad100 0x7fbf740b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf78c81700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf74147330 con 0x7fbf74147a00 2020-07-18T03:29:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf78c81700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf740603c0 con 0x7fbf741443f0 2020-07-18T03:29:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf78c81700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf740b57e0 con 0x7fbf740ad100 2020-07-18T03:29:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf72d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbf741443f0 0x7fbf7405fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42420/0 (socket says 172.21.15.47:42420) 2020-07-18T03:29:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf72d9d700 1 -- 172.21.15.47:0/2823625293 learned_addr learned my addr 172.21.15.47:0/2823625293 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf7259c700 1 --2- 172.21.15.47:0/2823625293 >> v2:172.21.15.18:3300/0 conn(0x7fbf740ad100 0x7fbf740b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018117638 0 0) 0x7fbf740603c0 con 0x7fbf741443f0 2020-07-18T03:29:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.990+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf58003240 con 0x7fbf741443f0 2020-07-18T03:29:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 745453093 0 0) 0x7fbf58003240 con 0x7fbf741443f0 2020-07-18T03:29:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 >> v2:172.21.15.18:3300/0 conn(0x7fbf740ad100 msgr2=0x7fbf740b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 --2- 172.21.15.47:0/2823625293 >> v2:172.21.15.18:3300/0 conn(0x7fbf740ad100 0x7fbf740b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 >> v1:172.21.15.18:6789/0 conn(0x7fbf74147a00 legacy=0x7fbf7405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740b5a80 con 0x7fbf741443f0 2020-07-18T03:29:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fbf6c001a50 con 0x7fbf741443f0 2020-07-18T03:29:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 >> v1:172.21.15.47:6789/0 conn(0x7fbf741443f0 legacy=0x7fbf7405fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf58003780 0x7fbf58003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf580044e0 0x7fbf58004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf58004e50 0x7fbf5800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740b57e0 con 0x7fbf58003780 2020-07-18T03:29:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5800bc20 con 0x7fbf580044e0 2020-07-18T03:29:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5800bd60 con 0x7fbf58004e50 2020-07-18T03:29:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf580044e0 0x7fbf58004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7259c700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf58003780 0x7fbf58003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf72d9d700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf58004e50 0x7fbf5800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf58004e50 msgr2=0x7fbf5800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf58004e50 0x7fbf5800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf58003780 msgr2=0x7fbf58003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf58003780 0x7fbf58003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2823625293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58003240 con 0x7fbf580044e0 2020-07-18T03:29:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf72d9d700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbf58004e50 0x7fbf5800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf72d9d700 1 -- 172.21.15.47:0/2823625293 reap_dead start 2020-07-18T03:29:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf71d9b700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf580044e0 0x7fbf58004960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fbf7405ca30 tx=0x7fbf64007b70).ready entity=mon.1 client_cookie=6ce792c0fb0ae6be server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.991+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6400c660 con 0x7fbf580044e0 2020-07-18T03:29:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6400c7c0 con 0x7fbf580044e0 2020-07-18T03:29:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf7159a700 1 -- 172.21.15.47:0/2823625293 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf6400cb20 con 0x7fbf580044e0 2020-07-18T03:29:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 -- 172.21.15.47:0/2823625293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf580044e0 msgr2=0x7fbf58004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf580044e0 0x7fbf58004960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fbf7405ca30 tx=0x7fbf64007b70).stop 2020-07-18T03:29:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 -- 172.21.15.47:0/2823625293 shutdown_connections 2020-07-18T03:29:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 --2- 172.21.15.47:0/2823625293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf580044e0 0x7fbf58004960 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 -- 172.21.15.47:0/2823625293 >> 172.21.15.47:0/2823625293 conn(0x7fbf740b9340 msgr2=0x7fbf740b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 -- 172.21.15.47:0/2823625293 shutdown_connections 2020-07-18T03:29:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.992+0000 7fbf78c81700 1 -- 172.21.15.47:0/2823625293 wait complete. 2020-07-18T03:29:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.993+0000 7fbf78c81700 1 Processor -- start 2020-07-18T03:29:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.993+0000 7fbf78c81700 1 -- start start 2020-07-18T03:29:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf78c81700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf74066480 0x7fbf74141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf78c81700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf741588e0 con 0x7fbf740b1e40 2020-07-18T03:29:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf78c81700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf740b2360 con 0x7fbf740660a0 2020-07-18T03:29:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf78c81700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf74159490 con 0x7fbf74066480 2020-07-18T03:29:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf74066480 0x7fbf74141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf74066480 0x7fbf74141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33110/0 (socket says 172.21.15.47:33110) 2020-07-18T03:29:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf7259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbf740660a0 0x7fbf74158d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42428/0 (socket says 172.21.15.47:42428) 2020-07-18T03:29:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2062514686 learned_addr learned my addr 172.21.15.47:0/2062514686 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2062514686 >> v1:172.21.15.47:6789/0 conn(0x7fbf740660a0 legacy=0x7fbf74158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2062514686 >> v1:172.21.15.18:6789/0 conn(0x7fbf740b1e40 legacy=0x7fbf74155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 -- 172.21.15.47:0/2062514686 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf741124b0 con 0x7fbf74066480 2020-07-18T03:29:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990113092 0 0) 0x7fbf740b2360 con 0x7fbf740660a0 2020-07-18T03:29:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf71d9b700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74066480 0x7fbf74141fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fbf74143900 tx=0x7fbf64016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.994+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf64007240 con 0x7fbf74066480 2020-07-18T03:29:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.995+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf640073a0 con 0x7fbf74066480 2020-07-18T03:29:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.995+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf64012c80 con 0x7fbf74066480 2020-07-18T03:29:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.995+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf74155720 con 0x7fbf74066480 2020-07-18T03:29:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.995+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf740b3020 con 0x7fbf74066480 2020-07-18T03:29:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.995+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf74112640 con 0x7fbf74066480 2020-07-18T03:29:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.996+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbf64025030 con 0x7fbf74066480 2020-07-18T03:29:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.996+0000 7fbf627fc700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4804fe20 0x7fbf480522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.996+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbf64068e20 con 0x7fbf74066480 2020-07-18T03:29:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.996+0000 7fbf7259c700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4804fe20 0x7fbf480522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.996+0000 7fbf7259c700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4804fe20 0x7fbf480522e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fbf68007000 tx=0x7fbf68008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:45.999+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbf640415b0 con 0x7fbf74066480 2020-07-18T03:29:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.218+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]} v 0) v1 -- 0x7fbf74158730 con 0x7fbf74066480 2020-07-18T03:29:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.236+0000 7fbf627fc700 1 -- 172.21.15.47:0/2062514686 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]}]=0 v183) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbf64023020 con 0x7fbf74066480 2020-07-18T03:29:46.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.237+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4804fe20 msgr2=0x7fbf480522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.237+0000 7fbf78c81700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4804fe20 0x7fbf480522e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fbf68007000 tx=0x7fbf68008040).stop 2020-07-18T03:29:46.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.237+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74066480 msgr2=0x7fbf74141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.237+0000 7fbf78c81700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74066480 0x7fbf74141fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fbf74143900 tx=0x7fbf64016000).stop 2020-07-18T03:29:46.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.237+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 shutdown_connections 2020-07-18T03:29:46.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.238+0000 7fbf78c81700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf4804fe20 0x7fbf480522e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.238+0000 7fbf78c81700 1 --2- 172.21.15.47:0/2062514686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf74066480 0x7fbf74141fc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.238+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 >> 172.21.15.47:0/2062514686 conn(0x7fbf740b9340 msgr2=0x7fbf740b8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:46.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.238+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 shutdown_connections 2020-07-18T03:29:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.238+0000 7fbf78c81700 1 -- 172.21.15.47:0/2062514686 wait complete. 2020-07-18T03:29:46.249 INFO:tasks.cephfs_test_runner:test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2020-07-18T03:29:46.250 INFO:tasks.cephfs_test_runner:Starting test: test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) 2020-07-18T03:29:46.250 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout' 2020-07-18T03:29:46.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.364+0000 7ff5ab50f700 1 Processor -- start 2020-07-18T03:29:46.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.364+0000 7ff5ab50f700 1 -- start start 2020-07-18T03:29:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5ab50f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a40ad100 0x7ff5a40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5ab50f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5a41443f0 0x7ff5a4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5ab50f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a4147330 con 0x7ff5a4147a00 2020-07-18T03:29:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5ab50f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a40ae530 con 0x7ff5a41443f0 2020-07-18T03:29:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5ab50f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40b57e0 con 0x7ff5a40ad100 2020-07-18T03:29:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5a41443f0 0x7ff5a4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5a41443f0 0x7ff5a4147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33118/0 (socket says 172.21.15.47:33118) 2020-07-18T03:29:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 -- 172.21.15.47:0/4142039722 learned_addr learned my addr 172.21.15.47:0/4142039722 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a92ab700 1 --2- 172.21.15.47:0/4142039722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a40ad100 0x7ff5a40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 -- 172.21.15.47:0/4142039722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a40ad100 msgr2=0x7ff5a40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/4142039722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a40ad100 0x7ff5a40b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 -- 172.21.15.47:0/4142039722 >> v1:172.21.15.18:6789/0 conn(0x7ff5a4147a00 legacy=0x7ff5a405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 -- 172.21.15.47:0/4142039722 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40b5a80 con 0x7ff5a41443f0 2020-07-18T03:29:46.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.365+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/4142039722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a41443f0 0x7ff5a4147e40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7ff59c0032b0 tx=0x7ff59c009270).ready entity=mon.1 client_cookie=d0149cf9b24c728c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:46.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff593fff700 1 -- 172.21.15.47:0/4142039722 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59c00bc00 con 0x7ff5a41443f0 2020-07-18T03:29:46.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff593fff700 1 -- 172.21.15.47:0/4142039722 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c00bd60 con 0x7ff5a41443f0 2020-07-18T03:29:46.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff593fff700 1 -- 172.21.15.47:0/4142039722 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59c011400 con 0x7ff5a41443f0 2020-07-18T03:29:46.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 -- 172.21.15.47:0/4142039722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a41443f0 msgr2=0x7ff5a4147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 --2- 172.21.15.47:0/4142039722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a41443f0 0x7ff5a4147e40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7ff59c0032b0 tx=0x7ff59c009270).stop 2020-07-18T03:29:46.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 -- 172.21.15.47:0/4142039722 shutdown_connections 2020-07-18T03:29:46.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 --2- 172.21.15.47:0/4142039722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a41443f0 0x7ff5a4147e40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 --2- 172.21.15.47:0/4142039722 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a40ad100 0x7ff5a40b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 -- 172.21.15.47:0/4142039722 >> 172.21.15.47:0/4142039722 conn(0x7ff5a40b9340 msgr2=0x7ff5a40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:46.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 -- 172.21.15.47:0/4142039722 shutdown_connections 2020-07-18T03:29:46.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.366+0000 7ff5ab50f700 1 -- 172.21.15.47:0/4142039722 wait complete. 2020-07-18T03:29:46.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.367+0000 7ff5ab50f700 1 Processor -- start 2020-07-18T03:29:46.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.367+0000 7ff5ab50f700 1 -- start start 2020-07-18T03:29:46.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5ab50f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a4113cb0 0x7ff5a40b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5ab50f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a4156570 con 0x7ff5a4147a00 2020-07-18T03:29:46.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5ab50f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a4159200 con 0x7ff5a40ad100 2020-07-18T03:29:46.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5ab50f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41594a0 con 0x7ff5a4113cb0 2020-07-18T03:29:46.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5a92ab700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5a40ad100 0x7ff5a4066070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42442/0 (socket says 172.21.15.47:42442) 2020-07-18T03:29:46.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5a92ab700 1 -- 172.21.15.47:0/895787593 learned_addr learned my addr 172.21.15.47:0/895787593 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:46.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.368+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436874391 0 0) 0x7ff5a4159200 con 0x7ff5a40ad100 2020-07-18T03:29:46.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff57c003240 con 0x7ff5a40ad100 2020-07-18T03:29:46.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a4113cb0 0x7ff5a40b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2043640565 0 0) 0x7ff57c003240 con 0x7ff5a40ad100 2020-07-18T03:29:46.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5a4159200 con 0x7ff5a40ad100 2020-07-18T03:29:46.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7ff594006e60 con 0x7ff5a40ad100 2020-07-18T03:29:46.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a4113cb0 msgr2=0x7ff5a40b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff5a4113cb0 0x7ff5a40b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 >> v1:172.21.15.47:6789/0 conn(0x7ff5a40ad100 legacy=0x7ff5a4066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 >> v1:172.21.15.18:6789/0 conn(0x7ff5a4147a00 legacy=0x7ff5a4114560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c003800 0x7ff57c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff57c0044e0 0x7ff57c004900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57c004df0 0x7ff57c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41594a0 con 0x7ff57c003800 2020-07-18T03:29:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c004160 con 0x7ff57c004df0 2020-07-18T03:29:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c004320 con 0x7ff57c0044e0 2020-07-18T03:29:46.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57c004df0 0x7ff57c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a92ab700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c003800 0x7ff57c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a8aaa700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff57c0044e0 0x7ff57c004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 -- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff57c0044e0 msgr2=0x7ff57c004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff57c0044e0 0x7ff57c004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 -- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c003800 msgr2=0x7ff57c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:46.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c003800 0x7ff57c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:46.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5a9aac700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4159740 con 0x7ff57c004df0 2020-07-18T03:29:46.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a4114c70 con 0x7ff57c004df0 2020-07-18T03:29:46.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.369+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a40b2fc0 con 0x7ff57c004df0 2020-07-18T03:29:46.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.370+0000 7ff5a9aac700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57c004df0 0x7ff57c00b730 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7ff59c01b000 tx=0x7ff59c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.370+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59c010820 con 0x7ff57c004df0 2020-07-18T03:29:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.370+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c010980 con 0x7ff57c004df0 2020-07-18T03:29:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.370+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59c010ce0 con 0x7ff57c004df0 2020-07-18T03:29:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.370+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a40aec80 con 0x7ff57c004df0 2020-07-18T03:29:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.371+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff59c00f070 con 0x7ff57c004df0 2020-07-18T03:29:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.371+0000 7ff5917fa700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff57c058670 0x7ff57c05ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.371+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff59c069c60 con 0x7ff57c004df0 2020-07-18T03:29:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.371+0000 7ff5a92ab700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff57c058670 0x7ff57c05ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.372+0000 7ff5a92ab700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff57c058670 0x7ff57c05ab30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7ff59400f000 tx=0x7ff594008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.374+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff59c045d80 con 0x7ff57c004df0 2020-07-18T03:29:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:46.590+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]} v 0) v1 -- 0x7ff5a4057450 con 0x7ff57c004df0 2020-07-18T03:29:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.239+0000 7ff5917fa700 1 -- 172.21.15.47:0/895787593 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]}]=0 v184) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff59c0270e0 con 0x7ff57c004df0 2020-07-18T03:29:47.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff57c058670 msgr2=0x7ff57c05ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff57c058670 0x7ff57c05ab30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7ff59400f000 tx=0x7ff594008040).stop 2020-07-18T03:29:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57c004df0 msgr2=0x7ff57c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 --2- 172.21.15.47:0/895787593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57c004df0 0x7ff57c00b730 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7ff59c01b000 tx=0x7ff59c016000).stop 2020-07-18T03:29:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5a9aac700 1 -- 172.21.15.47:0/895787593 reap_dead start 2020-07-18T03:29:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5a9aac700 1 -- 172.21.15.47:0/895787593 reap_dead start 2020-07-18T03:29:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 shutdown_connections 2020-07-18T03:29:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 >> 172.21.15.47:0/895787593 conn(0x7ff5a40b9340 msgr2=0x7ff5a405f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 shutdown_connections 2020-07-18T03:29:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.241+0000 7ff5ab50f700 1 -- 172.21.15.47:0/895787593 wait complete. 2020-07-18T03:29:47.255 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:29:47.256 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:47.313 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:29:47.369 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:29:47.370 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-07-18T03:29:47.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.490+0000 7f29523fa700 1 Processor -- start 2020-07-18T03:29:47.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.490+0000 7f29523fa700 1 -- start start 2020-07-18T03:29:47.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29523fa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f294c0ade90 0x7f294c142f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:47.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29523fa700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c143400 con 0x7f294c0b04c0 2020-07-18T03:29:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29523fa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c147930 con 0x7f294c0b0950 2020-07-18T03:29:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29523fa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0afbd0 con 0x7f294c0ade90 2020-07-18T03:29:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29513f8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f294c0ade90 0x7f294c142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f2951bf9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f294c0b0950 0x7f294c147220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42460/0 (socket says 172.21.15.47:42460) 2020-07-18T03:29:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f2951bf9700 1 -- 172.21.15.47:0/4102960330 learned_addr learned my addr 172.21.15.47:0/4102960330 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29513f8700 1 -- 172.21.15.47:0/4102960330 >> v1:172.21.15.47:6789/0 conn(0x7f294c0b0950 legacy=0x7f294c147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29513f8700 1 -- 172.21.15.47:0/4102960330 >> v1:172.21.15.18:6789/0 conn(0x7f294c0b04c0 legacy=0x7f294c146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29513f8700 1 -- 172.21.15.47:0/4102960330 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0afe70 con 0x7f294c0ade90 2020-07-18T03:29:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29513f8700 1 --2- 172.21.15.47:0/4102960330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0ade90 0x7f294c142f10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f294c0b0110 tx=0x7f29440091d0).ready entity=mon.1 client_cookie=e5715387d0843927 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f294bfff700 1 -- 172.21.15.47:0/4102960330 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2944009e80 con 0x7f294c0ade90 2020-07-18T03:29:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f294bfff700 1 -- 172.21.15.47:0/4102960330 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294400bbc0 con 0x7f294c0ade90 2020-07-18T03:29:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f294bfff700 1 -- 172.21.15.47:0/4102960330 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f294400bd30 con 0x7f294c0ade90 2020-07-18T03:29:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29523fa700 1 -- 172.21.15.47:0/4102960330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0ade90 msgr2=0x7f294c142f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.491+0000 7f29523fa700 1 --2- 172.21.15.47:0/4102960330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0ade90 0x7f294c142f10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f294c0b0110 tx=0x7f29440091d0).stop 2020-07-18T03:29:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.492+0000 7f29523fa700 1 -- 172.21.15.47:0/4102960330 shutdown_connections 2020-07-18T03:29:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.492+0000 7f29523fa700 1 --2- 172.21.15.47:0/4102960330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0ade90 0x7f294c142f10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.492+0000 7f29523fa700 1 -- 172.21.15.47:0/4102960330 >> 172.21.15.47:0/4102960330 conn(0x7f294c0b9e50 msgr2=0x7f294c0b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.492+0000 7f29523fa700 1 -- 172.21.15.47:0/4102960330 shutdown_connections 2020-07-18T03:29:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.492+0000 7f29523fa700 1 -- 172.21.15.47:0/4102960330 wait complete. 2020-07-18T03:29:47.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.493+0000 7f29523fa700 1 Processor -- start 2020-07-18T03:29:47.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.493+0000 7f29523fa700 1 -- start start 2020-07-18T03:29:47.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f29523fa700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f294c0b04c0 0x7f294c136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:47.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f29523fa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f294c0b0950 0x7f294c13a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:47.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f29523fa700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c136940 con 0x7f294c146390 2020-07-18T03:29:47.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f29523fa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c139b50 con 0x7f294c0b0950 2020-07-18T03:29:47.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f29523fa700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c13ae00 con 0x7f294c0b04c0 2020-07-18T03:29:47.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f294c0b0950 0x7f294c13a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:47.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f294c0b0950 0x7f294c13a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33144/0 (socket says 172.21.15.47:33144) 2020-07-18T03:29:47.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 -- 172.21.15.47:0/1651436448 learned_addr learned my addr 172.21.15.47:0/1651436448 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:47.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 -- 172.21.15.47:0/1651436448 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f294c0b04c0 msgr2=0x7f294c136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f29513f8700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f294c0b04c0 0x7f294c136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:47.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f294c0b04c0 0x7f294c136080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:47.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 -- 172.21.15.47:0/1651436448 >> v1:172.21.15.18:6789/0 conn(0x7f294c146390 legacy=0x7f294c1383a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.494+0000 7f2950bf7700 1 -- 172.21.15.47:0/1651436448 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c137cf0 con 0x7f294c0b0950 2020-07-18T03:29:47.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.495+0000 7f2950bf7700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0b0950 0x7f294c13a910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f294c136e00 tx=0x7f2940007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:47.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.495+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2940007030 con 0x7f294c0b0950 2020-07-18T03:29:47.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.495+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940007190 con 0x7f294c0b0950 2020-07-18T03:29:47.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.495+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c136ed0 con 0x7f294c0b0950 2020-07-18T03:29:47.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.495+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29400064f0 con 0x7f294c0b0950 2020-07-18T03:29:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.495+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c138c70 con 0x7f294c0b0950 2020-07-18T03:29:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.496+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c1399b0 con 0x7f294c0b0950 2020-07-18T03:29:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.496+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2940010070 con 0x7f294c0b0950 2020-07-18T03:29:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.496+0000 7f29497fa700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2924050060 0x7f2924052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.496+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2940066fb0 con 0x7f294c0b0950 2020-07-18T03:29:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.496+0000 7f29513f8700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2924050060 0x7f2924052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.496+0000 7f29513f8700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2924050060 0x7f2924052520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f2944017000 tx=0x7f2944016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.499+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f294c1399b0 con 0x7f294c0b0950 2020-07-18T03:29:47.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.658+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f294c1399b0 con 0x7f294c0b0950 2020-07-18T03:29:47.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.659+0000 7f29497fa700 1 -- 172.21.15.47:0/1651436448 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v43) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f2940028510 con 0x7f294c0b0950 2020-07-18T03:29:47.661 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:47.662 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:47.662 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 43, 2020-07-18T03:29:47.662 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", 2020-07-18T03:29:47.662 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2020-07-18T03:25:24.387359+0000", 2020-07-18T03:29:47.663 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2020-07-18T03:29:17.951029+0000", 2020-07-18T03:29:47.663 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:29:47.663 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:29:47.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-07-18T03:29:47.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2020-07-18T03:29:47.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2020-07-18T03:29:47.664 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2020-07-18T03:29:47.664 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2020-07-18T03:29:47.664 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2020-07-18T03:29:47.664 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2020-07-18T03:29:47.665 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:47.665 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2020-07-18T03:29:47.665 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2020-07-18T03:29:47.665 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-07-18T03:29:47.665 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2020-07-18T03:29:47.666 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2020-07-18T03:29:47.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 8, 2020-07-18T03:29:47.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2020-07-18T03:29:47.666 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "luminous", 2020-07-18T03:29:47.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2020-07-18T03:29:47.666 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "pacific", 2020-07-18T03:29:47.667 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2020-07-18T03:29:47.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.667 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2020-07-18T03:29:47.667 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2020-07-18T03:29:47.667 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:27.725131+0000", 2020-07-18T03:29:47.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:47.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:47.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:47.668 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:47.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:47.669 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:47.669 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:47.669 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:47.669 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2020-07-18T03:29:47.669 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2020-07-18T03:29:47.670 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2020-07-18T03:29:47.670 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2020-07-18T03:29:47.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2020-07-18T03:29:47.671 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:47.671 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:47.671 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:47.671 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:47.672 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:47.672 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:47.672 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:47.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.672 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "15", 2020-07-18T03:29:47.672 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:47.673 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:47.673 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:47.673 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:47.673 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:47.673 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:47.674 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:47.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:47.674 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:47.674 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:47.674 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:47.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:47.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:47.675 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:47.675 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:47.675 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:47.676 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:47.676 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:47.676 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:47.676 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:47.676 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:47.677 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:47.677 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:47.677 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:47.677 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:47.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:47.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.678 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:47.678 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:47.678 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:47.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:47.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:47.679 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:47.679 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:47.679 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:47.679 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:47.679 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:47.680 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:47.680 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:29:47.680 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2020-07-18T03:29:47.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.683 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:47.683 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2020-07-18T03:29:47.684 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.684 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2020-07-18T03:29:47.684 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2020-07-18T03:29:47.685 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:37.896092+0000", 2020-07-18T03:29:47.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:47.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:47.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:47.685 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:47.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:47.686 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:47.686 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:47.686 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:47.686 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:29:47.686 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:29:47.687 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:29:47.687 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:29:47.687 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:29:47.687 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:47.687 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:47.688 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:47.688 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:47.688 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:47.688 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:47.688 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:47.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.689 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "18", 2020-07-18T03:29:47.689 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:47.689 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:47.689 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:47.689 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:47.690 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:47.690 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:47.690 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:47.690 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:47.690 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:47.691 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:47.691 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:47.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:47.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:47.691 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:47.691 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:47.692 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:47.692 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:47.692 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:47.692 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:47.692 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:47.693 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:47.710 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:47.710 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:47.710 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:47.710 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:47.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:47.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.711 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:47.711 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:47.711 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:29:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2020-07-18T03:29:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 7, 2020-07-18T03:29:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2020-07-18T03:29:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:29:14.526774+0000", 2020-07-18T03:29:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:29:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:29:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:29:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:29:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:29:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "42", 2020-07-18T03:29:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:29:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2020-07-18T03:29:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2020-07-18T03:29:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2020-07-18T03:29:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:29:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2020-07-18T03:29:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 8, 2020-07-18T03:29:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2020-07-18T03:29:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:29:15.768057+0000", 2020-07-18T03:29:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:29:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:29:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:29:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:29:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:29:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:29:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:29:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:29:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:29:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:29:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:29:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:29:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:29:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:29:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:29:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:29:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:29:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:29:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:29:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:29:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "43", 2020-07-18T03:29:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:29:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:29:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:29:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:29:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:29:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:29:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:29:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:29:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:29:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:29:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:29:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:29:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:29:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:29:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:29:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:29:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:29:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:29:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:29:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:29:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:29:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:29:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:29:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:29:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:29:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:29:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:29:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:29:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:29:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:29:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:29:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:29:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:29:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:29:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:29:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:29:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:29:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:29:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:29:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:29:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2020-07-18T03:29:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2020-07-18T03:29:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:29:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", 2020-07-18T03:29:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 31, 2020-07-18T03:29:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2020-07-18T03:29:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2020-07-18T03:29:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2020-07-18T03:29:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2020-07-18T03:29:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2020-07-18T03:29:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2020-07-18T03:29:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2020-07-18T03:29:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2020-07-18T03:29:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:29:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6821/23865", 2020-07-18T03:29:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6825/23865", 2020-07-18T03:29:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6832/23865", 2020-07-18T03:29:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6829/23865", 2020-07-18T03:29:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:29:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", 2020-07-18T03:29:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 41, 2020-07-18T03:29:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2020-07-18T03:29:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2020-07-18T03:29:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2020-07-18T03:29:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2020-07-18T03:29:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2020-07-18T03:29:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2020-07-18T03:29:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2020-07-18T03:29:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2020-07-18T03:29:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:29:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/23853", 2020-07-18T03:29:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/23853", 2020-07-18T03:29:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/23853", 2020-07-18T03:29:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/23853", 2020-07-18T03:29:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:29:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "dc0b1a97-efca-483d-839c-37c286699283", 2020-07-18T03:29:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 41, 2020-07-18T03:29:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2020-07-18T03:29:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2020-07-18T03:29:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2020-07-18T03:29:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2020-07-18T03:29:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2020-07-18T03:29:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2020-07-18T03:29:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2020-07-18T03:29:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2020-07-18T03:29:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:29:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/23872", 2020-07-18T03:29:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6824/23872", 2020-07-18T03:29:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/23872", 2020-07-18T03:29:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6828/23872", 2020-07-18T03:29:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:29:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", 2020-07-18T03:29:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 39, 2020-07-18T03:29:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2020-07-18T03:29:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2020-07-18T03:29:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2020-07-18T03:29:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2020-07-18T03:29:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2020-07-18T03:29:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2020-07-18T03:29:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2020-07-18T03:29:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2020-07-18T03:29:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:29:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/23854", 2020-07-18T03:29:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/23854", 2020-07-18T03:29:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/23854", 2020-07-18T03:29:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/23854", 2020-07-18T03:29:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:29:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", 2020-07-18T03:29:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 39, 2020-07-18T03:29:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6808", 2020-07-18T03:29:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6809", 2020-07-18T03:29:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6810", 2020-07-18T03:29:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6811", 2020-07-18T03:29:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6814", 2020-07-18T03:29:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6815", 2020-07-18T03:29:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6812", 2020-07-18T03:29:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6813", 2020-07-18T03:29:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:29:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6809/24092", 2020-07-18T03:29:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6811/24092", 2020-07-18T03:29:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6815/24092", 2020-07-18T03:29:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6813/24092", 2020-07-18T03:29:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:29:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", 2020-07-18T03:29:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 41, 2020-07-18T03:29:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6800", 2020-07-18T03:29:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6801", 2020-07-18T03:29:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6802", 2020-07-18T03:29:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6803", 2020-07-18T03:29:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6806", 2020-07-18T03:29:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6807", 2020-07-18T03:29:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6804", 2020-07-18T03:29:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6805", 2020-07-18T03:29:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:29:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6801/24091", 2020-07-18T03:29:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6803/24091", 2020-07-18T03:29:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6807/24091", 2020-07-18T03:29:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6805/24091", 2020-07-18T03:29:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:29:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", 2020-07-18T03:29:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 41, 2020-07-18T03:29:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6816", 2020-07-18T03:29:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6817", 2020-07-18T03:29:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6819", 2020-07-18T03:29:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6821", 2020-07-18T03:29:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6828", 2020-07-18T03:29:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6830", 2020-07-18T03:29:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6823", 2020-07-18T03:29:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6826", 2020-07-18T03:29:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:29:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6817/24093", 2020-07-18T03:29:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6821/24093", 2020-07-18T03:29:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6830/24093", 2020-07-18T03:29:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6826/24093", 2020-07-18T03:29:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:29:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", 2020-07-18T03:29:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:29:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:29:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:29:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:29:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:29:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:29:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:29:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 41, 2020-07-18T03:29:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:29:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:29:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:29:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6818", 2020-07-18T03:29:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6820", 2020-07-18T03:29:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:29:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6822", 2020-07-18T03:29:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6824", 2020-07-18T03:29:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:29:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6829", 2020-07-18T03:29:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6831", 2020-07-18T03:29:47.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.857 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.857 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.857 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:29:47.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:29:47.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:29:47.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6825", 2020-07-18T03:29:47.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:29:47.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6827", 2020-07-18T03:29:47.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:29:47.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.860 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6820/24094", 2020-07-18T03:29:47.860 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6824/24094", 2020-07-18T03:29:47.860 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6831/24094", 2020-07-18T03:29:47.861 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6827/24094", 2020-07-18T03:29:47.861 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:29:47.861 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:29:47.861 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:29:47.861 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:47.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.862 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:47.862 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2020-07-18T03:29:47.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.862 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:29:47.863 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.863 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.863 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.863 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.863 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.864 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.933644+0000", 2020-07-18T03:29:47.864 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.865 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:29:47.865 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.865 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.865 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.866 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.866 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.866 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.929073+0000", 2020-07-18T03:29:47.866 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.867 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:29:47.867 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.867 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.867 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.867 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.868 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.868 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.974285+0000", 2020-07-18T03:29:47.868 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.869 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:29:47.869 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.869 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.869 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.869 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.869 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.870 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.909674+0000", 2020-07-18T03:29:47.870 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.870 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.870 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:29:47.871 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.871 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.871 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.871 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.871 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.872 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.935590+0000", 2020-07-18T03:29:47.872 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.872 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:29:47.872 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.873 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.873 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.873 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.873 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.873 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.916457+0000", 2020-07-18T03:29:47.874 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.874 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.874 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:29:47.874 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.875 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.875 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.875 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.875 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.875 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.990672+0000", 2020-07-18T03:29:47.875 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.876 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:47.876 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:29:47.876 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:29:47.876 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:29:47.877 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:29:47.877 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:29:47.877 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:29:47.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.948777+0000", 2020-07-18T03:29:47.877 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:29:47.878 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.878 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:29:47.878 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2020-07-18T03:29:47.878 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2020-07-18T03:29:47.878 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2020-07-18T03:29:47.879 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2020-07-18T03:29:47.879 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2020-07-18T03:29:47.879 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2020-07-18T03:29:47.879 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2020-07-18T03:29:47.880 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2020-07-18T03:29:47.880 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:29:47.880 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2020-07-18T03:29:47.880 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2020-07-18T03:29:47.880 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:29:47.881 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2020-07-18T03:29:47.881 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:29:47.881 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:29:47.881 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2020-07-18T03:29:47.881 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2020-07-18T03:29:47.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.882 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2020-07-18T03:29:47.882 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:29:47.882 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:29:47.882 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2020-07-18T03:29:47.882 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:29:47.883 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:29:47.883 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2020-07-18T03:29:47.883 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:47.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:47.883 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2020-07-18T03:29:47.884 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2020-07-18T03:29:47.884 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2020-07-18T03:29:47.884 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2020-07-18T03:29:47.884 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2020-07-18T03:29:47.884 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2924050060 msgr2=0x7f2924052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2924050060 0x7f2924052520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f2944017000 tx=0x7f2944016000).stop 2020-07-18T03:29:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0b0950 msgr2=0x7f294c13a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0b0950 0x7f294c13a910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f294c136e00 tx=0x7f2940007c30).stop 2020-07-18T03:29:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 shutdown_connections 2020-07-18T03:29:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2924050060 0x7f2924052520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f294c0b04c0 0x7f294c136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 --2- 172.21.15.47:0/1651436448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f294c0b0950 0x7f294c13a910 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 >> 172.21.15.47:0/1651436448 conn(0x7f294c0b9e50 msgr2=0x7f294c060090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 shutdown_connections 2020-07-18T03:29:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:47.661+0000 7f29523fa700 1 -- 172.21.15.47:0/1651436448 wait complete. 2020-07-18T03:29:47.890 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:48.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.026+0000 7f942568a700 1 Processor -- start 2020-07-18T03:29:48.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.026+0000 7f942568a700 1 -- start start 2020-07-18T03:29:48.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.026+0000 7f942568a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 0x7f94201437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.026+0000 7f942568a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f94200b6990 0x7f94200b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.026+0000 7f942568a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420143ca0 con 0x7f94200b6530 2020-07-18T03:29:48.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f942568a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200b3130 con 0x7f94200b6990 2020-07-18T03:29:48.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f942568a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94200b33d0 con 0x7f94200b2290 2020-07-18T03:29:48.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f9424e89700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f94200b6990 0x7f94200b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 0x7f94201437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 0x7f94201437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:53980/0 (socket says 172.21.15.47:53980) 2020-07-18T03:29:48.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f9424e89700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f94200b6990 0x7f94200b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:47066/0 (socket says 172.21.15.47:47066) 2020-07-18T03:29:48.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 -- 172.21.15.47:0/4282549619 learned_addr learned my addr 172.21.15.47:0/4282549619 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:48.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941f7fe700 1 --1- 172.21.15.47:0/4282549619 >> v1:172.21.15.18:6789/0 conn(0x7f94200b6530 0x7f94200b4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:33004/0 (socket says 172.21.15.47:33004) 2020-07-18T03:29:48.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 -- 172.21.15.47:0/4282549619 >> v2:172.21.15.18:3300/0 conn(0x7f94200b6990 msgr2=0x7f94200b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:48.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 --2- 172.21.15.47:0/4282549619 >> v2:172.21.15.18:3300/0 conn(0x7f94200b6990 0x7f94200b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 -- 172.21.15.47:0/4282549619 >> v1:172.21.15.18:6789/0 conn(0x7f94200b6530 legacy=0x7f94200b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 -- 172.21.15.47:0/4282549619 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200b2670 con 0x7f94200b2290 2020-07-18T03:29:48.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.027+0000 7f941ffff700 1 --2- 172.21.15.47:0/4282549619 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 0x7f94201437b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f9410002f30 tx=0x7f9410006e90).ready entity=mon.2 client_cookie=18d8efe4b177e256 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:48.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f941effd700 1 -- 172.21.15.47:0/4282549619 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9410009850 con 0x7f94200b2290 2020-07-18T03:29:48.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f941effd700 1 -- 172.21.15.47:0/4282549619 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94100099b0 con 0x7f94200b2290 2020-07-18T03:29:48.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f941effd700 1 -- 172.21.15.47:0/4282549619 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9410009d10 con 0x7f94200b2290 2020-07-18T03:29:48.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 -- 172.21.15.47:0/4282549619 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 msgr2=0x7f94201437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 --2- 172.21.15.47:0/4282549619 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 0x7f94201437b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f9410002f30 tx=0x7f9410006e90).stop 2020-07-18T03:29:48.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 -- 172.21.15.47:0/4282549619 shutdown_connections 2020-07-18T03:29:48.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 --2- 172.21.15.47:0/4282549619 >> v2:172.21.15.18:3300/0 conn(0x7f94200b6990 0x7f94200b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 --2- 172.21.15.47:0/4282549619 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94200b2290 0x7f94201437b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 -- 172.21.15.47:0/4282549619 >> 172.21.15.47:0/4282549619 conn(0x7f94200b9e50 msgr2=0x7f94200b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:48.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 -- 172.21.15.47:0/4282549619 shutdown_connections 2020-07-18T03:29:48.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.028+0000 7f942568a700 1 -- 172.21.15.47:0/4282549619 wait complete. 2020-07-18T03:29:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.029+0000 7f942568a700 1 Processor -- start 2020-07-18T03:29:48.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f942568a700 1 -- start start 2020-07-18T03:29:48.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f942568a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94200b6530 0x7f9420152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f942568a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420064380 con 0x7f9420063e70 2020-07-18T03:29:48.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f942568a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420064620 con 0x7f94200b6990 2020-07-18T03:29:48.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f942568a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200648c0 con 0x7f94200b6530 2020-07-18T03:29:48.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f941ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94200b6530 0x7f9420152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f941f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f94200b6990 0x7f9420145470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42478/0 (socket says 172.21.15.47:42478) 2020-07-18T03:29:48.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.030+0000 7f941f7fe700 1 -- 172.21.15.47:0/1317369392 learned_addr learned my addr 172.21.15.47:0/1317369392 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:48.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783215453 0 0) 0x7f9420064620 con 0x7f94200b6990 2020-07-18T03:29:48.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc003240 con 0x7f94200b6990 2020-07-18T03:29:48.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f941ffff700 1 -- 172.21.15.47:0/1317369392 >> v1:172.21.15.47:6789/0 conn(0x7f94200b6990 legacy=0x7f9420145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f941ffff700 1 -- 172.21.15.47:0/1317369392 >> v1:172.21.15.18:6789/0 conn(0x7f9420063e70 legacy=0x7f9420146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f941ffff700 1 -- 172.21.15.47:0/1317369392 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9420145b80 con 0x7f94200b6530 2020-07-18T03:29:48.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f941ffff700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94200b6530 0x7f9420152c80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f9410002f30 tx=0x7f9410015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:48.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f941000bec0 con 0x7f94200b6530 2020-07-18T03:29:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94200625a0 con 0x7f94200b6530 2020-07-18T03:29:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410011e40 con 0x7f94200b6530 2020-07-18T03:29:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f941001d7c0 con 0x7f94200b6530 2020-07-18T03:29:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.031+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9420064f20 con 0x7f94200b6530 2020-07-18T03:29:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.032+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420057450 con 0x7f94200b6530 2020-07-18T03:29:48.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.032+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9410025030 con 0x7f94200b6530 2020-07-18T03:29:48.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.033+0000 7f9407fff700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f93fc0524f0 0x7f93fc0549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.033+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f94100693a0 con 0x7f94200b6530 2020-07-18T03:29:48.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.033+0000 7f941f7fe700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f93fc0524f0 0x7f93fc0549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.033+0000 7f941f7fe700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f93fc0524f0 0x7f93fc0549b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f9414000eb0 tx=0x7f94140077e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.036+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9410041af0 con 0x7f94200b6530 2020-07-18T03:29:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.244+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9420068c50 con 0x7f94200b6530 2020-07-18T03:29:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.244+0000 7f9407fff700 1 -- 172.21.15.47:0/1317369392 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2134 (secure 0 0 0) 0x7f9410023090 con 0x7f94200b6530 2020-07-18T03:29:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:48.246 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:17.950573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5530":{"gid":5530,"name":"a","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/1368579801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1368579801},{"type":"v1","addr":"172.21.15.47:6835","nonce":1368579801}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f93fc0524f0 msgr2=0x7f93fc0549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f93fc0524f0 0x7f93fc0549b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f9414000eb0 tx=0x7f94140077e0).stop 2020-07-18T03:29:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94200b6530 msgr2=0x7f9420152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94200b6530 0x7f9420152c80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f9410002f30 tx=0x7f9410015000).stop 2020-07-18T03:29:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 shutdown_connections 2020-07-18T03:29:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94200b6530 0x7f9420152c80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 --2- 172.21.15.47:0/1317369392 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f93fc0524f0 0x7f93fc0549b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 >> 172.21.15.47:0/1317369392 conn(0x7f94200b9e50 msgr2=0x7f94200b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 shutdown_connections 2020-07-18T03:29:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.246+0000 7f942568a700 1 -- 172.21.15.47:0/1317369392 wait complete. 2020-07-18T03:29:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2020-07-18T03:29:48.258 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-07-18T03:29:48.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.380+0000 7fbc21a82700 1 Processor -- start 2020-07-18T03:29:48.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.380+0000 7fbc21a82700 1 -- start start 2020-07-18T03:29:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc21a82700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c05ffb0 0x7fbc1c0625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc21a82700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc1c0aea60 0x7fbc1c142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc21a82700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c0b5830 con 0x7fbc1c143330 2020-07-18T03:29:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc21a82700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c062290 con 0x7fbc1c0aea60 2020-07-18T03:29:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc21a82700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0b5a60 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc1c0aea60 0x7fbc1c142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc1c0aea60 0x7fbc1c142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33166/0 (socket says 172.21.15.47:33166) 2020-07-18T03:29:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 -- 172.21.15.47:0/4196816150 learned_addr learned my addr 172.21.15.47:0/4196816150 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1b7fe700 1 --2- 172.21.15.47:0/4196816150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c05ffb0 0x7fbc1c0625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 -- 172.21.15.47:0/4196816150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c05ffb0 msgr2=0x7fbc1c0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 --2- 172.21.15.47:0/4196816150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c05ffb0 0x7fbc1c0625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 -- 172.21.15.47:0/4196816150 >> v1:172.21.15.18:6789/0 conn(0x7fbc1c143330 legacy=0x7fbc1c061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 -- 172.21.15.47:0/4196816150 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c146da0 con 0x7fbc1c0aea60 2020-07-18T03:29:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1affd700 1 --2- 172.21.15.47:0/4196816150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c0aea60 0x7fbc1c142e40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fbc04002f30 tx=0x7fbc04006e90).ready entity=mon.1 client_cookie=59ec8868082e08e6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1a7fc700 1 -- 172.21.15.47:0/4196816150 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc04009850 con 0x7fbc1c0aea60 2020-07-18T03:29:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1a7fc700 1 -- 172.21.15.47:0/4196816150 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc040099b0 con 0x7fbc1c0aea60 2020-07-18T03:29:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.381+0000 7fbc1a7fc700 1 -- 172.21.15.47:0/4196816150 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc04009d10 con 0x7fbc1c0aea60 2020-07-18T03:29:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 -- 172.21.15.47:0/4196816150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c0aea60 msgr2=0x7fbc1c142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 --2- 172.21.15.47:0/4196816150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c0aea60 0x7fbc1c142e40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fbc04002f30 tx=0x7fbc04006e90).stop 2020-07-18T03:29:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 -- 172.21.15.47:0/4196816150 shutdown_connections 2020-07-18T03:29:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 --2- 172.21.15.47:0/4196816150 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c05ffb0 0x7fbc1c0625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 --2- 172.21.15.47:0/4196816150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c0aea60 0x7fbc1c142e40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 -- 172.21.15.47:0/4196816150 >> 172.21.15.47:0/4196816150 conn(0x7fbc1c0b93c0 msgr2=0x7fbc1c0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 -- 172.21.15.47:0/4196816150 shutdown_connections 2020-07-18T03:29:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.382+0000 7fbc21a82700 1 -- 172.21.15.47:0/4196816150 wait complete. 2020-07-18T03:29:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.383+0000 7fbc21a82700 1 Processor -- start 2020-07-18T03:29:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.383+0000 7fbc21a82700 1 -- start start 2020-07-18T03:29:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc21a82700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc1c05ffb0 0x7fbc1c153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc21a82700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c143330 0x7fbc1c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc21a82700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c157700 con 0x7fbc1c0b31f0 2020-07-18T03:29:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc21a82700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c1533e0 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc21a82700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c153680 con 0x7fbc1c143330 2020-07-18T03:29:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc1c05ffb0 0x7fbc1c153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc1c05ffb0 0x7fbc1c153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33170/0 (socket says 172.21.15.47:33170) 2020-07-18T03:29:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 -- 172.21.15.47:0/2892216761 learned_addr learned my addr 172.21.15.47:0/2892216761 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1affd700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c143330 0x7fbc1c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 -- 172.21.15.47:0/2892216761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c143330 msgr2=0x7fbc1c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c143330 0x7fbc1c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 -- 172.21.15.47:0/2892216761 >> v1:172.21.15.18:6789/0 conn(0x7fbc1c0b31f0 legacy=0x7fbc1c145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 -- 172.21.15.47:0/2892216761 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c153920 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1affd700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c143330 0x7fbc1c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.384+0000 7fbc1b7fe700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c05ffb0 0x7fbc1c153dc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fbc1400c000 tx=0x7fbc140091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.385+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1400cbb0 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.385+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1400cd10 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.385+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc140125e0 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.385+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c13bdd0 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.385+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c13c2f0 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.386+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c057450 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.386+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbc14022070 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.386+0000 7fbc137fe700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbbf8050060 0x7fbbf8052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.386+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbc140684d0 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.386+0000 7fbc1affd700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbbf8050060 0x7fbbf8052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.386+0000 7fbc1affd700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbbf8050060 0x7fbbf8052520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fbc0401a000 tx=0x7fbc04015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.389+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbc14040c60 con 0x7fbc1c05ffb0 2020-07-18T03:29:48.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:48.596+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc1c0681b0 con 0x7fbc1c05ffb0 2020-07-18T03:29:49.242 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:29:49.241+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:29:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.245+0000 7fbc137fe700 1 -- 172.21.15.47:0/2892216761 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc14044280 con 0x7fbc1c05ffb0 2020-07-18T03:29:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbbf8050060 msgr2=0x7fbbf8052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbbf8050060 0x7fbbf8052520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fbc0401a000 tx=0x7fbc04015000).stop 2020-07-18T03:29:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c05ffb0 msgr2=0x7fbc1c153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c05ffb0 0x7fbc1c153dc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fbc1400c000 tx=0x7fbc140091d0).stop 2020-07-18T03:29:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 shutdown_connections 2020-07-18T03:29:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbbf8050060 0x7fbbf8052520 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc1c05ffb0 0x7fbc1c153dc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 --2- 172.21.15.47:0/2892216761 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbc1c143330 0x7fbc1c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 >> 172.21.15.47:0/2892216761 conn(0x7fbc1c0b93c0 msgr2=0x7fbc1c142240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 shutdown_connections 2020-07-18T03:29:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.248+0000 7fbc21a82700 1 -- 172.21.15.47:0/2892216761 wait complete. 2020-07-18T03:29:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-07-18T03:29:49.273 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:49.282 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:29:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.399+0000 7f7e94caa700 1 Processor -- start 2020-07-18T03:29:49.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.399+0000 7f7e94caa700 1 -- start start 2020-07-18T03:29:49.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e94caa700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7e90147e40 0x7f7e90143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e94caa700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e90147330 con 0x7f7e90147a00 2020-07-18T03:29:49.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e94caa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900b57e0 con 0x7f7e900ad100 2020-07-18T03:29:49.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e94caa700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e900ae530 con 0x7f7e90147e40 2020-07-18T03:29:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e8e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e900ad100 0x7f7e900b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42510/0 (socket says 172.21.15.47:42510) 2020-07-18T03:29:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e8e59c700 1 -- 172.21.15.47:0/3804125759 learned_addr learned my addr 172.21.15.47:0/3804125759 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e8ed9d700 1 --2- 172.21.15.47:0/3804125759 >> v2:172.21.15.18:3300/0 conn(0x7f7e90147e40 0x7f7e90143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e8dd9b700 1 -- 172.21.15.47:0/3804125759 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684844997 0 0) 0x7f7e900b57e0 con 0x7f7e900ad100 2020-07-18T03:29:49.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e8dd9b700 1 -- 172.21.15.47:0/3804125759 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e70003240 con 0x7f7e900ad100 2020-07-18T03:29:49.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.400+0000 7f7e8dd9b700 1 -- 172.21.15.47:0/3804125759 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2127860758 0 0) 0x7f7e70003240 con 0x7f7e900ad100 2020-07-18T03:29:49.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8ed9d700 1 -- 172.21.15.47:0/3804125759 >> v1:172.21.15.47:6789/0 conn(0x7f7e900ad100 legacy=0x7f7e900b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8ed9d700 1 -- 172.21.15.47:0/3804125759 >> v1:172.21.15.18:6789/0 conn(0x7f7e90147a00 legacy=0x7f7e901433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8ed9d700 1 -- 172.21.15.47:0/3804125759 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900b5a80 con 0x7f7e90147e40 2020-07-18T03:29:49.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8ed9d700 1 --2- 172.21.15.47:0/3804125759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e90147e40 0x7f7e90143af0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f7e780010d0 tx=0x7f7e880091d0).ready entity=mon.0 client_cookie=b9042d2c1e559f56 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:49.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8dd9b700 1 -- 172.21.15.47:0/3804125759 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e800016d0 con 0x7f7e90147e40 2020-07-18T03:29:49.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8dd9b700 1 -- 172.21.15.47:0/3804125759 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8800b7e0 con 0x7f7e90147e40 2020-07-18T03:29:49.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e8dd9b700 1 -- 172.21.15.47:0/3804125759 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e8800b950 con 0x7f7e90147e40 2020-07-18T03:29:49.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 -- 172.21.15.47:0/3804125759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e90147e40 msgr2=0x7f7e90143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3804125759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e90147e40 0x7f7e90143af0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f7e780010d0 tx=0x7f7e880091d0).stop 2020-07-18T03:29:49.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 -- 172.21.15.47:0/3804125759 shutdown_connections 2020-07-18T03:29:49.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3804125759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e90147e40 0x7f7e90143af0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 -- 172.21.15.47:0/3804125759 >> 172.21.15.47:0/3804125759 conn(0x7f7e900b9340 msgr2=0x7f7e900b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:49.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 -- 172.21.15.47:0/3804125759 shutdown_connections 2020-07-18T03:29:49.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.401+0000 7f7e94caa700 1 -- 172.21.15.47:0/3804125759 wait complete. 2020-07-18T03:29:49.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.402+0000 7f7e94caa700 1 Processor -- start 2020-07-18T03:29:49.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.402+0000 7f7e94caa700 1 -- start start 2020-07-18T03:29:49.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e94caa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e900ad100 0x7f7e90133070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e94caa700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e90147a00 0x7f7e90133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.417 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e94caa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e90063990 con 0x7f7e90133d80 2020-07-18T03:29:49.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e94caa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e90063c30 con 0x7f7e900ad100 2020-07-18T03:29:49.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e94caa700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90064f90 con 0x7f7e90147a00 2020-07-18T03:29:49.418 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e8e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e900ad100 0x7f7e90133070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e8e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e900ad100 0x7f7e90133070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33196/0 (socket says 172.21.15.47:33196) 2020-07-18T03:29:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e8ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e90133d80 0x7f7e90066350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42518/0 (socket says 172.21.15.47:42518) 2020-07-18T03:29:49.419 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e8e59c700 1 -- 172.21.15.47:0/3195373591 learned_addr learned my addr 172.21.15.47:0/3195373591 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.403+0000 7f7e87fff700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e90147a00 0x7f7e90133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e8e59c700 1 -- 172.21.15.47:0/3195373591 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e90147a00 msgr2=0x7f7e90133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e8e59c700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e90147a00 0x7f7e90133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e8e59c700 1 -- 172.21.15.47:0/3195373591 >> v1:172.21.15.47:6789/0 conn(0x7f7e90133d80 legacy=0x7f7e90066350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878343551 0 0) 0x7f7e90063990 con 0x7f7e90133d80 2020-07-18T03:29:49.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e8e59c700 1 -- 172.21.15.47:0/3195373591 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90133560 con 0x7f7e900ad100 2020-07-18T03:29:49.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e87fff700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e90147a00 0x7f7e90133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:49.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e8e59c700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e900ad100 0x7f7e90133070 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f7e80001930 tx=0x7f7e800011f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:49.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e80007170 con 0x7f7e900ad100 2020-07-18T03:29:49.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e800072d0 con 0x7f7e900ad100 2020-07-18T03:29:49.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e90066a60 con 0x7f7e900ad100 2020-07-18T03:29:49.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e80004900 con 0x7f7e900ad100 2020-07-18T03:29:49.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.404+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e90064290 con 0x7f7e900ad100 2020-07-18T03:29:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.405+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e90057450 con 0x7f7e900ad100 2020-07-18T03:29:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.405+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7e80010070 con 0x7f7e900ad100 2020-07-18T03:29:49.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.405+0000 7f7e867fc700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e6c04fe50 0x7f7e6c052310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.406+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7e800678f0 con 0x7f7e900ad100 2020-07-18T03:29:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.406+0000 7f7e87fff700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e6c04fe50 0x7f7e6c052310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.406+0000 7f7e87fff700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e6c04fe50 0x7f7e6c052310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f7e7800b000 tx=0x7f7e78009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:49.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.409+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7e8003ffb0 con 0x7f7e900ad100 2020-07-18T03:29:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.619+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e901393e0 con 0x7f7e900ad100 2020-07-18T03:29:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.619+0000 7f7e867fc700 1 -- 172.21.15.47:0/3195373591 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+1773 (secure 0 0 0) 0x7f7e800435d0 con 0x7f7e900ad100 2020-07-18T03:29:49.621 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:49.621 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/2700459732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":2700459732},{"type":"v1","addr":"172.21.15.18:6833","nonce":2700459732}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:16.949357+0000","modified":"2020-07-18T03:29:49.241518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-07-18T03:29:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e6c04fe50 msgr2=0x7f7e6c052310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e6c04fe50 0x7f7e6c052310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f7e7800b000 tx=0x7f7e78009040).stop 2020-07-18T03:29:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e900ad100 msgr2=0x7f7e90133070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e900ad100 0x7f7e90133070 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f7e80001930 tx=0x7f7e800011f0).stop 2020-07-18T03:29:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 shutdown_connections 2020-07-18T03:29:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e6c04fe50 0x7f7e6c052310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e900ad100 0x7f7e90133070 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 --2- 172.21.15.47:0/3195373591 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e90147a00 0x7f7e90133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 >> 172.21.15.47:0/3195373591 conn(0x7f7e900b9340 msgr2=0x7f7e901428e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 shutdown_connections 2020-07-18T03:29:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.621+0000 7f7e94caa700 1 -- 172.21.15.47:0/3195373591 wait complete. 2020-07-18T03:29:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2020-07-18T03:29:49.633 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-07-18T03:29:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.750+0000 7fb736ed4700 1 Processor -- start 2020-07-18T03:29:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.750+0000 7fb736ed4700 1 -- start start 2020-07-18T03:29:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb736ed4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb730141fb0 0x7fb730142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb736ed4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb730061db0 con 0x7fb7300b04b0 2020-07-18T03:29:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb736ed4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb730062050 con 0x7fb730144e70 2020-07-18T03:29:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb736ed4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb730145250 con 0x7fb730141fb0 2020-07-18T03:29:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb730141fb0 0x7fb730142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb734c70700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb730144e70 0x7fb7300b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42524/0 (socket says 172.21.15.47:42524) 2020-07-18T03:29:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb734c70700 1 -- 172.21.15.47:0/3871189945 learned_addr learned my addr 172.21.15.47:0/3871189945 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72ffff700 1 -- 172.21.15.47:0/3871189945 >> v1:172.21.15.47:6789/0 conn(0x7fb730144e70 legacy=0x7fb7300b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72ffff700 1 -- 172.21.15.47:0/3871189945 >> v1:172.21.15.18:6789/0 conn(0x7fb7300b04b0 legacy=0x7fb7300b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72ffff700 1 -- 172.21.15.47:0/3871189945 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300622f0 con 0x7fb730141fb0 2020-07-18T03:29:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72ffff700 1 --2- 172.21.15.47:0/3871189945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb730141fb0 0x7fb730142430 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fb7240010d0 tx=0x7fb71c006dd0).ready entity=mon.1 client_cookie=120ae83f4bfff504 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72f7fe700 1 -- 172.21.15.47:0/3871189945 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb71c009730 con 0x7fb730141fb0 2020-07-18T03:29:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.751+0000 7fb72f7fe700 1 -- 172.21.15.47:0/3871189945 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c009890 con 0x7fb730141fb0 2020-07-18T03:29:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb72f7fe700 1 -- 172.21.15.47:0/3871189945 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb71c009bf0 con 0x7fb730141fb0 2020-07-18T03:29:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 -- 172.21.15.47:0/3871189945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb730141fb0 msgr2=0x7fb730142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 --2- 172.21.15.47:0/3871189945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb730141fb0 0x7fb730142430 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fb7240010d0 tx=0x7fb71c006dd0).stop 2020-07-18T03:29:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 -- 172.21.15.47:0/3871189945 shutdown_connections 2020-07-18T03:29:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 --2- 172.21.15.47:0/3871189945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb730141fb0 0x7fb730142430 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 -- 172.21.15.47:0/3871189945 >> 172.21.15.47:0/3871189945 conn(0x7fb73005f5a0 msgr2=0x7fb73005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 -- 172.21.15.47:0/3871189945 shutdown_connections 2020-07-18T03:29:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.752+0000 7fb736ed4700 1 -- 172.21.15.47:0/3871189945 wait complete. 2020-07-18T03:29:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.753+0000 7fb736ed4700 1 Processor -- start 2020-07-18T03:29:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.753+0000 7fb736ed4700 1 -- start start 2020-07-18T03:29:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb736ed4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb730144e70 0x7fb73012a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb736ed4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7300b5420 0x7fb73012a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb736ed4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300b3f90 con 0x7fb7300b04b0 2020-07-18T03:29:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb736ed4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300b4230 con 0x7fb730144e70 2020-07-18T03:29:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb736ed4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300b44d0 con 0x7fb7300b5420 2020-07-18T03:29:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7300b5420 0x7fb73012a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7300b5420 0x7fb73012a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33214/0 (socket says 172.21.15.47:33214) 2020-07-18T03:29:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 -- 172.21.15.47:0/244479714 learned_addr learned my addr 172.21.15.47:0/244479714 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb72ffff700 1 --2- 172.21.15.47:0/244479714 >> v2:172.21.15.18:3300/0 conn(0x7fb730144e70 0x7fb73012a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 -- 172.21.15.47:0/244479714 >> v2:172.21.15.18:3300/0 conn(0x7fb730144e70 msgr2=0x7fb73012a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 --2- 172.21.15.47:0/244479714 >> v2:172.21.15.18:3300/0 conn(0x7fb730144e70 0x7fb73012a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 -- 172.21.15.47:0/244479714 >> v1:172.21.15.18:6789/0 conn(0x7fb7300b04b0 legacy=0x7fb730143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 -- 172.21.15.47:0/244479714 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300b4970 con 0x7fb7300b5420 2020-07-18T03:29:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.754+0000 7fb735471700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7300b5420 0x7fb73012a7e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fb7300b4d30 tx=0x7fb728007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.755+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72800a5c0 con 0x7fb7300b5420 2020-07-18T03:29:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.755+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800a720 con 0x7fb7300b5420 2020-07-18T03:29:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.755+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72800aa80 con 0x7fb7300b5420 2020-07-18T03:29:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.755+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73012acd0 con 0x7fb7300b5420 2020-07-18T03:29:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.755+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7300b3c50 con 0x7fb7300b5420 2020-07-18T03:29:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.756+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73005b120 con 0x7fb7300b5420 2020-07-18T03:29:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.756+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb728019030 con 0x7fb7300b5420 2020-07-18T03:29:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.756+0000 7fb72cff9700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb70804ff70 0x7fb708052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.756+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb7280675f0 con 0x7fb7300b5420 2020-07-18T03:29:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.756+0000 7fb734c70700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb70804ff70 0x7fb708052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.756+0000 7fb734c70700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb70804ff70 0x7fb708052430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb730155430 tx=0x7fb724007860).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.760+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb72803fd30 con 0x7fb7300b5420 2020-07-18T03:29:49.813 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:29:49.821 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:29:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:49.970+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb7300690d0 con 0x7fb7300b5420 2020-07-18T03:29:50.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.248+0000 7fb72cff9700 1 -- 172.21.15.47:0/244479714 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb728043350 con 0x7fb7300b5420 2020-07-18T03:29:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb70804ff70 msgr2=0x7fb708052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb70804ff70 0x7fb708052430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb730155430 tx=0x7fb724007860).stop 2020-07-18T03:29:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7300b5420 msgr2=0x7fb73012a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7300b5420 0x7fb73012a7e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fb7300b4d30 tx=0x7fb728007e50).stop 2020-07-18T03:29:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 shutdown_connections 2020-07-18T03:29:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb70804ff70 0x7fb708052430 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 --2- 172.21.15.47:0/244479714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7300b5420 0x7fb73012a7e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 --2- 172.21.15.47:0/244479714 >> v2:172.21.15.18:3300/0 conn(0x7fb730144e70 0x7fb73012a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 >> 172.21.15.47:0/244479714 conn(0x7fb73005f5a0 msgr2=0x7fb7300ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.249+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 shutdown_connections 2020-07-18T03:29:50.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.250+0000 7fb736ed4700 1 -- 172.21.15.47:0/244479714 wait complete. 2020-07-18T03:29:50.262 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-07-18T03:29:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.380+0000 7f5561032700 1 Processor -- start 2020-07-18T03:29:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.381+0000 7f5561032700 1 -- start start 2020-07-18T03:29:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.381+0000 7f5561032700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 0x7f555c0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.381+0000 7f5561032700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.381+0000 7f5561032700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0af4b0 con 0x7f555c142a20 2020-07-18T03:29:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.381+0000 7f5561032700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0b6d80 con 0x7f555c0b3af0 2020-07-18T03:29:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.381+0000 7f5561032700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0af710 con 0x7f555c061c60 2020-07-18T03:29:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555b59e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 0x7f555c0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 0x7f555c0620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54050/0 (socket says 172.21.15.47:54050) 2020-07-18T03:29:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555b59e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:47136/0 (socket says 172.21.15.47:47136) 2020-07-18T03:29:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 -- 172.21.15.47:0/2002164885 learned_addr learned my addr 172.21.15.47:0/2002164885 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555ad9d700 1 --1- 172.21.15.47:0/2002164885 >> v1:172.21.15.18:6789/0 conn(0x7f555c142a20 0x7f555c142e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:33076/0 (socket says 172.21.15.47:33076) 2020-07-18T03:29:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 -- 172.21.15.47:0/2002164885 >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 msgr2=0x7f555c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 --2- 172.21.15.47:0/2002164885 >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 -- 172.21.15.47:0/2002164885 >> v1:172.21.15.18:6789/0 conn(0x7f555c142a20 legacy=0x7f555c142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 -- 172.21.15.47:0/2002164885 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0af9b0 con 0x7f555c061c60 2020-07-18T03:29:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555b59e700 1 --2- 172.21.15.47:0/2002164885 >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f555a59c700 1 --2- 172.21.15.47:0/2002164885 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 0x7f555c0620c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f554c002f30 tx=0x7f554c006e90).ready entity=mon.2 client_cookie=a4c86ebfb29eb812 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5559d9b700 1 -- 172.21.15.47:0/2002164885 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f554c009850 con 0x7f555c061c60 2020-07-18T03:29:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5559d9b700 1 -- 172.21.15.47:0/2002164885 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c0099b0 con 0x7f555c061c60 2020-07-18T03:29:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5559d9b700 1 -- 172.21.15.47:0/2002164885 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f554c009d10 con 0x7f555c061c60 2020-07-18T03:29:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 -- 172.21.15.47:0/2002164885 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 msgr2=0x7f555c0620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 --2- 172.21.15.47:0/2002164885 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 0x7f555c0620c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f554c002f30 tx=0x7f554c006e90).stop 2020-07-18T03:29:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 -- 172.21.15.47:0/2002164885 shutdown_connections 2020-07-18T03:29:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 --2- 172.21.15.47:0/2002164885 >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 --2- 172.21.15.47:0/2002164885 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f555c061c60 0x7f555c0620c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 -- 172.21.15.47:0/2002164885 >> 172.21.15.47:0/2002164885 conn(0x7f555c05f5a0 msgr2=0x7f555c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.382+0000 7f5561032700 1 -- 172.21.15.47:0/2002164885 shutdown_connections 2020-07-18T03:29:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.383+0000 7f5561032700 1 -- 172.21.15.47:0/2002164885 wait complete. 2020-07-18T03:29:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.384+0000 7f5561032700 1 Processor -- start 2020-07-18T03:29:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.384+0000 7f5561032700 1 -- start start 2020-07-18T03:29:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5561032700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5561032700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0b2e50 con 0x7f555c0667e0 2020-07-18T03:29:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5561032700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0b30f0 con 0x7f555c142a20 2020-07-18T03:29:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5561032700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c13dac0 con 0x7f555c0b3af0 2020-07-18T03:29:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f555a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f555c142a20 0x7f555c143760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42550/0 (socket says 172.21.15.47:42550) 2020-07-18T03:29:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f555a59c700 1 -- 172.21.15.47:0/3664354661 learned_addr learned my addr 172.21.15.47:0/3664354661 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f555ad9d700 1 --2- 172.21.15.47:0/3664354661 >> v2:172.21.15.18:3300/0 conn(0x7f555c0b3af0 0x7f555c066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161848537 0 0) 0x7f555c0b30f0 con 0x7f555c142a20 2020-07-18T03:29:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5538003240 con 0x7f555c142a20 2020-07-18T03:29:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f555ad9d700 1 -- 172.21.15.47:0/3664354661 >> v1:172.21.15.47:6789/0 conn(0x7f555c142a20 legacy=0x7f555c143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f555ad9d700 1 -- 172.21.15.47:0/3664354661 >> v1:172.21.15.18:6789/0 conn(0x7f555c0667e0 legacy=0x7f555c13d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085021442 0 0) 0x7f555c0b2e50 con 0x7f555c0667e0 2020-07-18T03:29:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.385+0000 7f555ad9d700 1 -- 172.21.15.47:0/3664354661 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0b1840 con 0x7f555c0b3af0 2020-07-18T03:29:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.386+0000 7f555ad9d700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f555c0b3af0 0x7f555c066d50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f555c0afd30 tx=0x7f5544007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.386+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55440072e0 con 0x7f555c0b3af0 2020-07-18T03:29:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.386+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544005530 con 0x7f555c0b3af0 2020-07-18T03:29:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.386+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c0b1a90 con 0x7f555c0b3af0 2020-07-18T03:29:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.386+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5544005890 con 0x7f555c0b3af0 2020-07-18T03:29:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.386+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c150fd0 con 0x7f555c0b3af0 2020-07-18T03:29:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.387+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c0b3280 con 0x7f555c0b3af0 2020-07-18T03:29:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.387+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5544010070 con 0x7f555c0b3af0 2020-07-18T03:29:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.387+0000 7f5552ffd700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5538050310 0x7f55380527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.387+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5544067190 con 0x7f555c0b3af0 2020-07-18T03:29:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.387+0000 7f555a59c700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5538050310 0x7f55380527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.388+0000 7f555a59c700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5538050310 0x7f55380527d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f554c006f90 tx=0x7f554c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.391+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f554403f830 con 0x7f555c0b3af0 2020-07-18T03:29:50.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:50.548+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f555c05b120 con 0x7f555c0b3af0 2020-07-18T03:29:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.282+0000 7f5552ffd700 1 -- 172.21.15.47:0/3664354661 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5544042e50 con 0x7f555c0b3af0 2020-07-18T03:29:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5538050310 msgr2=0x7f55380527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5538050310 0x7f55380527d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f554c006f90 tx=0x7f554c015000).stop 2020-07-18T03:29:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f555c0b3af0 msgr2=0x7f555c066d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f555c0b3af0 0x7f555c066d50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f555c0afd30 tx=0x7f5544007a90).stop 2020-07-18T03:29:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 shutdown_connections 2020-07-18T03:29:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f555c0b3af0 0x7f555c066d50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 --2- 172.21.15.47:0/3664354661 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5538050310 0x7f55380527d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 >> 172.21.15.47:0/3664354661 conn(0x7f555c05f5a0 msgr2=0x7f555c0b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 shutdown_connections 2020-07-18T03:29:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.285+0000 7f5561032700 1 -- 172.21.15.47:0/3664354661 wait complete. 2020-07-18T03:29:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2020-07-18T03:29:51.300 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-07-18T03:29:51.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.424+0000 7fbfb24eb700 1 Processor -- start 2020-07-18T03:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.424+0000 7fbfb24eb700 1 -- start start 2020-07-18T03:29:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfb24eb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfac142a20 0x7fbfac142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfb24eb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfb24eb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfac0af4b0 con 0x7fbfac0b5dc0 2020-07-18T03:29:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfb24eb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac0b6d80 con 0x7fbfac142a20 2020-07-18T03:29:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfb24eb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac0af710 con 0x7fbfac061c60 2020-07-18T03:29:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfac142a20 0x7fbfac142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfac142a20 0x7fbfac142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33234/0 (socket says 172.21.15.47:33234) 2020-07-18T03:29:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 -- 172.21.15.47:0/1378291775 learned_addr learned my addr 172.21.15.47:0/1378291775 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfab7fe700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 -- 172.21.15.47:0/1378291775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 msgr2=0x7fbfac0620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac0620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 -- 172.21.15.47:0/1378291775 >> v1:172.21.15.18:6789/0 conn(0x7fbfac0b5dc0 legacy=0x7fbfac146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.425+0000 7fbfabfff700 1 -- 172.21.15.47:0/1378291775 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0af9b0 con 0x7fbfac142a20 2020-07-18T03:29:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfab7fe700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac0620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfabfff700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac142a20 0x7fbfac142e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fbf9c002f30 tx=0x7fbf9c006e90).ready entity=mon.1 client_cookie=49206e063bae3157 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfaaffd700 1 -- 172.21.15.47:0/1378291775 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9c009970 con 0x7fbfac142a20 2020-07-18T03:29:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfaaffd700 1 -- 172.21.15.47:0/1378291775 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9c009ad0 con 0x7fbfac142a20 2020-07-18T03:29:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfaaffd700 1 -- 172.21.15.47:0/1378291775 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9c009e30 con 0x7fbfac142a20 2020-07-18T03:29:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 -- 172.21.15.47:0/1378291775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac142a20 msgr2=0x7fbfac142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac142a20 0x7fbfac142e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fbf9c002f30 tx=0x7fbf9c006e90).stop 2020-07-18T03:29:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 -- 172.21.15.47:0/1378291775 shutdown_connections 2020-07-18T03:29:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac142a20 0x7fbfac142e00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/1378291775 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac0620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 -- 172.21.15.47:0/1378291775 >> 172.21.15.47:0/1378291775 conn(0x7fbfac05f5a0 msgr2=0x7fbfac05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 -- 172.21.15.47:0/1378291775 shutdown_connections 2020-07-18T03:29:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.426+0000 7fbfb24eb700 1 -- 172.21.15.47:0/1378291775 wait complete. 2020-07-18T03:29:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.427+0000 7fbfb24eb700 1 Processor -- start 2020-07-18T03:29:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb24eb700 1 -- start start 2020-07-18T03:29:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb24eb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb24eb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfac144e90 0x7fbfac145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb24eb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfac155db0 con 0x7fbfac0b5dc0 2020-07-18T03:29:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb24eb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac0b0630 con 0x7fbfac144e90 2020-07-18T03:29:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb24eb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac143900 con 0x7fbfac061c60 2020-07-18T03:29:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb0a88700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfac144e90 0x7fbfac145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb0a88700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbfac144e90 0x7fbfac145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33244/0 (socket says 172.21.15.47:33244) 2020-07-18T03:29:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfb0a88700 1 -- 172.21.15.47:0/4234910608 learned_addr learned my addr 172.21.15.47:0/4234910608 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.428+0000 7fbfabfff700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb0a88700 1 -- 172.21.15.47:0/4234910608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 msgr2=0x7fbfac146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb0a88700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb0a88700 1 -- 172.21.15.47:0/4234910608 >> v1:172.21.15.18:6789/0 conn(0x7fbfac0b5dc0 legacy=0x7fbfac0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:51.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380367026 0 0) 0x7fbfac155db0 con 0x7fbfac0b5dc0 2020-07-18T03:29:51.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb0a88700 1 -- 172.21.15.47:0/4234910608 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac155a00 con 0x7fbfac144e90 2020-07-18T03:29:51.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb0a88700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac144e90 0x7fbfac145340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fbfac0afd30 tx=0x7fbfa4007e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:51.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfa400a5a0 con 0x7fbfac144e90 2020-07-18T03:29:51.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa400a700 con 0x7fbfac144e90 2020-07-18T03:29:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac0b0ad0 con 0x7fbfac144e90 2020-07-18T03:29:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfa400aa60 con 0x7fbfac144e90 2020-07-18T03:29:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.429+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfac0b0fc0 con 0x7fbfac144e90 2020-07-18T03:29:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.430+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfac155f40 con 0x7fbfac144e90 2020-07-18T03:29:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.430+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbfa4019030 con 0x7fbfac144e90 2020-07-18T03:29:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.431+0000 7fbf8bfff700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf8404fe10 0x7fbf840522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.431+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fbfa4067480 con 0x7fbfac144e90 2020-07-18T03:29:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.431+0000 7fbfabfff700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf8404fe10 0x7fbf840522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.431+0000 7fbfabfff700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf8404fe10 0x7fbf840522d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fbf9c01c000 tx=0x7fbf9c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:51.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.434+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbfa403fcf0 con 0x7fbfac144e90 2020-07-18T03:29:51.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:51.592+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbfac05b120 con 0x7fbfac144e90 2020-07-18T03:29:52.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.287+0000 7fbf8bfff700 1 -- 172.21.15.47:0/4234910608 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbfa4043310 con 0x7fbfac144e90 2020-07-18T03:29:52.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf8404fe10 msgr2=0x7fbf840522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf8404fe10 0x7fbf840522d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fbf9c01c000 tx=0x7fbf9c015000).stop 2020-07-18T03:29:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac144e90 msgr2=0x7fbfac145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac144e90 0x7fbfac145340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fbfac0afd30 tx=0x7fbfa4007e60).stop 2020-07-18T03:29:52.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 shutdown_connections 2020-07-18T03:29:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbf8404fe10 0x7fbf840522d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfac144e90 0x7fbfac145340 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 --2- 172.21.15.47:0/4234910608 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbfac061c60 0x7fbfac146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 >> 172.21.15.47:0/4234910608 conn(0x7fbfac05f5a0 msgr2=0x7fbfac13ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 shutdown_connections 2020-07-18T03:29:52.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.290+0000 7fbfb24eb700 1 -- 172.21.15.47:0/4234910608 wait complete. 2020-07-18T03:29:52.294 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2020-07-18T03:29:52.306 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:29:52.306 INFO:tasks.ceph.mds.a:Stopping old one... 2020-07-18T03:29:52.306 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:29:52.307 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:29:52.307 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:29:52.307 INFO:tasks.ceph.mds.b:Stopping old one... 2020-07-18T03:29:52.308 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:29:52.308 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:29:52.316 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:29:52.317 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:29:52.329 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:29:52.329 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:29:52.359 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:29:52.360 DEBUG:teuthology.parallel:result is None 2020-07-18T03:29:52.361 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:29:52.361 DEBUG:teuthology.parallel:result is None 2020-07-18T03:29:52.362 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-07-18T03:29:52.446 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:29:52.449 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:29:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.485+0000 7fc342b9b700 1 Processor -- start 2020-07-18T03:29:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.485+0000 7fc342b9b700 1 -- start start 2020-07-18T03:29:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc342b9b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc33c148e30 0x7fc33c1492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc342b9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc33c0b0500 0x7fc33c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc342b9b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c145500 con 0x7fc33c145120 2020-07-18T03:29:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc342b9b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c146150 con 0x7fc33c148e30 2020-07-18T03:29:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc342b9b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c145770 con 0x7fc33c0b0500 2020-07-18T03:29:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc33c0b0500 0x7fc33c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc33c0b0500 0x7fc33c0b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33264/0 (socket says 172.21.15.47:33264) 2020-07-18T03:29:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc340937700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc33c145120 0x7fc33c061c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42580/0 (socket says 172.21.15.47:42580) 2020-07-18T03:29:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 -- 172.21.15.47:0/908002131 learned_addr learned my addr 172.21.15.47:0/908002131 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc33bfff700 1 --2- 172.21.15.47:0/908002131 >> v2:172.21.15.18:3300/0 conn(0x7fc33c148e30 0x7fc33c1492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 -- 172.21.15.47:0/908002131 >> v2:172.21.15.18:3300/0 conn(0x7fc33c148e30 msgr2=0x7fc33c1492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 --2- 172.21.15.47:0/908002131 >> v2:172.21.15.18:3300/0 conn(0x7fc33c148e30 0x7fc33c1492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 -- 172.21.15.47:0/908002131 >> v1:172.21.15.47:6789/0 conn(0x7fc33c145120 legacy=0x7fc33c061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 -- 172.21.15.47:0/908002131 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c145a10 con 0x7fc33c0b0500 2020-07-18T03:29:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc33bfff700 1 --2- 172.21.15.47:0/908002131 >> v2:172.21.15.18:3300/0 conn(0x7fc33c148e30 0x7fc33c1492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.486+0000 7fc341138700 1 --2- 172.21.15.47:0/908002131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0b0500 0x7fc33c0b0970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fc3340032b0 tx=0x7fc334009270).ready entity=mon.1 client_cookie=83f5a624d909070 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc33b7fe700 1 -- 172.21.15.47:0/908002131 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33400bc00 con 0x7fc33c0b0500 2020-07-18T03:29:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc33b7fe700 1 -- 172.21.15.47:0/908002131 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33400bd60 con 0x7fc33c0b0500 2020-07-18T03:29:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc33b7fe700 1 -- 172.21.15.47:0/908002131 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc334011400 con 0x7fc33c0b0500 2020-07-18T03:29:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 -- 172.21.15.47:0/908002131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0b0500 msgr2=0x7fc33c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 --2- 172.21.15.47:0/908002131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0b0500 0x7fc33c0b0970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fc3340032b0 tx=0x7fc334009270).stop 2020-07-18T03:29:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 -- 172.21.15.47:0/908002131 shutdown_connections 2020-07-18T03:29:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 --2- 172.21.15.47:0/908002131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0b0500 0x7fc33c0b0970 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 --2- 172.21.15.47:0/908002131 >> v2:172.21.15.18:3300/0 conn(0x7fc33c148e30 0x7fc33c1492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 -- 172.21.15.47:0/908002131 >> 172.21.15.47:0/908002131 conn(0x7fc33c0b93c0 msgr2=0x7fc33c0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 -- 172.21.15.47:0/908002131 shutdown_connections 2020-07-18T03:29:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.487+0000 7fc342b9b700 1 -- 172.21.15.47:0/908002131 wait complete. 2020-07-18T03:29:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.488+0000 7fc342b9b700 1 Processor -- start 2020-07-18T03:29:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.488+0000 7fc342b9b700 1 -- start start 2020-07-18T03:29:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc342b9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc33c145120 0x7fc33c0b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc342b9b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c156280 con 0x7fc33c12b340 2020-07-18T03:29:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc342b9b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c156520 con 0x7fc33c148e30 2020-07-18T03:29:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc342b9b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c0b8aa0 con 0x7fc33c145120 2020-07-18T03:29:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc340937700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc33c145120 0x7fc33c0b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc33bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc33c148e30 0x7fc33c12ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42588/0 (socket says 172.21.15.47:42588) 2020-07-18T03:29:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.489+0000 7fc33bfff700 1 -- 172.21.15.47:0/2559754438 learned_addr learned my addr 172.21.15.47:0/2559754438 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc340937700 1 -- 172.21.15.47:0/2559754438 >> v1:172.21.15.47:6789/0 conn(0x7fc33c148e30 legacy=0x7fc33c12ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc340937700 1 -- 172.21.15.47:0/2559754438 >> v1:172.21.15.18:6789/0 conn(0x7fc33c12b340 legacy=0x7fc33c12d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494812109 0 0) 0x7fc33c156520 con 0x7fc33c148e30 2020-07-18T03:29:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc340937700 1 -- 172.21.15.47:0/2559754438 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0b8be0 con 0x7fc33c145120 2020-07-18T03:29:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc340937700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c145120 0x7fc33c0b85b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fc33c05d300 tx=0x7fc330007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3300072e0 con 0x7fc33c145120 2020-07-18T03:29:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc330005530 con 0x7fc33c145120 2020-07-18T03:29:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c158e30 con 0x7fc33c145120 2020-07-18T03:29:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc330005890 con 0x7fc33c145120 2020-07-18T03:29:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.490+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c1558b0 con 0x7fc33c145120 2020-07-18T03:29:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.491+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c057450 con 0x7fc33c145120 2020-07-18T03:29:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.491+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc330010070 con 0x7fc33c145120 2020-07-18T03:29:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.491+0000 7fc338ff9700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc31404fd90 0x7fc314052250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.491+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc330066ee0 con 0x7fc33c145120 2020-07-18T03:29:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.491+0000 7fc33bfff700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc31404fd90 0x7fc314052250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.492+0000 7fc33bfff700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc31404fd90 0x7fc314052250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fc328002f30 tx=0x7fc328008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.495+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc33003f8d0 con 0x7fc33c145120 2020-07-18T03:29:52.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:52.668+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fc33c1566b0 con 0x7fc33c145120 2020-07-18T03:29:53.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.470+0000 7fc338ff9700 1 -- 172.21.15.47:0/2559754438 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc330018080 con 0x7fc33c145120 2020-07-18T03:29:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc31404fd90 msgr2=0x7fc314052250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc31404fd90 0x7fc314052250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fc328002f30 tx=0x7fc328008040).stop 2020-07-18T03:29:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c145120 msgr2=0x7fc33c0b85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c145120 0x7fc33c0b85b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fc33c05d300 tx=0x7fc330007ad0).stop 2020-07-18T03:29:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 shutdown_connections 2020-07-18T03:29:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c145120 0x7fc33c0b85b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 --2- 172.21.15.47:0/2559754438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc31404fd90 0x7fc314052250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 >> 172.21.15.47:0/2559754438 conn(0x7fc33c0b93c0 msgr2=0x7fc33c0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 shutdown_connections 2020-07-18T03:29:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.472+0000 7fc342b9b700 1 -- 172.21.15.47:0/2559754438 wait complete. 2020-07-18T03:29:53.477 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2020-07-18T03:29:53.487 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2020-07-18T03:29:53.488 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-07-18T03:29:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.607+0000 7f4c40c83700 1 Processor -- start 2020-07-18T03:29:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.607+0000 7f4c40c83700 1 -- start start 2020-07-18T03:29:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.607+0000 7f4c40c83700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4c3c147a00 0x7f4c3c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c40c83700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c147330 con 0x7f4c3c1433e0 2020-07-18T03:29:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c40c83700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0b57e0 con 0x7f4c3c0ad100 2020-07-18T03:29:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c40c83700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0ae530 con 0x7f4c3c147a00 2020-07-18T03:29:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4c3c0ad100 0x7f4c3c0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42594/0 (socket says 172.21.15.47:42594) 2020-07-18T03:29:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3a59c700 1 -- 172.21.15.47:0/3856065024 learned_addr learned my addr 172.21.15.47:0/3856065024 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438448942 0 0) 0x7f4c3c0b57e0 con 0x7f4c3c0ad100 2020-07-18T03:29:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c39d9b700 1 --2- 172.21.15.47:0/3856065024 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c147a00 0x7f4c3c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c1c003240 con 0x7f4c3c0ad100 2020-07-18T03:29:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2214747452 0 0) 0x7f4c1c003240 con 0x7f4c3c0ad100 2020-07-18T03:29:53.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c147a00 msgr2=0x7f4c3c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:53.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 --2- 172.21.15.47:0/3856065024 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c147a00 0x7f4c3c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 >> v1:172.21.15.18:6789/0 conn(0x7f4c3c1433e0 legacy=0x7f4c3c143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0b5a80 con 0x7f4c3c0ad100 2020-07-18T03:29:53.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4c240016d0 con 0x7f4c3c0ad100 2020-07-18T03:29:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 >> v1:172.21.15.47:6789/0 conn(0x7f4c3c0ad100 legacy=0x7f4c3c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c1c003780 0x7f4c1c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c1c0044e0 0x7f4c1c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c1c004e50 0x7f4c1c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0ae530 con 0x7f4c1c0044e0 2020-07-18T03:29:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c00bc20 con 0x7f4c1c004e50 2020-07-18T03:29:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.608+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c00bd60 con 0x7f4c1c003780 2020-07-18T03:29:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c1c004e50 0x7f4c1c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3a59c700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c1c003780 0x7f4c1c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c39d9b700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c1c0044e0 0x7f4c1c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c1c003780 msgr2=0x7f4c1c003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c1c003780 0x7f4c1c003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c1c0044e0 msgr2=0x7f4c1c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c1c0044e0 0x7f4c1c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3856065024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c003240 con 0x7f4c1c004e50 2020-07-18T03:29:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3856065024 reap_dead start 2020-07-18T03:29:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c1c004e50 0x7f4c1c00b730 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f4c3c05ca30 tx=0x7f4c34007ef0).ready entity=mon.1 client_cookie=dcb4c8274eb184cc server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3400c9a0 con 0x7f4c1c004e50 2020-07-18T03:29:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400cb00 con 0x7f4c1c004e50 2020-07-18T03:29:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c3959a700 1 -- 172.21.15.47:0/3856065024 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3400ce60 con 0x7f4c1c004e50 2020-07-18T03:29:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 -- 172.21.15.47:0/3856065024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c1c004e50 msgr2=0x7f4c1c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c1c004e50 0x7f4c1c00b730 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f4c3c05ca30 tx=0x7f4c34007ef0).stop 2020-07-18T03:29:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 -- 172.21.15.47:0/3856065024 shutdown_connections 2020-07-18T03:29:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3856065024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c1c004e50 0x7f4c1c00b730 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 -- 172.21.15.47:0/3856065024 >> 172.21.15.47:0/3856065024 conn(0x7f4c3c0b9340 msgr2=0x7f4c3c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 -- 172.21.15.47:0/3856065024 shutdown_connections 2020-07-18T03:29:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.609+0000 7f4c40c83700 1 -- 172.21.15.47:0/3856065024 wait complete. 2020-07-18T03:29:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 Processor -- start 2020-07-18T03:29:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 -- start start 2020-07-18T03:29:53.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4c3c0b3ee0 0x7f4c3c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c3c13ef10 0x7f4c3c141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0b1cd0 con 0x7f4c3c0b3b00 2020-07-18T03:29:53.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c156930 con 0x7f4c3c0b3ee0 2020-07-18T03:29:53.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c40c83700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c156bd0 con 0x7f4c3c13ef10 2020-07-18T03:29:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.611+0000 7f4c3a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4c3c0b3b00 0x7f4c3c13aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42606/0 (socket says 172.21.15.47:42606) 2020-07-18T03:29:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3a59c700 1 -- 172.21.15.47:0/3286592086 learned_addr learned my addr 172.21.15.47:0/3286592086 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c3c13ef10 0x7f4c3c141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c39d9b700 1 --2- 172.21.15.47:0/3286592086 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c0b3ee0 0x7f4c3c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436876897 0 0) 0x7f4c3c0b1cd0 con 0x7f4c3c0b3b00 2020-07-18T03:29:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c18003240 con 0x7f4c3c0b3b00 2020-07-18T03:29:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2800582182 0 0) 0x7f4c18003240 con 0x7f4c3c0b3b00 2020-07-18T03:29:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3286592086 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c0b3ee0 msgr2=0x7f4c3c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3286592086 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c0b3ee0 0x7f4c3c1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3286592086 >> v1:172.21.15.47:6789/0 conn(0x7f4c3c0b3b00 legacy=0x7f4c3c13aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3ad9d700 1 -- 172.21.15.47:0/3286592086 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c156e70 con 0x7f4c3c13ef10 2020-07-18T03:29:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c3ad9d700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c3c13ef10 0x7f4c3c141280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4c24007830 tx=0x7f4c34016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c340011d0 con 0x7f4c3c13ef10 2020-07-18T03:29:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24007e90 con 0x7f4c3c13ef10 2020-07-18T03:29:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c34007870 con 0x7f4c3c13ef10 2020-07-18T03:29:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c13b1e0 con 0x7f4c3c13ef10 2020-07-18T03:29:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.612+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c141770 con 0x7f4c3c13ef10 2020-07-18T03:29:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.613+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c057450 con 0x7f4c3c13ef10 2020-07-18T03:29:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.614+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4c34026030 con 0x7f4c3c13ef10 2020-07-18T03:29:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.614+0000 7f4c327fc700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4c18050330 0x7f4c180527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.614+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4c34068e30 con 0x7f4c3c13ef10 2020-07-18T03:29:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.614+0000 7f4c3a59c700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4c18050330 0x7f4c180527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.614+0000 7f4c3a59c700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4c18050330 0x7f4c180527f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f4c240077f0 tx=0x7f4c3c0b5d20).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:53.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.618+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4c34041900 con 0x7f4c3c13ef10 2020-07-18T03:29:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.830+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4c3c157000 con 0x7f4c3c13ef10 2020-07-18T03:29:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.830+0000 7f4c327fc700 1 -- 172.21.15.47:0/3286592086 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f4c34024020 con 0x7f4c3c13ef10 2020-07-18T03:29:53.833 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:53.833 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:29:53.833 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2020-07-18T03:29:53.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.834 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2020-07-18T03:29:53.834 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g==", 2020-07-18T03:29:53.834 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.835 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:53.835 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.835 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.835 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.836 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2020-07-18T03:29:53.836 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ==", 2020-07-18T03:29:53.836 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.836 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:53.837 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.837 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.837 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.838 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2020-07-18T03:29:53.838 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA==", 2020-07-18T03:29:53.839 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.839 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.839 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.839 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.839 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.840 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2020-07-18T03:29:53.840 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ==", 2020-07-18T03:29:53.840 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.841 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.842 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.842 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.842 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.843 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2020-07-18T03:29:53.843 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA==", 2020-07-18T03:29:53.843 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.843 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.844 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.844 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.845 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2020-07-18T03:29:53.845 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ==", 2020-07-18T03:29:53.845 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.845 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.845 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.846 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.846 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2020-07-18T03:29:53.847 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ==", 2020-07-18T03:29:53.847 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.847 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.847 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.847 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.847 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.848 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2020-07-18T03:29:53.848 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA==", 2020-07-18T03:29:53.848 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.849 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.849 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.849 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.850 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2020-07-18T03:29:53.850 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg==", 2020-07-18T03:29:53.850 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.850 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.851 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.851 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.853 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.854 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2020-07-18T03:29:53.855 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg==", 2020-07-18T03:29:53.855 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.855 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.855 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.856 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.856 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.857 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2020-07-18T03:29:53.857 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw==", 2020-07-18T03:29:53.857 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.858 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:53.858 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:53.858 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:53.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.859 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2020-07-18T03:29:53.859 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g==", 2020-07-18T03:29:53.859 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.859 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:53.860 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:53.860 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:53.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.861 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2020-07-18T03:29:53.861 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w==", 2020-07-18T03:29:53.861 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.862 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:53.862 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:53.863 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:53.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.864 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2020-07-18T03:29:53.864 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog==", 2020-07-18T03:29:53.864 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.864 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:29:53.864 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:29:53.865 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:29:53.865 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.865 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.865 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2020-07-18T03:29:53.866 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfi9GyDxAACUH4y/92DOhKqZyh9ejEnA==", 2020-07-18T03:29:53.866 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.866 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:29:53.866 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:29:53.866 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:29:53.867 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.867 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.867 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2020-07-18T03:29:53.870 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ==", 2020-07-18T03:29:53.870 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.871 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:29:53.871 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:29:53.871 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.871 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:53.873 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:53.873 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2020-07-18T03:29:53.874 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw==", 2020-07-18T03:29:53.874 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:29:53.874 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:29:53.874 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:29:53.874 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:29:53.875 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.875 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:53.875 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:29:53.875 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:29:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4c18050330 msgr2=0x7f4c180527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4c18050330 0x7f4c180527f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f4c240077f0 tx=0x7f4c3c0b5d20).stop 2020-07-18T03:29:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c3c13ef10 msgr2=0x7f4c3c141280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:53.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c3c13ef10 0x7f4c3c141280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4c24007830 tx=0x7f4c34016000).stop 2020-07-18T03:29:53.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 shutdown_connections 2020-07-18T03:29:53.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4c3c13ef10 0x7f4c3c141280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3286592086 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4c18050330 0x7f4c180527f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 --2- 172.21.15.47:0/3286592086 >> v2:172.21.15.18:3300/0 conn(0x7f4c3c0b3ee0 0x7f4c3c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 >> 172.21.15.47:0/3286592086 conn(0x7f4c3c0b9340 msgr2=0x7f4c3c0ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:53.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 shutdown_connections 2020-07-18T03:29:53.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.832+0000 7f4c40c83700 1 -- 172.21.15.47:0/3286592086 wait complete. 2020-07-18T03:29:53.880 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.992+0000 7f8c28ed5700 1 Processor -- start 2020-07-18T03:29:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.992+0000 7f8c28ed5700 1 -- start start 2020-07-18T03:29:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c28ed5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c240b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c28ed5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c240b7420 0x7f8c240b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c28ed5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c240b95a0 con 0x7f8c240b82f0 2020-07-18T03:29:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c28ed5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c240b5a90 con 0x7f8c240b42a0 2020-07-18T03:29:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c28ed5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c240b5c80 con 0x7f8c240b7420 2020-07-18T03:29:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c21d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c240b7420 0x7f8c240b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c22d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8c240b82f0 0x7f8c24061800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42616/0 (socket says 172.21.15.47:42616) 2020-07-18T03:29:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.993+0000 7f8c22d9d700 1 -- 172.21.15.47:0/3721687159 learned_addr learned my addr 172.21.15.47:0/3721687159 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c2259c700 1 --2- 172.21.15.47:0/3721687159 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c240b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c21d9b700 1 -- 172.21.15.47:0/3721687159 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 msgr2=0x7f8c240b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c21d9b700 1 --2- 172.21.15.47:0/3721687159 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c240b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c21d9b700 1 -- 172.21.15.47:0/3721687159 >> v1:172.21.15.47:6789/0 conn(0x7f8c240b82f0 legacy=0x7f8c24061800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c2159a700 1 -- 172.21.15.47:0/3721687159 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689771849 0 0) 0x7f8c240b95a0 con 0x7f8c240b82f0 2020-07-18T03:29:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c21d9b700 1 -- 172.21.15.47:0/3721687159 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c240b5f20 con 0x7f8c240b7420 2020-07-18T03:29:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c2259c700 1 --2- 172.21.15.47:0/3721687159 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c240b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:29:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c21d9b700 1 --2- 172.21.15.47:0/3721687159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c240b7420 0x7f8c240b7e00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8c14002f30 tx=0x7f8c14006dd0).ready entity=mon.1 client_cookie=1aa109071a2bcfa3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c2159a700 1 -- 172.21.15.47:0/3721687159 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c140097f0 con 0x7f8c240b7420 2020-07-18T03:29:54.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c2159a700 1 -- 172.21.15.47:0/3721687159 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c14009950 con 0x7f8c240b7420 2020-07-18T03:29:54.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c2159a700 1 -- 172.21.15.47:0/3721687159 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c14009cb0 con 0x7f8c240b7420 2020-07-18T03:29:54.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 -- 172.21.15.47:0/3721687159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c240b7420 msgr2=0x7f8c240b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/3721687159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c240b7420 0x7f8c240b7e00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8c14002f30 tx=0x7f8c14006dd0).stop 2020-07-18T03:29:54.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 -- 172.21.15.47:0/3721687159 shutdown_connections 2020-07-18T03:29:54.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/3721687159 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c240b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/3721687159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c240b7420 0x7f8c240b7e00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 -- 172.21.15.47:0/3721687159 >> 172.21.15.47:0/3721687159 conn(0x7f8c2405f5a0 msgr2=0x7f8c2405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 -- 172.21.15.47:0/3721687159 shutdown_connections 2020-07-18T03:29:54.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.994+0000 7f8c28ed5700 1 -- 172.21.15.47:0/3721687159 wait complete. 2020-07-18T03:29:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c28ed5700 1 Processor -- start 2020-07-18T03:29:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c28ed5700 1 -- start start 2020-07-18T03:29:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c28ed5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c241595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c28ed5700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c24159ac0 con 0x7f8c240b82f0 2020-07-18T03:29:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c28ed5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2414a870 con 0x7f8c24146610 2020-07-18T03:29:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c28ed5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2414ab10 con 0x7f8c240b42a0 2020-07-18T03:29:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c22d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8c24146610 0x7f8c2413c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42622/0 (socket says 172.21.15.47:42622) 2020-07-18T03:29:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.996+0000 7f8c22d9d700 1 -- 172.21.15.47:0/2322945160 learned_addr learned my addr 172.21.15.47:0/2322945160 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259430880 0 0) 0x7f8c2414a870 con 0x7f8c24146610 2020-07-18T03:29:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c00003240 con 0x7f8c24146610 2020-07-18T03:29:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c2259c700 1 --2- 172.21.15.47:0/2322945160 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c241595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3808651973 0 0) 0x7f8c00003240 con 0x7f8c24146610 2020-07-18T03:29:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8c2414a870 con 0x7f8c24146610 2020-07-18T03:29:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8c1c001be0 con 0x7f8c24146610 2020-07-18T03:29:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 msgr2=0x7f8c241595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 --2- 172.21.15.47:0/2322945160 >> v2:172.21.15.18:3300/0 conn(0x7f8c240b42a0 0x7f8c241595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 >> v1:172.21.15.47:6789/0 conn(0x7f8c24146610 legacy=0x7f8c2413c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 >> v1:172.21.15.18:6789/0 conn(0x7f8c240b82f0 legacy=0x7f8c2413d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c00003800 0x7f8c00003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8c000044e0 0x7f8c00004940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c00004e30 0x7f8c0000b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2414ab10 con 0x7f8c00003800 2020-07-18T03:29:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0000bc20 con 0x7f8c00004e30 2020-07-18T03:29:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0000bd90 con 0x7f8c000044e0 2020-07-18T03:29:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c22d9d700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c00004e30 0x7f8c0000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c2259c700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c00003800 0x7f8c00003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c21d9b700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8c000044e0 0x7f8c00004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c22d9d700 1 -- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8c000044e0 msgr2=0x7f8c00004940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c22d9d700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8c000044e0 0x7f8c00004940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c22d9d700 1 -- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c00003800 msgr2=0x7f8c00003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.997+0000 7f8c22d9d700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c00003800 0x7f8c00003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c22d9d700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2414adb0 con 0x7f8c00004e30 2020-07-18T03:29:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2413c8a0 con 0x7f8c00004e30 2020-07-18T03:29:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c22d9d700 1 -- 172.21.15.47:0/2322945160 reap_dead start 2020-07-18T03:29:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2414c4a0 con 0x7f8c00004e30 2020-07-18T03:29:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c22d9d700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c00004e30 0x7f8c0000b730 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f8c1c0058d0 tx=0x7f8c1c007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1c005b80 con 0x7f8c00004e30 2020-07-18T03:29:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c005ce0 con 0x7f8c00004e30 2020-07-18T03:29:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1c007300 con 0x7f8c00004e30 2020-07-18T03:29:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.998+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c240be670 con 0x7f8c00004e30 2020-07-18T03:29:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.999+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8c1c010030 con 0x7f8c00004e30 2020-07-18T03:29:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.999+0000 7f8c1a7fc700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8c00053f50 0x7f8c00056410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.999+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8c1c068060 con 0x7f8c00004e30 2020-07-18T03:29:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.999+0000 7f8c2259c700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8c00053f50 0x7f8c00056410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:53.999+0000 7f8c2259c700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8c00053f50 0x7f8c00056410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f8c0c00d000 tx=0x7f8c0c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.002+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8c1c040b30 con 0x7f8c00004e30 2020-07-18T03:29:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.159+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c24057450 con 0x7f8c00004e30 2020-07-18T03:29:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.160+0000 7f8c1a7fc700 1 -- 172.21.15.47:0/2322945160 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13417 (secure 0 0 0) 0x7f8c1c044150 con 0x7f8c00004e30 2020-07-18T03:29:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8c00053f50 msgr2=0x7f8c00056410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8c00053f50 0x7f8c00056410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f8c0c00d000 tx=0x7f8c0c008040).stop 2020-07-18T03:29:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c00004e30 msgr2=0x7f8c0000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c00004e30 0x7f8c0000b730 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f8c1c0058d0 tx=0x7f8c1c007b80).stop 2020-07-18T03:29:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 shutdown_connections 2020-07-18T03:29:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8c00053f50 0x7f8c00056410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 --2- 172.21.15.47:0/2322945160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c00004e30 0x7f8c0000b730 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 >> 172.21.15.47:0/2322945160 conn(0x7f8c2405f5a0 msgr2=0x7f8c240bcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 shutdown_connections 2020-07-18T03:29:54.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.162+0000 7f8c28ed5700 1 -- 172.21.15.47:0/2322945160 wait complete. 2020-07-18T03:29:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:54.173 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":48,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:53.468310+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:54.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-07-18T03:29:54.175 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:29:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.301+0000 7f6bfe1e1700 1 Processor -- start 2020-07-18T03:29:54.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.301+0000 7f6bfe1e1700 1 -- start start 2020-07-18T03:29:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bfe1e1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bf80ad330 0x7f6bf8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bfe1e1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b59b0 0x7f6bf80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bfe1e1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf8147de0 con 0x7f6bf80b6300 2020-07-18T03:29:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bfe1e1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf80ae000 con 0x7f6bf80ad330 2020-07-18T03:29:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bfe1e1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf8148010 con 0x7f6bf80b59b0 2020-07-18T03:29:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bf77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bf80ad330 0x7f6bf8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bf7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6bf80b6300 0x7f6bf80b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42638/0 (socket says 172.21.15.47:42638) 2020-07-18T03:29:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bf7fff700 1 -- 172.21.15.47:0/3562560299 learned_addr learned my addr 172.21.15.47:0/3562560299 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.302+0000 7f6bf6ffd700 1 --2- 172.21.15.47:0/3562560299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b59b0 0x7f6bf80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf77fe700 1 -- 172.21.15.47:0/3562560299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b59b0 msgr2=0x7f6bf80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf77fe700 1 --2- 172.21.15.47:0/3562560299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b59b0 0x7f6bf80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf77fe700 1 -- 172.21.15.47:0/3562560299 >> v1:172.21.15.47:6789/0 conn(0x7f6bf80b6300 legacy=0x7f6bf80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf77fe700 1 -- 172.21.15.47:0/3562560299 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80ad710 con 0x7f6bf80ad330 2020-07-18T03:29:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf67fc700 1 -- 172.21.15.47:0/3562560299 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526431468 0 0) 0x7f6bf8147de0 con 0x7f6bf80b6300 2020-07-18T03:29:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf77fe700 1 --2- 172.21.15.47:0/3562560299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf80ad330 0x7f6bf8143640 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f6be8002f30 tx=0x7f6be8006dd0).ready entity=mon.1 client_cookie=9e6417e5812bc6da server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf67fc700 1 -- 172.21.15.47:0/3562560299 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6be80097f0 con 0x7f6bf80ad330 2020-07-18T03:29:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf67fc700 1 -- 172.21.15.47:0/3562560299 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be8009950 con 0x7f6bf80ad330 2020-07-18T03:29:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bf67fc700 1 -- 172.21.15.47:0/3562560299 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6be8009cb0 con 0x7f6bf80ad330 2020-07-18T03:29:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3562560299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf80ad330 msgr2=0x7f6bf8143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3562560299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf80ad330 0x7f6bf8143640 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f6be8002f30 tx=0x7f6be8006dd0).stop 2020-07-18T03:29:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3562560299 shutdown_connections 2020-07-18T03:29:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3562560299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf80ad330 0x7f6bf8143640 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3562560299 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b59b0 0x7f6bf80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3562560299 >> 172.21.15.47:0/3562560299 conn(0x7f6bf805f5a0 msgr2=0x7f6bf805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3562560299 shutdown_connections 2020-07-18T03:29:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.303+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3562560299 wait complete. 2020-07-18T03:29:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 Processor -- start 2020-07-18T03:29:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 -- start start 2020-07-18T03:29:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b6300 0x7f6bf81279d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bf8066b30 0x7f6bf8127ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf81475b0 con 0x7f6bf80b59b0 2020-07-18T03:29:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf81283b0 con 0x7f6bf8066b30 2020-07-18T03:29:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bfe1e1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf81477e0 con 0x7f6bf80b6300 2020-07-18T03:29:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bf7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bf8066b30 0x7f6bf8127ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bf7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bf8066b30 0x7f6bf8127ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33324/0 (socket says 172.21.15.47:33324) 2020-07-18T03:29:54.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.305+0000 7f6bf7fff700 1 -- 172.21.15.47:0/3264790050 learned_addr learned my addr 172.21.15.47:0/3264790050 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:54.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bf6ffd700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b6300 0x7f6bf81279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bf7fff700 1 -- 172.21.15.47:0/3264790050 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b6300 msgr2=0x7f6bf81279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bf7fff700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b6300 0x7f6bf81279d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bf7fff700 1 -- 172.21.15.47:0/3264790050 >> v1:172.21.15.18:6789/0 conn(0x7f6bf80b59b0 legacy=0x7f6bf8146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bf7fff700 1 -- 172.21.15.47:0/3264790050 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8147a80 con 0x7f6bf8066b30 2020-07-18T03:29:54.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bf7fff700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf8066b30 0x7f6bf8127ec0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f6bf0000f00 tx=0x7f6bf0007ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf0005b80 con 0x7f6bf8066b30 2020-07-18T03:29:54.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0005ce0 con 0x7f6bf8066b30 2020-07-18T03:29:54.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf81394a0 con 0x7f6bf8066b30 2020-07-18T03:29:54.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf0007300 con 0x7f6bf8066b30 2020-07-18T03:29:54.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.306+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf81452c0 con 0x7f6bf8066b30 2020-07-18T03:29:54.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.307+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf814f0d0 con 0x7f6bf8066b30 2020-07-18T03:29:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.307+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6bf0010030 con 0x7f6bf8066b30 2020-07-18T03:29:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.307+0000 7f6bd77fe700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bd0050010 0x7f6bd00524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.308+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6bf00679d0 con 0x7f6bf8066b30 2020-07-18T03:29:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.308+0000 7f6bf77fe700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bd0050010 0x7f6bd00524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.308+0000 7f6bf77fe700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bd0050010 0x7f6bd00524d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6bf8066a10 tx=0x7f6be8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.311+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6bf00404a0 con 0x7f6bf8066b30 2020-07-18T03:29:54.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.470+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6bf805b120 con 0x7f6bf8066b30 2020-07-18T03:29:54.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.471+0000 7f6bd77fe700 1 -- 172.21.15.47:0/3264790050 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v48) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6bf0043ac0 con 0x7f6bf8066b30 2020-07-18T03:29:54.474 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:29:54.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bd0050010 msgr2=0x7f6bd00524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bd0050010 0x7f6bd00524d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6bf8066a10 tx=0x7f6be8015000).stop 2020-07-18T03:29:54.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf8066b30 msgr2=0x7f6bf8127ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf8066b30 0x7f6bf8127ec0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f6bf0000f00 tx=0x7f6bf0007ce0).stop 2020-07-18T03:29:54.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 shutdown_connections 2020-07-18T03:29:54.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bd0050010 0x7f6bd00524d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bf8066b30 0x7f6bf8127ec0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 --2- 172.21.15.47:0/3264790050 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bf80b6300 0x7f6bf81279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.472+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 >> 172.21.15.47:0/3264790050 conn(0x7f6bf805f5a0 msgr2=0x7f6bf80b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.473+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 shutdown_connections 2020-07-18T03:29:54.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.473+0000 7f6bfe1e1700 1 -- 172.21.15.47:0/3264790050 wait complete. 2020-07-18T03:29:54.485 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:29:54.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.604+0000 7fd562e18700 1 Processor -- start 2020-07-18T03:29:54.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.604+0000 7fd562e18700 1 -- start start 2020-07-18T03:29:54.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd562e18700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c143c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd562e18700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd55c05f760 0x7fd55c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd562e18700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c0ad810 con 0x7fd55c141290 2020-07-18T03:29:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd562e18700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c0adab0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd562e18700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c141690 con 0x7fd55c05f760 2020-07-18T03:29:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd561615700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd55c05f760 0x7fd55c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd561615700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd55c05f760 0x7fd55c140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33328/0 (socket says 172.21.15.47:33328) 2020-07-18T03:29:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.605+0000 7fd561615700 1 -- 172.21.15.47:0/3398282905 learned_addr learned my addr 172.21.15.47:0/3398282905 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561e16700 1 --2- 172.21.15.47:0/3398282905 >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561615700 1 -- 172.21.15.47:0/3398282905 >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 msgr2=0x7fd55c143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561615700 1 --2- 172.21.15.47:0/3398282905 >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561615700 1 -- 172.21.15.47:0/3398282905 >> v1:172.21.15.18:6789/0 conn(0x7fd55c141290 legacy=0x7fd55c144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561615700 1 -- 172.21.15.47:0/3398282905 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0606c0 con 0x7fd55c05f760 2020-07-18T03:29:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561e16700 1 --2- 172.21.15.47:0/3398282905 >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd561615700 1 --2- 172.21.15.47:0/3398282905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c05f760 0x7fd55c140da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd5580032b0 tx=0x7fd558009270).ready entity=mon.1 client_cookie=d0b6dbeae7683982 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd560e14700 1 -- 172.21.15.47:0/3398282905 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55800bc00 con 0x7fd55c05f760 2020-07-18T03:29:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd560e14700 1 -- 172.21.15.47:0/3398282905 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800bd60 con 0x7fd55c05f760 2020-07-18T03:29:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd560e14700 1 -- 172.21.15.47:0/3398282905 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd558011400 con 0x7fd55c05f760 2020-07-18T03:29:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 -- 172.21.15.47:0/3398282905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c05f760 msgr2=0x7fd55c140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 --2- 172.21.15.47:0/3398282905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c05f760 0x7fd55c140da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd5580032b0 tx=0x7fd558009270).stop 2020-07-18T03:29:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 -- 172.21.15.47:0/3398282905 shutdown_connections 2020-07-18T03:29:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 --2- 172.21.15.47:0/3398282905 >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 --2- 172.21.15.47:0/3398282905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c05f760 0x7fd55c140da0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 -- 172.21.15.47:0/3398282905 >> 172.21.15.47:0/3398282905 conn(0x7fd55c0b91d0 msgr2=0x7fd55c0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 -- 172.21.15.47:0/3398282905 shutdown_connections 2020-07-18T03:29:54.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.606+0000 7fd562e18700 1 -- 172.21.15.47:0/3398282905 wait complete. 2020-07-18T03:29:54.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 Processor -- start 2020-07-18T03:29:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 -- start start 2020-07-18T03:29:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd55c061b70 0x7fd55c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c12cd20 con 0x7fd55c141290 2020-07-18T03:29:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c12aeb0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.608+0000 7fd562e18700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c12ca50 con 0x7fd55c061b70 2020-07-18T03:29:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561e16700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd55c0ad0d0 0x7fd55c148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561615700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fd55c141290 0x7fd55c1587b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:33188/0 (socket says 172.21.15.47:33188) 2020-07-18T03:29:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561615700 1 -- 172.21.15.47:0/114162895 learned_addr learned my addr 172.21.15.47:0/114162895 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd562617700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd55c061b70 0x7fd55c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211079080 0 0) 0x7fd55c12cd20 con 0x7fd55c141290 2020-07-18T03:29:54.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd534003240 con 0x7fd55c141290 2020-07-18T03:29:54.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561e16700 1 -- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd55c061b70 msgr2=0x7fd55c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561e16700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd55c061b70 0x7fd55c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561e16700 1 -- 172.21.15.47:0/114162895 >> v1:172.21.15.18:6789/0 conn(0x7fd55c141290 legacy=0x7fd55c1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561e16700 1 -- 172.21.15.47:0/114162895 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c148d70 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd562617700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd55c061b70 0x7fd55c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd561e16700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55c0ad0d0 0x7fd55c148880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fd55c149640 tx=0x7fd54c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54c0127e0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c012940 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c149170 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.609+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54c012ca0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.610+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c159570 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.610+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c0690d0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.611+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd54c013480 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.611+0000 7fd549ffb700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd534050300 0x7fd5340527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.611+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd54c067fd0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.611+0000 7fd561615700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd534050300 0x7fd5340527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.611+0000 7fd561615700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd534050300 0x7fd5340527c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd558006ae0 tx=0x7fd558016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.615+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd54c040960 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.770+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd55c12aff0 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.772+0000 7fd549ffb700 1 -- 172.21.15.47:0/114162895 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd54c043f80 con 0x7fd55c0ad0d0 2020-07-18T03:29:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.772+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd534050300 msgr2=0x7fd5340527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.772+0000 7fd562e18700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd534050300 0x7fd5340527c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd558006ae0 tx=0x7fd558016000).stop 2020-07-18T03:29:54.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.772+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55c0ad0d0 msgr2=0x7fd55c148880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.772+0000 7fd562e18700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55c0ad0d0 0x7fd55c148880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fd55c149640 tx=0x7fd54c008040).stop 2020-07-18T03:29:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 shutdown_connections 2020-07-18T03:29:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd534050300 0x7fd5340527c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd55c061b70 0x7fd55c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 --2- 172.21.15.47:0/114162895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55c0ad0d0 0x7fd55c148880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 >> 172.21.15.47:0/114162895 conn(0x7fd55c0b91d0 msgr2=0x7fd55c05d350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 shutdown_connections 2020-07-18T03:29:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.773+0000 7fd562e18700 1 -- 172.21.15.47:0/114162895 wait complete. 2020-07-18T03:29:54.790 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:29:54.792 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-07-18T03:29:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.906+0000 7f894af4d700 1 Processor -- start 2020-07-18T03:29:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.906+0000 7f894af4d700 1 -- start start 2020-07-18T03:29:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f894af4d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f89440ad100 0x7f89440b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f894af4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8944147a00 0x7f8944147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f894af4d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8944147330 con 0x7f894405f5a0 2020-07-18T03:29:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f894af4d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89440ae530 con 0x7f89440ad100 2020-07-18T03:29:54.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f894af4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89440b57e0 con 0x7f8944147a00 2020-07-18T03:29:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8944147a00 0x7f8944147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8944147a00 0x7f8944147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33340/0 (socket says 172.21.15.47:33340) 2020-07-18T03:29:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 -- 172.21.15.47:0/2431058853 learned_addr learned my addr 172.21.15.47:0/2431058853 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8948ce9700 1 --2- 172.21.15.47:0/2431058853 >> v2:172.21.15.18:3300/0 conn(0x7f89440ad100 0x7f89440b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 -- 172.21.15.47:0/2431058853 >> v2:172.21.15.18:3300/0 conn(0x7f89440ad100 msgr2=0x7f89440b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 --2- 172.21.15.47:0/2431058853 >> v2:172.21.15.18:3300/0 conn(0x7f89440ad100 0x7f89440b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 -- 172.21.15.47:0/2431058853 >> v1:172.21.15.18:6789/0 conn(0x7f894405f5a0 legacy=0x7f894405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 -- 172.21.15.47:0/2431058853 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89440b5a80 con 0x7f8944147a00 2020-07-18T03:29:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f8943fff700 1 --2- 172.21.15.47:0/2431058853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944147a00 0x7f8944147e40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f894405ca30 tx=0x7f8938006e90).ready entity=mon.1 client_cookie=4c640428430b0084 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f89437fe700 1 -- 172.21.15.47:0/2431058853 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8938009930 con 0x7f8944147a00 2020-07-18T03:29:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f89437fe700 1 -- 172.21.15.47:0/2431058853 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938009a90 con 0x7f8944147a00 2020-07-18T03:29:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.907+0000 7f89437fe700 1 -- 172.21.15.47:0/2431058853 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8938009e30 con 0x7f8944147a00 2020-07-18T03:29:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 -- 172.21.15.47:0/2431058853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944147a00 msgr2=0x7f8944147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 --2- 172.21.15.47:0/2431058853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944147a00 0x7f8944147e40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f894405ca30 tx=0x7f8938006e90).stop 2020-07-18T03:29:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 -- 172.21.15.47:0/2431058853 shutdown_connections 2020-07-18T03:29:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 --2- 172.21.15.47:0/2431058853 >> v2:172.21.15.18:3300/0 conn(0x7f89440ad100 0x7f89440b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 --2- 172.21.15.47:0/2431058853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944147a00 0x7f8944147e40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 -- 172.21.15.47:0/2431058853 >> 172.21.15.47:0/2431058853 conn(0x7f89440b9340 msgr2=0x7f89440b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 -- 172.21.15.47:0/2431058853 shutdown_connections 2020-07-18T03:29:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.908+0000 7f894af4d700 1 -- 172.21.15.47:0/2431058853 wait complete. 2020-07-18T03:29:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.909+0000 7f894af4d700 1 Processor -- start 2020-07-18T03:29:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.909+0000 7f894af4d700 1 -- start start 2020-07-18T03:29:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f894af4d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89440ad100 0x7f894413c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f894af4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8944065f30 0x7f8944139fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f894af4d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8944066930 con 0x7f894405f5a0 2020-07-18T03:29:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f894af4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89440663f0 con 0x7f8944065f30 2020-07-18T03:29:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f894af4d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8944066690 con 0x7f89440ad100 2020-07-18T03:29:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f8948ce9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f894405f5a0 0x7f89440b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42666/0 (socket says 172.21.15.47:42666) 2020-07-18T03:29:54.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f8948ce9700 1 -- 172.21.15.47:0/3450825114 learned_addr learned my addr 172.21.15.47:0/3450825114 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:54.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f89494ea700 1 --2- 172.21.15.47:0/3450825114 >> v2:172.21.15.47:3300/0 conn(0x7f8944065f30 0x7f8944139fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f8943fff700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89440ad100 0x7f894413c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610596613 0 0) 0x7f8944066930 con 0x7f894405f5a0 2020-07-18T03:29:54.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891c003240 con 0x7f894405f5a0 2020-07-18T03:29:54.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f89494ea700 1 -- 172.21.15.47:0/3450825114 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89440ad100 msgr2=0x7f894413c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f89494ea700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89440ad100 0x7f894413c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:54.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f89494ea700 1 -- 172.21.15.47:0/3450825114 >> v1:172.21.15.47:6789/0 conn(0x7f894405f5a0 legacy=0x7f89440b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:54.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.910+0000 7f89494ea700 1 -- 172.21.15.47:0/3450825114 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944065c10 con 0x7f8944065f30 2020-07-18T03:29:54.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.911+0000 7f89494ea700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944065f30 0x7f8944139fa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f893c002360 tx=0x7f893c007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.911+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c010510 con 0x7f8944065f30 2020-07-18T03:29:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.911+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c010670 con 0x7f8944065f30 2020-07-18T03:29:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.911+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894413ca30 con 0x7f8944065f30 2020-07-18T03:29:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.911+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c0107e0 con 0x7f8944065f30 2020-07-18T03:29:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.911+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8944063170 con 0x7f8944065f30 2020-07-18T03:29:54.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.912+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894413c230 con 0x7f8944065f30 2020-07-18T03:29:54.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.912+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f893c010070 con 0x7f8944065f30 2020-07-18T03:29:54.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.912+0000 7f8940ff9700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f891c0503a0 0x7f891c052860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:54.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.912+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f893c067360 con 0x7f8944065f30 2020-07-18T03:29:54.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.912+0000 7f8948ce9700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f891c0503a0 0x7f891c052860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:54.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.913+0000 7f8948ce9700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f891c0503a0 0x7f891c052860 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f8934002f30 tx=0x7f8934008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:54.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:54.916+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f893c03fe60 con 0x7f8944065f30 2020-07-18T03:29:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.076+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8944057450 con 0x7f8944065f30 2020-07-18T03:29:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.076+0000 7f8940ff9700 1 -- 172.21.15.47:0/3450825114 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+143 (secure 0 0 0) 0x7f893c043480 con 0x7f8944065f30 2020-07-18T03:29:55.080 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:55.080 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2020-07-18T03:29:55.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:55.081 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2020-07-18T03:29:55.081 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2020-07-18T03:29:55.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:29:55.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:29:55.082 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2020-07-18T03:29:55.082 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2020-07-18T03:29:55.082 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:29:55.083 INFO:teuthology.orchestra.run.smithi047.stdout:] 2020-07-18T03:29:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f891c0503a0 msgr2=0x7f891c052860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f891c0503a0 0x7f891c052860 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f8934002f30 tx=0x7f8934008040).stop 2020-07-18T03:29:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944065f30 msgr2=0x7f8944139fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944065f30 0x7f8944139fa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f893c002360 tx=0x7f893c007e40).stop 2020-07-18T03:29:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 shutdown_connections 2020-07-18T03:29:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8944065f30 0x7f8944139fa0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:55.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f891c0503a0 0x7f891c052860 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:55.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 --2- 172.21.15.47:0/3450825114 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f89440ad100 0x7f894413c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:55.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 >> 172.21.15.47:0/3450825114 conn(0x7f89440b9340 msgr2=0x7f89441431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:55.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 shutdown_connections 2020-07-18T03:29:55.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.078+0000 7f894af4d700 1 -- 172.21.15.47:0/3450825114 wait complete. 2020-07-18T03:29:55.090 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-07-18T03:29:55.091 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-07-18T03:29:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.207+0000 7f5e8ad38700 1 Processor -- start 2020-07-18T03:29:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.207+0000 7f5e8ad38700 1 -- start start 2020-07-18T03:29:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e8ad38700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 0x7f5e840b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e8ad38700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5e84142430 0x7f5e84148b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e8ad38700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e84061db0 con 0x7f5e84141fb0 2020-07-18T03:29:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e8ad38700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e840b14c0 con 0x7f5e84142430 2020-07-18T03:29:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e8ad38700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841428c0 con 0x7f5e84144e70 2020-07-18T03:29:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e88ad4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 0x7f5e840b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e892d5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5e84142430 0x7f5e84148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e88ad4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 0x7f5e840b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54184/0 (socket says 172.21.15.47:54184) 2020-07-18T03:29:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e88ad4700 1 -- 172.21.15.47:0/3362082632 learned_addr learned my addr 172.21.15.47:0/3362082632 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.208+0000 7f5e83fff700 1 --1- 172.21.15.47:0/3362082632 >> v1:172.21.15.18:6789/0 conn(0x7f5e84141fb0 0x7f5e84148440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:33208/0 (socket says 172.21.15.47:33208) 2020-07-18T03:29:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e88ad4700 1 -- 172.21.15.47:0/3362082632 >> v2:172.21.15.18:3300/0 conn(0x7f5e84142430 msgr2=0x7f5e84148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e88ad4700 1 --2- 172.21.15.47:0/3362082632 >> v2:172.21.15.18:3300/0 conn(0x7f5e84142430 0x7f5e84148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e88ad4700 1 -- 172.21.15.47:0/3362082632 >> v1:172.21.15.18:6789/0 conn(0x7f5e84141fb0 legacy=0x7f5e84148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e88ad4700 1 -- 172.21.15.47:0/3362082632 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84062050 con 0x7f5e84144e70 2020-07-18T03:29:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e88ad4700 1 --2- 172.21.15.47:0/3362082632 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 0x7f5e840b5f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f5e74002f30 tx=0x7f5e74006e90).ready entity=mon.2 client_cookie=37df0545462ad333 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e837fe700 1 -- 172.21.15.47:0/3362082632 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e74009850 con 0x7f5e84144e70 2020-07-18T03:29:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e837fe700 1 -- 172.21.15.47:0/3362082632 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e740099b0 con 0x7f5e84144e70 2020-07-18T03:29:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e837fe700 1 -- 172.21.15.47:0/3362082632 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e74009d10 con 0x7f5e84144e70 2020-07-18T03:29:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 -- 172.21.15.47:0/3362082632 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 msgr2=0x7f5e840b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/3362082632 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 0x7f5e840b5f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f5e74002f30 tx=0x7f5e74006e90).stop 2020-07-18T03:29:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 -- 172.21.15.47:0/3362082632 shutdown_connections 2020-07-18T03:29:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/3362082632 >> v2:172.21.15.18:3300/0 conn(0x7f5e84142430 0x7f5e84148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/3362082632 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5e84144e70 0x7f5e840b5f60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 -- 172.21.15.47:0/3362082632 >> 172.21.15.47:0/3362082632 conn(0x7f5e8405f5a0 msgr2=0x7f5e8405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 -- 172.21.15.47:0/3362082632 shutdown_connections 2020-07-18T03:29:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.209+0000 7f5e8ad38700 1 -- 172.21.15.47:0/3362082632 wait complete. 2020-07-18T03:29:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.211+0000 7f5e8ad38700 1 Processor -- start 2020-07-18T03:29:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.211+0000 7f5e8ad38700 1 -- start start 2020-07-18T03:29:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.211+0000 7f5e8ad38700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5e84141fb0 0x7f5e8412a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.211+0000 7f5e8ad38700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e84152eb0 con 0x7f5e84142430 2020-07-18T03:29:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.211+0000 7f5e8ad38700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e84066820 con 0x7f5e84066310 2020-07-18T03:29:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.211+0000 7f5e8ad38700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e84066ac0 con 0x7f5e84141fb0 2020-07-18T03:29:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e892d5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e84066310 0x7f5e840b9800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42682/0 (socket says 172.21.15.47:42682) 2020-07-18T03:29:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e892d5700 1 -- 172.21.15.47:0/2382521611 learned_addr learned my addr 172.21.15.47:0/2382521611 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e88ad4700 1 --2- 172.21.15.47:0/2382521611 >> v2:172.21.15.18:3300/0 conn(0x7f5e84141fb0 0x7f5e8412a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693409401 0 0) 0x7f5e84066820 con 0x7f5e84066310 2020-07-18T03:29:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e5c003240 con 0x7f5e84066310 2020-07-18T03:29:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e88ad4700 1 -- 172.21.15.47:0/2382521611 >> v1:172.21.15.47:6789/0 conn(0x7f5e84066310 legacy=0x7f5e840b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e88ad4700 1 -- 172.21.15.47:0/2382521611 >> v1:172.21.15.18:6789/0 conn(0x7f5e84142430 legacy=0x7f5e84153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073252947 0 0) 0x7f5e84152eb0 con 0x7f5e84142430 2020-07-18T03:29:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e88ad4700 1 -- 172.21.15.47:0/2382521611 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84066d60 con 0x7f5e84141fb0 2020-07-18T03:29:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e88ad4700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84141fb0 0x7f5e8412a8c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f5e74002f30 tx=0x7f5e74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e74009850 con 0x7f5e84141fb0 2020-07-18T03:29:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e74011e80 con 0x7f5e84141fb0 2020-07-18T03:29:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8412adb0 con 0x7f5e84141fb0 2020-07-18T03:29:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e7401d7c0 con 0x7f5e84141fb0 2020-07-18T03:29:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.212+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8412c1d0 con 0x7f5e84141fb0 2020-07-18T03:29:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.213+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8412c3c0 con 0x7f5e84141fb0 2020-07-18T03:29:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.214+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5e74025030 con 0x7f5e84141fb0 2020-07-18T03:29:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.214+0000 7f5e80ff9700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5e5c050310 0x7f5e5c0527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.214+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5e74069040 con 0x7f5e84141fb0 2020-07-18T03:29:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.214+0000 7f5e83fff700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5e5c050310 0x7f5e5c0527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.214+0000 7f5e83fff700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5e5c050310 0x7f5e5c0527d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f5e78000b80 tx=0x7f5e78007770).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.218+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5e74041b10 con 0x7f5e84141fb0 2020-07-18T03:29:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:55.379+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f5e84066ef0 con 0x7f5e84141fb0 2020-07-18T03:29:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.298+0000 7f5e80ff9700 1 -- 172.21.15.47:0/2382521611 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v49) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5e84066ef0 con 0x7f5e84141fb0 2020-07-18T03:29:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5e5c050310 msgr2=0x7f5e5c0527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5e5c050310 0x7f5e5c0527d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f5e78000b80 tx=0x7f5e78007770).stop 2020-07-18T03:29:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84141fb0 msgr2=0x7f5e8412a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84141fb0 0x7f5e8412a8c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f5e74002f30 tx=0x7f5e74015000).stop 2020-07-18T03:29:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 shutdown_connections 2020-07-18T03:29:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5e5c050310 0x7f5e5c0527d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 --2- 172.21.15.47:0/2382521611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84141fb0 0x7f5e8412a8c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 >> 172.21.15.47:0/2382521611 conn(0x7f5e8405f5a0 msgr2=0x7f5e840b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.300+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 shutdown_connections 2020-07-18T03:29:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.301+0000 7f5e8ad38700 1 -- 172.21.15.47:0/2382521611 wait complete. 2020-07-18T03:29:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2020-07-18T03:29:56.317 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-07-18T03:29:56.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.442+0000 7fa0e21a9700 1 Processor -- start 2020-07-18T03:29:56.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e21a9700 1 -- start start 2020-07-18T03:29:56.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e21a9700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc061800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:56.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e21a9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0dc05fff0 0x7fa0dc0ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e21a9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc13d850 con 0x7fa0dc0ad740 2020-07-18T03:29:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e21a9700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc143390 con 0x7fa0dc064710 2020-07-18T03:29:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e21a9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc064de0 con 0x7fa0dc05fff0 2020-07-18T03:29:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e09a6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0dc05fff0 0x7fa0dc0ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e19a8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa0dc0ad740 0x7fa0dc143780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42690/0 (socket says 172.21.15.47:42690) 2020-07-18T03:29:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e19a8700 1 -- 172.21.15.47:0/1251191463 learned_addr learned my addr 172.21.15.47:0/1251191463 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.443+0000 7fa0e09a6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0dc05fff0 0x7fa0dc0ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33366/0 (socket says 172.21.15.47:33366) 2020-07-18T03:29:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e11a7700 1 --2- 172.21.15.47:0/1251191463 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc061800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e09a6700 1 -- 172.21.15.47:0/1251191463 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 msgr2=0x7fa0dc061800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e09a6700 1 --2- 172.21.15.47:0/1251191463 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc061800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e09a6700 1 -- 172.21.15.47:0/1251191463 >> v1:172.21.15.47:6789/0 conn(0x7fa0dc0ad740 legacy=0x7fa0dc143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0cffff700 1 -- 172.21.15.47:0/1251191463 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245164167 0 0) 0x7fa0dc13d850 con 0x7fa0dc0ad740 2020-07-18T03:29:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e09a6700 1 -- 172.21.15.47:0/1251191463 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc065080 con 0x7fa0dc05fff0 2020-07-18T03:29:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e09a6700 1 --2- 172.21.15.47:0/1251191463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc05fff0 0x7fa0dc0ad250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fa0d8002f30 tx=0x7fa0d8006e60).ready entity=mon.1 client_cookie=40cebe628730b90c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0cffff700 1 -- 172.21.15.47:0/1251191463 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d800be30 con 0x7fa0dc05fff0 2020-07-18T03:29:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0cffff700 1 -- 172.21.15.47:0/1251191463 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8010470 con 0x7fa0dc05fff0 2020-07-18T03:29:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0cffff700 1 -- 172.21.15.47:0/1251191463 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d80107d0 con 0x7fa0dc05fff0 2020-07-18T03:29:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 -- 172.21.15.47:0/1251191463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc05fff0 msgr2=0x7fa0dc0ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/1251191463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc05fff0 0x7fa0dc0ad250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fa0d8002f30 tx=0x7fa0d8006e60).stop 2020-07-18T03:29:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 -- 172.21.15.47:0/1251191463 shutdown_connections 2020-07-18T03:29:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/1251191463 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc061800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/1251191463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc05fff0 0x7fa0dc0ad250 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 -- 172.21.15.47:0/1251191463 >> 172.21.15.47:0/1251191463 conn(0x7fa0dc0b91d0 msgr2=0x7fa0dc0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 -- 172.21.15.47:0/1251191463 shutdown_connections 2020-07-18T03:29:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.444+0000 7fa0e21a9700 1 -- 172.21.15.47:0/1251191463 wait complete. 2020-07-18T03:29:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.446+0000 7fa0e21a9700 1 Processor -- start 2020-07-18T03:29:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.446+0000 7fa0e21a9700 1 -- start start 2020-07-18T03:29:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e21a9700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e21a9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0dc0ad740 0x7fa0dc140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e21a9700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc13f7c0 con 0x7fa0dc13a270 2020-07-18T03:29:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e21a9700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc13fa60 con 0x7fa0dc064710 2020-07-18T03:29:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e21a9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc13fd00 con 0x7fa0dc0ad740 2020-07-18T03:29:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0dc0ad740 0x7fa0dc140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0dc0ad740 0x7fa0dc140440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33374/0 (socket says 172.21.15.47:33374) 2020-07-18T03:29:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 -- 172.21.15.47:0/3661289566 learned_addr learned my addr 172.21.15.47:0/3661289566 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e11a7700 1 --2- 172.21.15.47:0/3661289566 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 -- 172.21.15.47:0/3661289566 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 msgr2=0x7fa0dc154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 --2- 172.21.15.47:0/3661289566 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 -- 172.21.15.47:0/3661289566 >> v1:172.21.15.18:6789/0 conn(0x7fa0dc13a270 legacy=0x7fa0dc144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 -- 172.21.15.47:0/3661289566 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc13ffa0 con 0x7fa0dc0ad740 2020-07-18T03:29:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e09a6700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc0ad740 0x7fa0dc140440 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fa0d8002f30 tx=0x7fa0dc066690).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d8020070 con 0x7fa0dc0ad740 2020-07-18T03:29:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d801bb40 con 0x7fa0dc0ad740 2020-07-18T03:29:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.447+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc140930 con 0x7fa0dc0ad740 2020-07-18T03:29:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.448+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d801bea0 con 0x7fa0dc0ad740 2020-07-18T03:29:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.448+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc13e280 con 0x7fa0dc0ad740 2020-07-18T03:29:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.448+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc140130 con 0x7fa0dc0ad740 2020-07-18T03:29:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.449+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa0d802c070 con 0x7fa0dc0ad740 2020-07-18T03:29:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.449+0000 7fa0cd7fa700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa0b404ff50 0x7fa0b4052410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.449+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fa0d806bea0 con 0x7fa0dc0ad740 2020-07-18T03:29:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.449+0000 7fa0e11a7700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa0b404ff50 0x7fa0b4052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.449+0000 7fa0e11a7700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa0b404ff50 0x7fa0b4052410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fa0d000b000 tx=0x7fa0d0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.452+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa0d8044780 con 0x7fa0dc0ad740 2020-07-18T03:29:56.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:56.614+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa0dc0ba0f0 con 0x7fa0dc0ad740 2020-07-18T03:29:57.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.483+0000 7fa0cd7fa700 1 -- 172.21.15.47:0/3661289566 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v51) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa0d802a070 con 0x7fa0dc0ad740 2020-07-18T03:29:57.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa0b404ff50 msgr2=0x7fa0b4052410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa0b404ff50 0x7fa0b4052410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fa0d000b000 tx=0x7fa0d0008040).stop 2020-07-18T03:29:57.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc0ad740 msgr2=0x7fa0dc140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc0ad740 0x7fa0dc140440 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fa0d8002f30 tx=0x7fa0dc066690).stop 2020-07-18T03:29:57.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 shutdown_connections 2020-07-18T03:29:57.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa0b404ff50 0x7fa0b4052410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/3661289566 >> v2:172.21.15.18:3300/0 conn(0x7fa0dc064710 0x7fa0dc154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 --2- 172.21.15.47:0/3661289566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0dc0ad740 0x7fa0dc140440 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 >> 172.21.15.47:0/3661289566 conn(0x7fa0dc0b91d0 msgr2=0x7fa0dc13cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:57.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 shutdown_connections 2020-07-18T03:29:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.485+0000 7fa0e21a9700 1 -- 172.21.15.47:0/3661289566 wait complete. 2020-07-18T03:29:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2020-07-18T03:29:57.502 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-07-18T03:29:57.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 Processor -- start 2020-07-18T03:29:57.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 -- start start 2020-07-18T03:29:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62b40b59b0 0x7f62b40b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b4147de0 con 0x7f62b40b6300 2020-07-18T03:29:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b40ae000 con 0x7f62b40ad330 2020-07-18T03:29:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.637+0000 7f62bb238700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b4148010 con 0x7f62b40b59b0 2020-07-18T03:29:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62b40b59b0 0x7f62b40b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62b40b59b0 0x7f62b40b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33386/0 (socket says 172.21.15.47:33386) 2020-07-18T03:29:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 -- 172.21.15.47:0/709471967 learned_addr learned my addr 172.21.15.47:0/709471967 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b8fd4700 1 --2- 172.21.15.47:0/709471967 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 -- 172.21.15.47:0/709471967 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 msgr2=0x7f62b4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 --2- 172.21.15.47:0/709471967 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b4143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 -- 172.21.15.47:0/709471967 >> v1:172.21.15.18:6789/0 conn(0x7f62b40b6300 legacy=0x7f62b40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 -- 172.21.15.47:0/709471967 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40ad710 con 0x7f62b40b59b0 2020-07-18T03:29:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b8fd4700 1 --2- 172.21.15.47:0/709471967 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b3fff700 1 --2- 172.21.15.47:0/709471967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62b40b59b0 0x7f62b40b5e10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f62a8002f30 tx=0x7f62a8006e90).ready entity=mon.1 client_cookie=5b2a060bdb3f4794 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b37fe700 1 -- 172.21.15.47:0/709471967 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62a80099c0 con 0x7f62b40b59b0 2020-07-18T03:29:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b37fe700 1 -- 172.21.15.47:0/709471967 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a8009b20 con 0x7f62b40b59b0 2020-07-18T03:29:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62b37fe700 1 -- 172.21.15.47:0/709471967 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62a8009e80 con 0x7f62b40b59b0 2020-07-18T03:29:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 -- 172.21.15.47:0/709471967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62b40b59b0 msgr2=0x7f62b40b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 --2- 172.21.15.47:0/709471967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62b40b59b0 0x7f62b40b5e10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f62a8002f30 tx=0x7f62a8006e90).stop 2020-07-18T03:29:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 -- 172.21.15.47:0/709471967 shutdown_connections 2020-07-18T03:29:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 --2- 172.21.15.47:0/709471967 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 --2- 172.21.15.47:0/709471967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62b40b59b0 0x7f62b40b5e10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 -- 172.21.15.47:0/709471967 >> 172.21.15.47:0/709471967 conn(0x7f62b405f5a0 msgr2=0x7f62b405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 -- 172.21.15.47:0/709471967 shutdown_connections 2020-07-18T03:29:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.638+0000 7f62bb238700 1 -- 172.21.15.47:0/709471967 wait complete. 2020-07-18T03:29:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.640+0000 7f62bb238700 1 Processor -- start 2020-07-18T03:29:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.640+0000 7f62bb238700 1 -- start start 2020-07-18T03:29:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62bb238700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b40b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62bb238700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 0x7f62b4156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62bb238700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b41531d0 con 0x7f62b40b6300 2020-07-18T03:29:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62bb238700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b41453e0 con 0x7f62b40ad330 2020-07-18T03:29:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62bb238700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4153470 con 0x7f62b40b9880 2020-07-18T03:29:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62b97d5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 0x7f62b4156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62b8fd4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b40b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62b97d5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 0x7f62b4156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54222/0 (socket says 172.21.15.47:54222) 2020-07-18T03:29:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.641+0000 7f62b97d5700 1 -- 172.21.15.47:0/4180212245 learned_addr learned my addr 172.21.15.47:0/4180212245 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b97d5700 1 -- 172.21.15.47:0/4180212245 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 msgr2=0x7f62b40b9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b97d5700 1 --2- 172.21.15.47:0/4180212245 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b40b9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b97d5700 1 -- 172.21.15.47:0/4180212245 >> v1:172.21.15.18:6789/0 conn(0x7f62b40b6300 legacy=0x7f62b4156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241684880 0 0) 0x7f62b41531d0 con 0x7f62b40b6300 2020-07-18T03:29:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b97d5700 1 -- 172.21.15.47:0/4180212245 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4153710 con 0x7f62b40b9880 2020-07-18T03:29:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b8fd4700 1 --2- 172.21.15.47:0/4180212245 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b40b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b97d5700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 0x7f62b4156f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f62b40ae4e0 tx=0x7f62ac007e90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ac00a5a0 con 0x7f62b40b9880 2020-07-18T03:29:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00a700 con 0x7f62b40b9880 2020-07-18T03:29:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b4141740 con 0x7f62b40b9880 2020-07-18T03:29:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ac00a870 con 0x7f62b40b9880 2020-07-18T03:29:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.642+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b4152d80 con 0x7f62b40b9880 2020-07-18T03:29:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.643+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b4153000 con 0x7f62b40b9880 2020-07-18T03:29:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.643+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f62ac010070 con 0x7f62b40b9880 2020-07-18T03:29:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.643+0000 7f62b0ff9700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f628c04fd80 0x7f628c052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.643+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f62ac0673a0 con 0x7f62b40b9880 2020-07-18T03:29:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.644+0000 7f62b8fd4700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f628c04fd80 0x7f628c052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.644+0000 7f62b8fd4700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f628c04fd80 0x7f628c052240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f62a400c000 tx=0x7f62a4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.647+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f62ac03fa30 con 0x7f62b40b9880 2020-07-18T03:29:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.857+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f62b405b120 con 0x7f62b40b9880 2020-07-18T03:29:57.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.865+0000 7f62b0ff9700 1 -- 172.21.15.47:0/4180212245 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f62ac043050 con 0x7f62b40b9880 2020-07-18T03:29:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.869+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f628c04fd80 msgr2=0x7f628c052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.869+0000 7f62bb238700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f628c04fd80 0x7f628c052240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f62a400c000 tx=0x7f62a4008040).stop 2020-07-18T03:29:57.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.869+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 msgr2=0x7f62b4156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:57.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.869+0000 7f62bb238700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 0x7f62b4156f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f62b40ae4e0 tx=0x7f62ac007e90).stop 2020-07-18T03:29:57.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.872+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 shutdown_connections 2020-07-18T03:29:57.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.872+0000 7f62bb238700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f628c04fd80 0x7f628c052240 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.872+0000 7f62bb238700 1 --2- 172.21.15.47:0/4180212245 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f62b40b9880 0x7f62b4156f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.872+0000 7f62bb238700 1 --2- 172.21.15.47:0/4180212245 >> v2:172.21.15.18:3300/0 conn(0x7f62b40ad330 0x7f62b40b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:57.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.872+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 >> 172.21.15.47:0/4180212245 conn(0x7f62b405f5a0 msgr2=0x7f62b4142420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:57.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.874+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 shutdown_connections 2020-07-18T03:29:57.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:57.875+0000 7f62bb238700 1 -- 172.21.15.47:0/4180212245 wait complete. 2020-07-18T03:29:57.879 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 9 and data pool 10 2020-07-18T03:29:57.889 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:58.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.001+0000 7f4b9ac36700 1 Processor -- start 2020-07-18T03:29:58.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.001+0000 7f4b9ac36700 1 -- start start 2020-07-18T03:29:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b9ac36700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4b940ad100 0x7f4b940b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b9ac36700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b94147330 con 0x7f4b94147a00 2020-07-18T03:29:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b9ac36700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b940603c0 con 0x7f4b941443f0 2020-07-18T03:29:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b9ac36700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b940b57e0 con 0x7f4b940ad100 2020-07-18T03:29:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b991d3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4b941443f0 0x7f4b9405fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42720/0 (socket says 172.21.15.47:42720) 2020-07-18T03:29:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b991d3700 1 -- 172.21.15.47:0/3587392139 learned_addr learned my addr 172.21.15.47:0/3587392139 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.002+0000 7f4b989d2700 1 --2- 172.21.15.47:0/3587392139 >> v2:172.21.15.18:3300/0 conn(0x7f4b940ad100 0x7f4b940b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167564087 0 0) 0x7f4b940603c0 con 0x7f4b941443f0 2020-07-18T03:29:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b80003240 con 0x7f4b941443f0 2020-07-18T03:29:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333087014 0 0) 0x7f4b94147330 con 0x7f4b94147a00 2020-07-18T03:29:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b940603c0 con 0x7f4b94147a00 2020-07-18T03:29:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2067811934 0 0) 0x7f4b80003240 con 0x7f4b941443f0 2020-07-18T03:29:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b989d2700 1 -- 172.21.15.47:0/3587392139 >> v1:172.21.15.47:6789/0 conn(0x7f4b941443f0 legacy=0x7f4b9405fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b989d2700 1 -- 172.21.15.47:0/3587392139 >> v1:172.21.15.18:6789/0 conn(0x7f4b94147a00 legacy=0x7f4b9405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b989d2700 1 -- 172.21.15.47:0/3587392139 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940b5a80 con 0x7f4b940ad100 2020-07-18T03:29:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b989d2700 1 --2- 172.21.15.47:0/3587392139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b940ad100 0x7f4b940b4c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f4b8c001450 tx=0x7f4b84006dd0).ready entity=mon.0 client_cookie=5804d793ebb86daa server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b8c001a50 con 0x7f4b940ad100 2020-07-18T03:29:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8400d610 con 0x7f4b940ad100 2020-07-18T03:29:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b937fe700 1 -- 172.21.15.47:0/3587392139 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b8400d780 con 0x7f4b940ad100 2020-07-18T03:29:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 -- 172.21.15.47:0/3587392139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b940ad100 msgr2=0x7f4b940b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/3587392139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b940ad100 0x7f4b940b4c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f4b8c001450 tx=0x7f4b84006dd0).stop 2020-07-18T03:29:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 -- 172.21.15.47:0/3587392139 shutdown_connections 2020-07-18T03:29:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/3587392139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b940ad100 0x7f4b940b4c10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 -- 172.21.15.47:0/3587392139 >> 172.21.15.47:0/3587392139 conn(0x7f4b940b9340 msgr2=0x7f4b940b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 -- 172.21.15.47:0/3587392139 shutdown_connections 2020-07-18T03:29:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.003+0000 7f4b9ac36700 1 -- 172.21.15.47:0/3587392139 wait complete. 2020-07-18T03:29:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.005+0000 7f4b9ac36700 1 Processor -- start 2020-07-18T03:29:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.005+0000 7f4b9ac36700 1 -- start start 2020-07-18T03:29:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b9ac36700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4b941443f0 0x7f4b9413edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b9ac36700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4b94062090 0x7f4b94152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b9ac36700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b94153610 con 0x7f4b94147a00 2020-07-18T03:29:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b9ac36700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b94064250 con 0x7f4b941443f0 2020-07-18T03:29:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b9ac36700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b941538b0 con 0x7f4b94062090 2020-07-18T03:29:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4b941443f0 0x7f4b9413edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4b941443f0 0x7f4b9413edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33404/0 (socket says 172.21.15.47:33404) 2020-07-18T03:29:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 -- 172.21.15.47:0/1492382152 learned_addr learned my addr 172.21.15.47:0/1492382152 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b991d3700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4b94062090 0x7f4b94152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 -- 172.21.15.47:0/1492382152 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4b94062090 msgr2=0x7f4b94152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4b94062090 0x7f4b94152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 -- 172.21.15.47:0/1492382152 >> v1:172.21.15.18:6789/0 conn(0x7f4b94147a00 legacy=0x7f4b94152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210961011 0 0) 0x7f4b94153610 con 0x7f4b94147a00 2020-07-18T03:29:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 -- 172.21.15.47:0/1492382152 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94153b50 con 0x7f4b941443f0 2020-07-18T03:29:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b989d2700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b941443f0 0x7f4b9413edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f4b8c001450 tx=0x7f4b84015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b8401e460 con 0x7f4b941443f0 2020-07-18T03:29:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.006+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8401e5c0 con 0x7f4b941443f0 2020-07-18T03:29:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.007+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9413f2a0 con 0x7f4b941443f0 2020-07-18T03:29:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.007+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b8401e920 con 0x7f4b941443f0 2020-07-18T03:29:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.007+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b941567f0 con 0x7f4b941443f0 2020-07-18T03:29:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.008+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b94057450 con 0x7f4b941443f0 2020-07-18T03:29:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.008+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4b84025030 con 0x7f4b941443f0 2020-07-18T03:29:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.008+0000 7f4b90ff9700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4b6c04fd40 0x7f4b6c052200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.008+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4b84069200 con 0x7f4b941443f0 2020-07-18T03:29:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.008+0000 7f4b93fff700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4b6c04fd40 0x7f4b6c052200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.008+0000 7f4b93fff700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4b6c04fd40 0x7f4b6c052200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4b88001050 tx=0x7f4b88007700).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.012+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4b84041990 con 0x7f4b941443f0 2020-07-18T03:29:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.169+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b94156aa0 con 0x7f4b941443f0 2020-07-18T03:29:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.170+0000 7f4b90ff9700 1 -- 172.21.15.47:0/1492382152 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16289 (secure 0 0 0) 0x7f4b84044fb0 con 0x7f4b941443f0 2020-07-18T03:29:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:58.173 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":52,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:57.859484+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:55.419365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:56.654688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4b6c04fd40 msgr2=0x7f4b6c052200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4b6c04fd40 0x7f4b6c052200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4b88001050 tx=0x7f4b88007700).stop 2020-07-18T03:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b941443f0 msgr2=0x7f4b9413edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b941443f0 0x7f4b9413edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f4b8c001450 tx=0x7f4b84015000).stop 2020-07-18T03:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 shutdown_connections 2020-07-18T03:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4b6c04fd40 0x7f4b6c052200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4b94062090 0x7f4b94152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 --2- 172.21.15.47:0/1492382152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b941443f0 0x7f4b9413edb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.171+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 >> 172.21.15.47:0/1492382152 conn(0x7f4b940b9340 msgr2=0x7f4b9405fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.172+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 shutdown_connections 2020-07-18T03:29:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.172+0000 7f4b9ac36700 1 -- 172.21.15.47:0/1492382152 wait complete. 2020-07-18T03:29:58.184 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 Processor -- start 2020-07-18T03:29:58.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 -- start start 2020-07-18T03:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f71c80b5370 0x7f71c805fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 0x7f71c80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80b3ac0 con 0x7f71c80b4620 2020-07-18T03:29:58.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80b5a10 con 0x7f71c80b5370 2020-07-18T03:29:58.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.298+0000 7f71cd706700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80b0040 con 0x7f71c80b41c0 2020-07-18T03:29:58.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 0x7f71c80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 0x7f71c80ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54240/0 (socket says 172.21.15.47:54240) 2020-07-18T03:29:58.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c6ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f71c80b5370 0x7f71c805fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 -- 172.21.15.47:0/1301102208 learned_addr learned my addr 172.21.15.47:0/1301102208 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:58.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c77fe700 1 --1- 172.21.15.47:0/1301102208 >> v1:172.21.15.18:6789/0 conn(0x7f71c80b4620 0x7f71c81428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:33264/0 (socket says 172.21.15.47:33264) 2020-07-18T03:29:58.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c5ffb700 1 -- 172.21.15.47:0/1301102208 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786941356 0 0) 0x7f71c80b3ac0 con 0x7f71c80b4620 2020-07-18T03:29:58.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 -- 172.21.15.47:0/1301102208 >> v2:172.21.15.18:3300/0 conn(0x7f71c80b5370 msgr2=0x7f71c805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:58.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 --2- 172.21.15.47:0/1301102208 >> v2:172.21.15.18:3300/0 conn(0x7f71c80b5370 0x7f71c805fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 -- 172.21.15.47:0/1301102208 >> v1:172.21.15.18:6789/0 conn(0x7f71c80b4620 legacy=0x7f71c81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 -- 172.21.15.47:0/1301102208 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80b3d20 con 0x7f71c80b41c0 2020-07-18T03:29:58.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c67fc700 1 --2- 172.21.15.47:0/1301102208 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 0x7f71c80ad2d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f71b0002f30 tx=0x7f71b0006f20).ready entity=mon.2 client_cookie=4e5913ae3ad3115 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c5ffb700 1 -- 172.21.15.47:0/1301102208 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b000dda0 con 0x7f71c80b41c0 2020-07-18T03:29:58.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c5ffb700 1 -- 172.21.15.47:0/1301102208 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0010470 con 0x7f71c80b41c0 2020-07-18T03:29:58.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.299+0000 7f71c5ffb700 1 -- 172.21.15.47:0/1301102208 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b00107d0 con 0x7f71c80b41c0 2020-07-18T03:29:58.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 -- 172.21.15.47:0/1301102208 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 msgr2=0x7f71c80ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 --2- 172.21.15.47:0/1301102208 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 0x7f71c80ad2d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f71b0002f30 tx=0x7f71b0006f20).stop 2020-07-18T03:29:58.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 -- 172.21.15.47:0/1301102208 shutdown_connections 2020-07-18T03:29:58.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 --2- 172.21.15.47:0/1301102208 >> v2:172.21.15.18:3300/0 conn(0x7f71c80b5370 0x7f71c805fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 --2- 172.21.15.47:0/1301102208 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c80b41c0 0x7f71c80ad2d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 -- 172.21.15.47:0/1301102208 >> 172.21.15.47:0/1301102208 conn(0x7f71c80b9340 msgr2=0x7f71c80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:58.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 -- 172.21.15.47:0/1301102208 shutdown_connections 2020-07-18T03:29:58.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.300+0000 7f71cd706700 1 -- 172.21.15.47:0/1301102208 wait complete. 2020-07-18T03:29:58.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.301+0000 7f71cd706700 1 Processor -- start 2020-07-18T03:29:58.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.301+0000 7f71cd706700 1 -- start start 2020-07-18T03:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71cd706700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71c80b5370 0x7f71c8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71cd706700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c813ba10 0x7f71c813be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71cd706700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c8154940 con 0x7f71c81544a0 2020-07-18T03:29:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71cd706700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c8154be0 con 0x7f71c80b5370 2020-07-18T03:29:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71cd706700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8154e80 con 0x7f71c813ba10 2020-07-18T03:29:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71c80b5370 0x7f71c8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71c80b5370 0x7f71c8150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33416/0 (socket says 172.21.15.47:33416) 2020-07-18T03:29:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 -- 172.21.15.47:0/3378632799 learned_addr learned my addr 172.21.15.47:0/3378632799 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c77fe700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c813ba10 0x7f71c813be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 -- 172.21.15.47:0/3378632799 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c813ba10 msgr2=0x7f71c813be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c813ba10 0x7f71c813be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 -- 172.21.15.47:0/3378632799 >> v1:172.21.15.18:6789/0 conn(0x7f71c81544a0 legacy=0x7f71c8141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.302+0000 7f71c6ffd700 1 -- 172.21.15.47:0/3378632799 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c813c370 con 0x7f71c80b5370 2020-07-18T03:29:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.303+0000 7f71c6ffd700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71c80b5370 0x7f71c8150ec0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f71c000c000 tx=0x7f71c00091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.304+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c000cbb0 con 0x7f71c80b5370 2020-07-18T03:29:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.304+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c000cd10 con 0x7f71c80b5370 2020-07-18T03:29:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.304+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c00125e0 con 0x7f71c80b5370 2020-07-18T03:29:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.304+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c8063fb0 con 0x7f71c80b5370 2020-07-18T03:29:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.304+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c8065440 con 0x7f71c80b5370 2020-07-18T03:29:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.305+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c8057450 con 0x7f71c80b5370 2020-07-18T03:29:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.305+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f71c0022070 con 0x7f71c80b5370 2020-07-18T03:29:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.305+0000 7f71beffd700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71a404ffe0 0x7f71a40524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.305+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71c0068390 con 0x7f71c80b5370 2020-07-18T03:29:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.305+0000 7f71c67fc700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71a404ffe0 0x7f71a40524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.306+0000 7f71c67fc700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71a404ffe0 0x7f71a40524a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f71c813e870 tx=0x7f71b0018000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.309+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f71c0040b20 con 0x7f71c80b5370 2020-07-18T03:29:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.465+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71c8063cb0 con 0x7f71c80b5370 2020-07-18T03:29:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.465+0000 7f71beffd700 1 -- 172.21.15.47:0/3378632799 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16289 (secure 0 0 0) 0x7f71c0044140 con 0x7f71c80b5370 2020-07-18T03:29:58.469 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:58.469 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":52,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:57.859484+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:55.419365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:56.654688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71a404ffe0 msgr2=0x7f71a40524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71a404ffe0 0x7f71a40524a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f71c813e870 tx=0x7f71b0018000).stop 2020-07-18T03:29:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71c80b5370 msgr2=0x7f71c8150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71c80b5370 0x7f71c8150ec0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f71c000c000 tx=0x7f71c00091d0).stop 2020-07-18T03:29:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 shutdown_connections 2020-07-18T03:29:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71a404ffe0 0x7f71a40524a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71c813ba10 0x7f71c813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 --2- 172.21.15.47:0/3378632799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71c80b5370 0x7f71c8150ec0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 >> 172.21.15.47:0/3378632799 conn(0x7f71c80b9340 msgr2=0x7f71c80ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 shutdown_connections 2020-07-18T03:29:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.467+0000 7f71cd706700 1 -- 172.21.15.47:0/3378632799 wait complete. 2020-07-18T03:29:58.480 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-07-18T03:29:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.596+0000 7f44deed4700 1 Processor -- start 2020-07-18T03:29:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.596+0000 7f44deed4700 1 -- start start 2020-07-18T03:29:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44deed4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f44d80ad330 0x7f44d8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44deed4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 0x7f44d80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44deed4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d8147de0 con 0x7f44d80b59b0 2020-07-18T03:29:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44deed4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d80ae000 con 0x7f44d80ad330 2020-07-18T03:29:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44deed4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8148010 con 0x7f44d80b3a10 2020-07-18T03:29:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44d7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f44d80b59b0 0x7f44d80b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42744/0 (socket says 172.21.15.47:42744) 2020-07-18T03:29:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.597+0000 7f44d7fff700 1 -- 172.21.15.47:0/800164334 learned_addr learned my addr 172.21.15.47:0/800164334 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 0x7f44d80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dcc70700 1 --2- 172.21.15.47:0/800164334 >> v2:172.21.15.18:3300/0 conn(0x7f44d80ad330 0x7f44d8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208898169 0 0) 0x7f44d8147de0 con 0x7f44d80b59b0 2020-07-18T03:29:58.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c4003240 con 0x7f44d80b59b0 2020-07-18T03:29:58.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2598703126 0 0) 0x7f44c4003240 con 0x7f44d80b59b0 2020-07-18T03:29:58.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 msgr2=0x7f44d80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:58.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 0x7f44d80b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 >> v2:172.21.15.18:3300/0 conn(0x7f44d80ad330 msgr2=0x7f44d8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:58.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 --2- 172.21.15.47:0/800164334 >> v2:172.21.15.18:3300/0 conn(0x7f44d80ad330 0x7f44d8143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d80ad710 con 0x7f44d80b59b0 2020-07-18T03:29:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f44cc0016d0 con 0x7f44d80b59b0 2020-07-18T03:29:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 >> v1:172.21.15.47:6789/0 conn(0x7f44d80b59b0 legacy=0x7f44d80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44c4003730 0x7f44c4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 0x7f44c40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44c40049d0 0x7f44c4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d80ae000 con 0x7f44c4003730 2020-07-18T03:29:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8148010 con 0x7f44c40049d0 2020-07-18T03:29:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40041d0 con 0x7f44d80b3a10 2020-07-18T03:29:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44c40049d0 0x7f44c4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dcc70700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44c4003730 0x7f44c4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44d7fff700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 0x7f44c40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 -- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 msgr2=0x7f44c40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f44d80b3a10 0x7f44c40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 -- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44c4003730 msgr2=0x7f44c4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44c4003730 0x7f44c4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.598+0000 7f44dd471700 1 -- 172.21.15.47:0/800164334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4003240 con 0x7f44c40049d0 2020-07-18T03:29:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44dd471700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44c40049d0 0x7f44c4004e50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f44d000c000 tx=0x7f44d00091d0).ready entity=mon.1 client_cookie=b6cc3eeadb3f39a0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44d000ec30 con 0x7f44c40049d0 2020-07-18T03:29:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d000ed90 con 0x7f44c40049d0 2020-07-18T03:29:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44d77fe700 1 -- 172.21.15.47:0/800164334 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44d00125e0 con 0x7f44c40049d0 2020-07-18T03:29:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44deed4700 1 -- 172.21.15.47:0/800164334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44c40049d0 msgr2=0x7f44c4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44deed4700 1 --2- 172.21.15.47:0/800164334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44c40049d0 0x7f44c4004e50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f44d000c000 tx=0x7f44d00091d0).stop 2020-07-18T03:29:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44dd471700 1 -- 172.21.15.47:0/800164334 reap_dead start 2020-07-18T03:29:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44deed4700 1 -- 172.21.15.47:0/800164334 shutdown_connections 2020-07-18T03:29:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44deed4700 1 -- 172.21.15.47:0/800164334 >> 172.21.15.47:0/800164334 conn(0x7f44d805f5a0 msgr2=0x7f44d805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44deed4700 1 -- 172.21.15.47:0/800164334 shutdown_connections 2020-07-18T03:29:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.599+0000 7f44deed4700 1 -- 172.21.15.47:0/800164334 wait complete. 2020-07-18T03:29:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.600+0000 7f44deed4700 1 Processor -- start 2020-07-18T03:29:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.600+0000 7f44deed4700 1 -- start start 2020-07-18T03:29:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44deed4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f44d80b31f0 0x7f44d813bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44deed4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44d8064e20 0x7f44d813c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44deed4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d80652c0 con 0x7f44d80b2e10 2020-07-18T03:29:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44deed4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d8065560 con 0x7f44d80b31f0 2020-07-18T03:29:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44deed4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d8065800 con 0x7f44d8064e20 2020-07-18T03:29:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44dd471700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44d8064e20 0x7f44d813c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44dd471700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44d8064e20 0x7f44d813c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33440/0 (socket says 172.21.15.47:33440) 2020-07-18T03:29:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44dd471700 1 -- 172.21.15.47:0/1664201438 learned_addr learned my addr 172.21.15.47:0/1664201438 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.601+0000 7f44d7fff700 1 --2- 172.21.15.47:0/1664201438 >> v2:172.21.15.18:3300/0 conn(0x7f44d80b31f0 0x7f44d813bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44dd471700 1 -- 172.21.15.47:0/1664201438 >> v2:172.21.15.18:3300/0 conn(0x7f44d80b31f0 msgr2=0x7f44d813bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44dd471700 1 --2- 172.21.15.47:0/1664201438 >> v2:172.21.15.18:3300/0 conn(0x7f44d80b31f0 0x7f44d813bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44dd471700 1 -- 172.21.15.47:0/1664201438 >> v1:172.21.15.18:6789/0 conn(0x7f44d80b2e10 legacy=0x7f44d814ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44dd471700 1 -- 172.21.15.47:0/1664201438 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8064020 con 0x7f44d8064e20 2020-07-18T03:29:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44d7fff700 1 --2- 172.21.15.47:0/1664201438 >> v2:172.21.15.18:3300/0 conn(0x7f44d80b31f0 0x7f44d813bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44dd471700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d8064e20 0x7f44d813c400 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f44d80b6920 tx=0x7f44d001a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44d0006ca0 con 0x7f44d8064e20 2020-07-18T03:29:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d0012e00 con 0x7f44d8064e20 2020-07-18T03:29:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d8062e50 con 0x7f44d8064e20 2020-07-18T03:29:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44d0025620 con 0x7f44d8064e20 2020-07-18T03:29:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.602+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d8065ca0 con 0x7f44d8064e20 2020-07-18T03:29:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.603+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d80ae200 con 0x7f44d8064e20 2020-07-18T03:29:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.603+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f44d002c070 con 0x7f44d8064e20 2020-07-18T03:29:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.603+0000 7f44d4ff9700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f44b004ff50 0x7f44b0052410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.603+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f44d0002030 con 0x7f44d8064e20 2020-07-18T03:29:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.603+0000 7f44dcc70700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f44b004ff50 0x7f44b0052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.604+0000 7f44dcc70700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f44b004ff50 0x7f44b0052410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f44d8148d60 tx=0x7f44c8008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.607+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f44d0045c50 con 0x7f44d8064e20 2020-07-18T03:29:58.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.817+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f44d80b50c0 con 0x7f44d8064e20 2020-07-18T03:29:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.866+0000 7f44d4ff9700 1 -- 172.21.15.47:0/1664201438 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f44d0045840 con 0x7f44d8064e20 2020-07-18T03:29:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f44b004ff50 msgr2=0x7f44b0052410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f44b004ff50 0x7f44b0052410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f44d8148d60 tx=0x7f44c8008040).stop 2020-07-18T03:29:58.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d8064e20 msgr2=0x7f44d813c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:58.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d8064e20 0x7f44d813c400 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f44d80b6920 tx=0x7f44d001a000).stop 2020-07-18T03:29:58.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 shutdown_connections 2020-07-18T03:29:58.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f44b004ff50 0x7f44b0052410 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 --2- 172.21.15.47:0/1664201438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d8064e20 0x7f44d813c400 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 --2- 172.21.15.47:0/1664201438 >> v2:172.21.15.18:3300/0 conn(0x7f44d80b31f0 0x7f44d813bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 >> 172.21.15.47:0/1664201438 conn(0x7f44d805f5a0 msgr2=0x7f44d80618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 shutdown_connections 2020-07-18T03:29:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.868+0000 7f44deed4700 1 -- 172.21.15.47:0/1664201438 wait complete. 2020-07-18T03:29:58.881 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:29:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.994+0000 7feacc7f4700 1 Processor -- start 2020-07-18T03:29:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.994+0000 7feacc7f4700 1 -- start start 2020-07-18T03:29:59.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feacc7f4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac40ad100 0x7feac40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feacc7f4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feac4147a00 0x7feac4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feacc7f4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac4147330 con 0x7feac405f5a0 2020-07-18T03:29:59.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feacc7f4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40ae530 con 0x7feac4147a00 2020-07-18T03:29:59.008 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feacc7f4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40b57e0 con 0x7feac40ad100 2020-07-18T03:29:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feaca590700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac40ad100 0x7feac40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feaca590700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feac40ad100 0x7feac40b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33444/0 (socket says 172.21.15.47:33444) 2020-07-18T03:29:59.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.995+0000 7feaca590700 1 -- 172.21.15.47:0/1509155535 learned_addr learned my addr 172.21.15.47:0/1509155535 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:29:59.010 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feac9d8f700 1 --2- 172.21.15.47:0/1509155535 >> v2:172.21.15.18:3300/0 conn(0x7feac4147a00 0x7feac4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feaca590700 1 -- 172.21.15.47:0/1509155535 >> v2:172.21.15.18:3300/0 conn(0x7feac4147a00 msgr2=0x7feac4147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.011 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feaca590700 1 --2- 172.21.15.47:0/1509155535 >> v2:172.21.15.18:3300/0 conn(0x7feac4147a00 0x7feac4147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feaca590700 1 -- 172.21.15.47:0/1509155535 >> v1:172.21.15.18:6789/0 conn(0x7feac405f5a0 legacy=0x7feac405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feaca590700 1 -- 172.21.15.47:0/1509155535 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40b5a80 con 0x7feac40ad100 2020-07-18T03:29:59.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feaca590700 1 --2- 172.21.15.47:0/1509155535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac40ad100 0x7feac40b4c10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7feabc0032b0 tx=0x7feabc009270).ready entity=mon.1 client_cookie=7b6bbf1d4ff31ddf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feac958e700 1 -- 172.21.15.47:0/1509155535 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feabc00bc00 con 0x7feac40ad100 2020-07-18T03:29:59.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feac958e700 1 -- 172.21.15.47:0/1509155535 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00bd60 con 0x7feac40ad100 2020-07-18T03:29:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feac958e700 1 -- 172.21.15.47:0/1509155535 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feabc011400 con 0x7feac40ad100 2020-07-18T03:29:59.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 -- 172.21.15.47:0/1509155535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac40ad100 msgr2=0x7feac40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 --2- 172.21.15.47:0/1509155535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac40ad100 0x7feac40b4c10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7feabc0032b0 tx=0x7feabc009270).stop 2020-07-18T03:29:59.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 -- 172.21.15.47:0/1509155535 shutdown_connections 2020-07-18T03:29:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 --2- 172.21.15.47:0/1509155535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feac40ad100 0x7feac40b4c10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 --2- 172.21.15.47:0/1509155535 >> v2:172.21.15.18:3300/0 conn(0x7feac4147a00 0x7feac4147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 -- 172.21.15.47:0/1509155535 >> 172.21.15.47:0/1509155535 conn(0x7feac40b9340 msgr2=0x7feac40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:59.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 -- 172.21.15.47:0/1509155535 shutdown_connections 2020-07-18T03:29:59.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.996+0000 7feacc7f4700 1 -- 172.21.15.47:0/1509155535 wait complete. 2020-07-18T03:29:59.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feacc7f4700 1 Processor -- start 2020-07-18T03:29:59.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feacc7f4700 1 -- start start 2020-07-18T03:29:59.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feacc7f4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7feac414ae90 0x7feac413b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feacc7f4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40b0600 con 0x7feac405f5a0 2020-07-18T03:29:59.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feacc7f4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40b08a0 con 0x7feac4147a00 2020-07-18T03:29:59.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feacc7f4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40b0410 con 0x7feac414ae90 2020-07-18T03:29:59.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feac9d8f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feac4147a00 0x7feac413b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42772/0 (socket says 172.21.15.47:42772) 2020-07-18T03:29:59.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.998+0000 7feac9d8f700 1 -- 172.21.15.47:0/3173687915 learned_addr learned my addr 172.21.15.47:0/3173687915 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacad91700 1 --2- 172.21.15.47:0/3173687915 >> v2:172.21.15.18:3300/0 conn(0x7feac414ae90 0x7feac413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641061792 0 0) 0x7feac40b08a0 con 0x7feac4147a00 2020-07-18T03:29:59.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea9c003240 con 0x7feac4147a00 2020-07-18T03:29:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092417620 0 0) 0x7feac40b0600 con 0x7feac405f5a0 2020-07-18T03:29:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac40b08a0 con 0x7feac405f5a0 2020-07-18T03:29:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3953769414 0 0) 0x7fea9c003240 con 0x7feac4147a00 2020-07-18T03:29:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacad91700 1 -- 172.21.15.47:0/3173687915 >> v1:172.21.15.47:6789/0 conn(0x7feac4147a00 legacy=0x7feac413b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacad91700 1 -- 172.21.15.47:0/3173687915 >> v1:172.21.15.18:6789/0 conn(0x7feac405f5a0 legacy=0x7feac4141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacad91700 1 -- 172.21.15.47:0/3173687915 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac413beb0 con 0x7feac414ae90 2020-07-18T03:29:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacad91700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac414ae90 0x7feac413b850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7feab8006fc0 tx=0x7feab4007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab8006e60 con 0x7feac414ae90 2020-07-18T03:29:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4156850 con 0x7feac414ae90 2020-07-18T03:29:59.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab400ce00 con 0x7feac414ae90 2020-07-18T03:29:59.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab400f470 con 0x7feac414ae90 2020-07-18T03:29:59.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:58.999+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac413c900 con 0x7feac414ae90 2020-07-18T03:29:59.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.000+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac40b0a30 con 0x7feac414ae90 2020-07-18T03:29:59.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.001+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7feab4010070 con 0x7feac414ae90 2020-07-18T03:29:59.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.001+0000 7feac27fc700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9c050450 0x7fea9c052910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.001+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feab4066d90 con 0x7feac414ae90 2020-07-18T03:29:59.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.001+0000 7feaca590700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9c050450 0x7fea9c052910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.001+0000 7feaca590700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9c050450 0x7fea9c052910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7feabc006ae0 tx=0x7feabc016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.005+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7feab403f520 con 0x7feac414ae90 2020-07-18T03:29:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.216+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feac4057450 con 0x7feac414ae90 2020-07-18T03:29:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.217+0000 7feac27fc700 1 -- 172.21.15.47:0/3173687915 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7feab4032020 con 0x7feac414ae90 2020-07-18T03:29:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:59.220 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:29:59.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9c050450 msgr2=0x7fea9c052910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9c050450 0x7fea9c052910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7feabc006ae0 tx=0x7feabc016000).stop 2020-07-18T03:29:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac414ae90 msgr2=0x7feac413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac414ae90 0x7feac413b850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7feab8006fc0 tx=0x7feab4007a80).stop 2020-07-18T03:29:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 shutdown_connections 2020-07-18T03:29:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feac414ae90 0x7feac413b850 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 --2- 172.21.15.47:0/3173687915 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea9c050450 0x7fea9c052910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 >> 172.21.15.47:0/3173687915 conn(0x7feac40b9340 msgr2=0x7feac40b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 shutdown_connections 2020-07-18T03:29:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.218+0000 7feacc7f4700 1 -- 172.21.15.47:0/3173687915 wait complete. 2020-07-18T03:29:59.224 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:29:59.235 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:29:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.350+0000 7f81fb744700 1 Processor -- start 2020-07-18T03:29:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.350+0000 7f81fb744700 1 -- start start 2020-07-18T03:29:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81fb744700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81f4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81fb744700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f4147330 con 0x7f81f40ad100 2020-07-18T03:29:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81fb744700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f40b57e0 con 0x7f81f41433e0 2020-07-18T03:29:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81fb744700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40ae530 con 0x7f81f4147a00 2020-07-18T03:29:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81f9ce1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f81f41433e0 0x7f81f4143850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42782/0 (socket says 172.21.15.47:42782) 2020-07-18T03:29:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81f9ce1700 1 -- 172.21.15.47:0/4274611049 learned_addr learned my addr 172.21.15.47:0/4274611049 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331819272 0 0) 0x7f81f40b57e0 con 0x7f81f41433e0 2020-07-18T03:29:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81f8cdf700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81f4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81dc003240 con 0x7f81f41433e0 2020-07-18T03:29:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1883091202 0 0) 0x7f81dc003240 con 0x7f81f41433e0 2020-07-18T03:29:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 msgr2=0x7f81f4147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81f4147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 >> v1:172.21.15.18:6789/0 conn(0x7f81f40ad100 legacy=0x7f81f40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40b5a80 con 0x7f81f41433e0 2020-07-18T03:29:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f81ec001a50 con 0x7f81f41433e0 2020-07-18T03:29:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 >> v1:172.21.15.47:6789/0 conn(0x7f81f41433e0 legacy=0x7f81f4143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81dc003730 0x7f81dc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81dc0044e0 0x7f81dc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81dc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40ae530 con 0x7f81dc0044e0 2020-07-18T03:29:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc005390 con 0x7f81dc003730 2020-07-18T03:29:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0042f0 con 0x7f81f4147a00 2020-07-18T03:29:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.351+0000 7f81f94e0700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81dc003730 0x7f81dc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f8cdf700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81dc0044e0 0x7f81dc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f9ce1700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81dc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f94e0700 1 -- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 msgr2=0x7f81dc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f94e0700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81dc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f94e0700 1 -- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81dc0044e0 msgr2=0x7f81dc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f94e0700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81dc0044e0 0x7f81dc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f94e0700 1 -- 172.21.15.47:0/4274611049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc003240 con 0x7f81dc003730 2020-07-18T03:29:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f9ce1700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4147a00 0x7f81dc004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f94e0700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81dc003730 0x7f81dc003b50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f81f41478e0 tx=0x7f81e8007ae0).ready entity=mon.1 client_cookie=9f43579c7f4ed7a5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e800c660 con 0x7f81dc003730 2020-07-18T03:29:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e800c7c0 con 0x7f81dc003730 2020-07-18T03:29:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81e7fff700 1 -- 172.21.15.47:0/4274611049 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e800cb20 con 0x7f81dc003730 2020-07-18T03:29:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81fb744700 1 -- 172.21.15.47:0/4274611049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81dc003730 msgr2=0x7f81dc003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81fb744700 1 --2- 172.21.15.47:0/4274611049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81dc003730 0x7f81dc003b50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f81f41478e0 tx=0x7f81e8007ae0).stop 2020-07-18T03:29:59.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81f9ce1700 1 -- 172.21.15.47:0/4274611049 reap_dead start 2020-07-18T03:29:59.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81fb744700 1 -- 172.21.15.47:0/4274611049 shutdown_connections 2020-07-18T03:29:59.370 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81fb744700 1 -- 172.21.15.47:0/4274611049 >> 172.21.15.47:0/4274611049 conn(0x7f81f40b9340 msgr2=0x7f81f40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81fb744700 1 -- 172.21.15.47:0/4274611049 shutdown_connections 2020-07-18T03:29:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.352+0000 7f81fb744700 1 -- 172.21.15.47:0/4274611049 wait complete. 2020-07-18T03:29:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 Processor -- start 2020-07-18T03:29:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 -- start start 2020-07-18T03:29:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f81f4063db0 0x7f81f4066480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4151790 0x7f81f41451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f4144a70 con 0x7f81f40639d0 2020-07-18T03:29:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f4151d90 con 0x7f81f4063db0 2020-07-18T03:29:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.354+0000 7f81fb744700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4144d10 con 0x7f81f4151790 2020-07-18T03:29:59.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f94e0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f81f40639d0 0x7f81f4065d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42790/0 (socket says 172.21.15.47:42790) 2020-07-18T03:29:59.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f94e0700 1 -- 172.21.15.47:0/4017577053 learned_addr learned my addr 172.21.15.47:0/4017577053 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:59.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f9ce1700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4151790 0x7f81f41451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f8cdf700 1 --2- 172.21.15.47:0/4017577053 >> v2:172.21.15.18:3300/0 conn(0x7f81f4063db0 0x7f81f4066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559602822 0 0) 0x7f81f4144a70 con 0x7f81f40639d0 2020-07-18T03:29:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81cc003240 con 0x7f81f40639d0 2020-07-18T03:29:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3819258991 0 0) 0x7f81cc003240 con 0x7f81f40639d0 2020-07-18T03:29:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81f4144a70 con 0x7f81f40639d0 2020-07-18T03:29:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f81e800c660 con 0x7f81f40639d0 2020-07-18T03:29:59.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f9ce1700 1 -- 172.21.15.47:0/4017577053 >> v2:172.21.15.18:3300/0 conn(0x7f81f4063db0 msgr2=0x7f81f4066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f9ce1700 1 --2- 172.21.15.47:0/4017577053 >> v2:172.21.15.18:3300/0 conn(0x7f81f4063db0 0x7f81f4066480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f9ce1700 1 -- 172.21.15.47:0/4017577053 >> v1:172.21.15.47:6789/0 conn(0x7f81f40639d0 legacy=0x7f81f4065d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f9ce1700 1 -- 172.21.15.47:0/4017577053 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f41426b0 con 0x7f81f4151790 2020-07-18T03:29:59.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f8cdf700 1 --2- 172.21.15.47:0/4017577053 >> v2:172.21.15.18:3300/0 conn(0x7f81f4063db0 0x7f81f4066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:29:59.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81f9ce1700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4151790 0x7f81f41451b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f81ec0011d0 tx=0x7f81f40b03e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ec00e730 con 0x7f81f4151790 2020-07-18T03:29:59.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec00e890 con 0x7f81f4151790 2020-07-18T03:29:59.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.355+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f4066970 con 0x7f81f4151790 2020-07-18T03:29:59.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.356+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ec00ebf0 con 0x7f81f4151790 2020-07-18T03:29:59.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.356+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f41456a0 con 0x7f81f4151790 2020-07-18T03:29:59.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.356+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4057450 con 0x7f81f4151790 2020-07-18T03:29:59.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.357+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f81ec00ed50 con 0x7f81f4151790 2020-07-18T03:29:59.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.357+0000 7f81e57fa700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f81cc0505e0 0x7f81cc052aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.357+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f81ec0675e0 con 0x7f81f4151790 2020-07-18T03:29:59.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.357+0000 7f81f94e0700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f81cc0505e0 0x7f81cc052aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.357+0000 7f81f94e0700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f81cc0505e0 0x7f81cc052aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f81e8000fc0 tx=0x7f81e8016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.362+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f81ec03fcf0 con 0x7f81f4151790 2020-07-18T03:29:59.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.520+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81f4144e50 con 0x7f81f4151790 2020-07-18T03:29:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.522+0000 7f81e57fa700 1 -- 172.21.15.47:0/4017577053 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16289 (secure 0 0 0) 0x7f81ec043310 con 0x7f81f4151790 2020-07-18T03:29:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:29:59.524 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:58.860824+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:55.419365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:56.654688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:29:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f81cc0505e0 msgr2=0x7f81cc052aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f81cc0505e0 0x7f81cc052aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f81e8000fc0 tx=0x7f81e8016000).stop 2020-07-18T03:29:59.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4151790 msgr2=0x7f81f41451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4151790 0x7f81f41451b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f81ec0011d0 tx=0x7f81f40b03e0).stop 2020-07-18T03:29:59.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 shutdown_connections 2020-07-18T03:29:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f81f4151790 0x7f81f41451b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 --2- 172.21.15.47:0/4017577053 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f81cc0505e0 0x7f81cc052aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 --2- 172.21.15.47:0/4017577053 >> v2:172.21.15.18:3300/0 conn(0x7f81f4063db0 0x7f81f4066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 >> 172.21.15.47:0/4017577053 conn(0x7f81f40b9340 msgr2=0x7f81f41469f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 shutdown_connections 2020-07-18T03:29:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.523+0000 7f81fb744700 1 -- 172.21.15.47:0/4017577053 wait complete. 2020-07-18T03:29:59.536 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:29:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.656+0000 7f4da957b700 1 Processor -- start 2020-07-18T03:29:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.656+0000 7f4da957b700 1 -- start start 2020-07-18T03:29:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da957b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4da41575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da957b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da4080bb0 con 0x7f4da4080510 2020-07-18T03:29:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da957b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da4157140 con 0x7f4da4153f80 2020-07-18T03:29:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da957b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40808f0 con 0x7f4da4153ad0 2020-07-18T03:29:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4da4153f80 0x7f4da4063450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42802/0 (socket says 172.21.15.47:42802) 2020-07-18T03:29:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da3fff700 1 -- 172.21.15.47:0/297669353 learned_addr learned my addr 172.21.15.47:0/297669353 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4da41575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110512677 0 0) 0x7f4da4157140 con 0x7f4da4153f80 2020-07-18T03:29:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.657+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8c003240 con 0x7f4da4153f80 2020-07-18T03:29:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590516887 0 0) 0x7f4da4080bb0 con 0x7f4da4080510 2020-07-18T03:29:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da4157140 con 0x7f4da4080510 2020-07-18T03:29:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2587029831 0 0) 0x7f4d8c003240 con 0x7f4da4153f80 2020-07-18T03:29:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 msgr2=0x7f4da41575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4da41575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 >> v1:172.21.15.18:6789/0 conn(0x7f4da4080510 legacy=0x7f4da40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4080e50 con 0x7f4da4153f80 2020-07-18T03:29:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f4d9c001a50 con 0x7f4da4153f80 2020-07-18T03:29:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 >> v1:172.21.15.47:6789/0 conn(0x7f4da4153f80 legacy=0x7f4da4063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d8c0038c0 0x7f4d8c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c0044e0 0x7f4d8c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4d8c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40808f0 con 0x7f4d8c0038c0 2020-07-18T03:29:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c005340 con 0x7f4d8c0044e0 2020-07-18T03:29:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c04f770 con 0x7f4da4153ad0 2020-07-18T03:29:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c0044e0 0x7f4d8c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da37fe700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d8c0038c0 0x7f4d8c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da3fff700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4d8c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 -- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 msgr2=0x7f4d8c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4d8c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 -- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d8c0038c0 msgr2=0x7f4d8c003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:29:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d8c0038c0 0x7f4d8c003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 -- 172.21.15.47:0/297669353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c003240 con 0x7f4d8c0044e0 2020-07-18T03:29:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da3fff700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f4da4153ad0 0x7f4d8c004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:29:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c0044e0 0x7f4d8c004960 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f4d9800c000 tx=0x7f4d98008040).ready entity=mon.1 client_cookie=d2ebaa5c904ffd14 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.658+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9800eb60 con 0x7f4d8c0044e0 2020-07-18T03:29:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9800ecc0 con 0x7f4d8c0044e0 2020-07-18T03:29:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da27fc700 1 -- 172.21.15.47:0/297669353 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d980125e0 con 0x7f4d8c0044e0 2020-07-18T03:29:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da957b700 1 -- 172.21.15.47:0/297669353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c0044e0 msgr2=0x7f4d8c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da957b700 1 --2- 172.21.15.47:0/297669353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c0044e0 0x7f4d8c004960 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f4d9800c000 tx=0x7f4d98008040).stop 2020-07-18T03:29:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da3fff700 1 -- 172.21.15.47:0/297669353 reap_dead start 2020-07-18T03:29:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da957b700 1 -- 172.21.15.47:0/297669353 shutdown_connections 2020-07-18T03:29:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da957b700 1 -- 172.21.15.47:0/297669353 >> 172.21.15.47:0/297669353 conn(0x7f4da405c620 msgr2=0x7f4da40b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da957b700 1 -- 172.21.15.47:0/297669353 shutdown_connections 2020-07-18T03:29:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.659+0000 7f4da957b700 1 -- 172.21.15.47:0/297669353 wait complete. 2020-07-18T03:29:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 Processor -- start 2020-07-18T03:29:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 -- start start 2020-07-18T03:29:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4da40b5ca0 0x7f4da40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f4da40b3a40 0x7f4da40b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da4140360 con 0x7f4da40b1830 2020-07-18T03:29:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da40b33b0 con 0x7f4da40b3a40 2020-07-18T03:29:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da957b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da40b6120 con 0x7f4da40b5ca0 2020-07-18T03:29:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4da40b1830 0x7f4da40af7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42810/0 (socket says 172.21.15.47:42810) 2020-07-18T03:29:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da37fe700 1 -- 172.21.15.47:0/3346387032 learned_addr learned my addr 172.21.15.47:0/3346387032 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:29:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.661+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/3346387032 >> v2:172.21.15.47:3300/0 conn(0x7f4da40b5ca0 0x7f4da40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da3fff700 1 --2- 172.21.15.47:0/3346387032 >> v2:172.21.15.18:3300/0 conn(0x7f4da40b3a40 0x7f4da40b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553410302 0 0) 0x7f4da4140360 con 0x7f4da40b1830 2020-07-18T03:29:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d80003240 con 0x7f4da40b1830 2020-07-18T03:29:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da2ffd700 1 -- 172.21.15.47:0/3346387032 >> v2:172.21.15.18:3300/0 conn(0x7f4da40b3a40 msgr2=0x7f4da40b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/3346387032 >> v2:172.21.15.18:3300/0 conn(0x7f4da40b3a40 0x7f4da40b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da2ffd700 1 -- 172.21.15.47:0/3346387032 >> v1:172.21.15.47:6789/0 conn(0x7f4da40b1830 legacy=0x7f4da40af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da2ffd700 1 -- 172.21.15.47:0/3346387032 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40b35a0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da2ffd700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da40b5ca0 0x7f4da40b1c10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f4d9800c000 tx=0x7f4d98019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9801c800 con 0x7f4da40b5ca0 2020-07-18T03:29:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9801c960 con 0x7f4da40b5ca0 2020-07-18T03:29:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da413ffe0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9801ccc0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.662+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40b45a0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.663+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da41401d0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.663+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4d98025470 con 0x7f4da40b5ca0 2020-07-18T03:29:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.663+0000 7f4d8affd700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4d800503b0 0x7f4d80052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:29:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.664+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4d980694e0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.664+0000 7f4da37fe700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4d800503b0 0x7f4d80052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:29:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.664+0000 7f4da37fe700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4d800503b0 0x7f4d80052870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f4d9400d000 tx=0x7f4d94007b70).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:29:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.667+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4d98041bb0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.880+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4da40b45a0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.884+0000 7f4d8affd700 1 -- 172.21.15.47:0/3346387032 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4da40b45a0 con 0x7f4da40b5ca0 2020-07-18T03:29:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4d800503b0 msgr2=0x7f4d80052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4d800503b0 0x7f4d80052870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f4d9400d000 tx=0x7f4d94007b70).stop 2020-07-18T03:29:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da40b5ca0 msgr2=0x7f4da40b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:29:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da40b5ca0 0x7f4da40b1c10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f4d9800c000 tx=0x7f4d98019000).stop 2020-07-18T03:29:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 shutdown_connections 2020-07-18T03:29:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 --2- 172.21.15.47:0/3346387032 >> v2:172.21.15.18:3300/0 conn(0x7f4da40b3a40 0x7f4da40b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f4d800503b0 0x7f4d80052870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 --2- 172.21.15.47:0/3346387032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da40b5ca0 0x7f4da40b1c10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:29:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 >> 172.21.15.47:0/3346387032 conn(0x7f4da405c620 msgr2=0x7f4da40621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:29:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 shutdown_connections 2020-07-18T03:29:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:29:59.885+0000 7f4da957b700 1 -- 172.21.15.47:0/3346387032 wait complete. 2020-07-18T03:29:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2020-07-18T03:29:59.898 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:30:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.018+0000 7f7dcdf6e700 1 Processor -- start 2020-07-18T03:30:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.018+0000 7f7dcdf6e700 1 -- start start 2020-07-18T03:30:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7dc8080510 0x7f7dc80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dc8157cf0 0x7f7dc8063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc8080bb0 con 0x7f7dc8153ad0 2020-07-18T03:30:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc8157140 con 0x7f7dc8080510 2020-07-18T03:30:00.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc80808f0 con 0x7f7dc8157cf0 2020-07-18T03:30:00.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dc8157cf0 0x7f7dc8063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dc8157cf0 0x7f7dc8063450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33502/0 (socket says 172.21.15.47:33502) 2020-07-18T03:30:00.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 -- 172.21.15.47:0/181776332 learned_addr learned my addr 172.21.15.47:0/181776332 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:00.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 -- 172.21.15.47:0/181776332 >> v2:172.21.15.18:3300/0 conn(0x7f7dc8080510 msgr2=0x7f7dc80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 --2- 172.21.15.47:0/181776332 >> v2:172.21.15.18:3300/0 conn(0x7f7dc8080510 0x7f7dc80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:30:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 -- 172.21.15.47:0/181776332 >> v1:172.21.15.18:6789/0 conn(0x7f7dc8153ad0 legacy=0x7f7dc81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 -- 172.21.15.47:0/181776332 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8080e50 con 0x7f7dc8157cf0 2020-07-18T03:30:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dccd0c700 1 --2- 172.21.15.47:0/181776332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8157cf0 0x7f7dc8063450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f7dc80b90e0 tx=0x7f7dc0009270).ready entity=mon.1 client_cookie=17a98fb09b8b3471 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dc77fe700 1 -- 172.21.15.47:0/181776332 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dc000bbc0 con 0x7f7dc8157cf0 2020-07-18T03:30:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dc77fe700 1 -- 172.21.15.47:0/181776332 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000bd20 con 0x7f7dc8157cf0 2020-07-18T03:30:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dc77fe700 1 -- 172.21.15.47:0/181776332 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dc000be90 con 0x7f7dc8157cf0 2020-07-18T03:30:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/181776332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8157cf0 msgr2=0x7f7dc8063450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/181776332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8157cf0 0x7f7dc8063450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f7dc80b90e0 tx=0x7f7dc0009270).stop 2020-07-18T03:30:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/181776332 shutdown_connections 2020-07-18T03:30:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/181776332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8157cf0 0x7f7dc8063450 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/181776332 >> v2:172.21.15.18:3300/0 conn(0x7f7dc8080510 0x7f7dc80b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:30:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.019+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/181776332 >> 172.21.15.47:0/181776332 conn(0x7f7dc805c620 msgr2=0x7f7dc80b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.020+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/181776332 shutdown_connections 2020-07-18T03:30:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.020+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/181776332 wait complete. 2020-07-18T03:30:00.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.021+0000 7f7dcdf6e700 1 Processor -- start 2020-07-18T03:30:00.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.021+0000 7f7dcdf6e700 1 -- start start 2020-07-18T03:30:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dcdf6e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dc8080510 0x7f7dc813a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dcdf6e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7dc80b4c80 0x7f7dc815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dcdf6e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc813ac80 con 0x7f7dc8153ad0 2020-07-18T03:30:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dcdf6e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc813af20 con 0x7f7dc80b4c80 2020-07-18T03:30:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dcdf6e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc813b1c0 con 0x7f7dc8080510 2020-07-18T03:30:00.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dc8080510 0x7f7dc813a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dc8080510 0x7f7dc813a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33504/0 (socket says 172.21.15.47:33504) 2020-07-18T03:30:00.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 -- 172.21.15.47:0/756425588 learned_addr learned my addr 172.21.15.47:0/756425588 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:00.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dccd0c700 1 --2- 172.21.15.47:0/756425588 >> v2:172.21.15.18:3300/0 conn(0x7f7dc80b4c80 0x7f7dc815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 -- 172.21.15.47:0/756425588 >> v2:172.21.15.18:3300/0 conn(0x7f7dc80b4c80 msgr2=0x7f7dc815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 --2- 172.21.15.47:0/756425588 >> v2:172.21.15.18:3300/0 conn(0x7f7dc80b4c80 0x7f7dc815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 -- 172.21.15.47:0/756425588 >> v1:172.21.15.18:6789/0 conn(0x7f7dc8153ad0 legacy=0x7f7dc805f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 -- 172.21.15.47:0/756425588 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc813a940 con 0x7f7dc8080510 2020-07-18T03:30:00.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dccd0c700 1 --2- 172.21.15.47:0/756425588 >> v2:172.21.15.18:3300/0 conn(0x7f7dc80b4c80 0x7f7dc815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:30:00.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.022+0000 7f7dc7fff700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8080510 0x7f7dc813a450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f7dc8060400 tx=0x7f7db8006ea0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.023+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7db800d810 con 0x7f7dc8080510 2020-07-18T03:30:00.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.023+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800d970 con 0x7f7dc8080510 2020-07-18T03:30:00.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.023+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc8158160 con 0x7f7dc8080510 2020-07-18T03:30:00.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.023+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7db800dcd0 con 0x7f7dc8080510 2020-07-18T03:30:00.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.023+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc815d230 con 0x7f7dc8080510 2020-07-18T03:30:00.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.024+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7db8011460 con 0x7f7dc8080510 2020-07-18T03:30:00.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.024+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc813b330 con 0x7f7dc8080510 2020-07-18T03:30:00.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.024+0000 7f7dbf7fe700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7da404ff40 0x7f7da4052400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.024+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7db8067970 con 0x7f7dc8080510 2020-07-18T03:30:00.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.025+0000 7f7dbffff700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7da404ff40 0x7f7da4052400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.025+0000 7f7dbffff700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7da404ff40 0x7f7da4052400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f7db0000f20 tx=0x7f7db00077d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.028+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7db8040020 con 0x7f7dc8080510 2020-07-18T03:30:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.240+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7dc80b76b0 con 0x7f7dc8080510 2020-07-18T03:30:00.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.244+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/756425588 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7db8043640 con 0x7f7dc8080510 2020-07-18T03:30:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.245+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7da404ff40 msgr2=0x7f7da4052400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.245+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7da404ff40 0x7f7da4052400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f7db0000f20 tx=0x7f7db00077d0).stop 2020-07-18T03:30:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.245+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8080510 msgr2=0x7f7dc813a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.245+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8080510 0x7f7dc813a450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f7dc8060400 tx=0x7f7db8006ea0).stop 2020-07-18T03:30:00.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 shutdown_connections 2020-07-18T03:30:00.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7da404ff40 0x7f7da4052400 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/756425588 >> v2:172.21.15.18:3300/0 conn(0x7f7dc80b4c80 0x7f7dc815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 --2- 172.21.15.47:0/756425588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8080510 0x7f7dc813a450 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 >> 172.21.15.47:0/756425588 conn(0x7f7dc805c620 msgr2=0x7f7dc80b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:00.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 shutdown_connections 2020-07-18T03:30:00.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.246+0000 7f7dcdf6e700 1 -- 172.21.15.47:0/756425588 wait complete. 2020-07-18T03:30:00.250 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2020-07-18T03:30:00.259 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:30:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.380+0000 7f72d0aad700 1 Processor -- start 2020-07-18T03:30:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.380+0000 7f72d0aad700 1 -- start start 2020-07-18T03:30:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.380+0000 7f72d0aad700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 0x7f72cc0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72d0aad700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f72cc157cf0 0x7f72cc063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72d0aad700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc157140 con 0x7f72cc153ad0 2020-07-18T03:30:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72d0aad700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc080c40 con 0x7f72cc157cf0 2020-07-18T03:30:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72d0aad700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0808f0 con 0x7f72cc080510 2020-07-18T03:30:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cad9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 0x7f72cc0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cb59e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f72cc157cf0 0x7f72cc063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cad9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 0x7f72cc0b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54342/0 (socket says 172.21.15.47:54342) 2020-07-18T03:30:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cad9d700 1 -- 172.21.15.47:0/584600778 learned_addr learned my addr 172.21.15.47:0/584600778 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cb59e700 1 -- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 msgr2=0x7f72cc0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cb59e700 1 --2- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 0x7f72cc0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cb59e700 1 -- 172.21.15.47:0/584600778 >> v1:172.21.15.18:6789/0 conn(0x7f72cc153ad0 legacy=0x7f72cc1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cb59e700 1 -- 172.21.15.47:0/584600778 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc080e50 con 0x7f72cc157cf0 2020-07-18T03:30:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72cad9d700 1 --2- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 0x7f72cc0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.381+0000 7f72ca59c700 1 -- 172.21.15.47:0/584600778 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186909838 0 0) 0x7f72cc157140 con 0x7f72cc153ad0 2020-07-18T03:30:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72cb59e700 1 --2- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72cc157cf0 0x7f72cc063450 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f72c40032b0 tx=0x7f72c40091d0).ready entity=mon.0 client_cookie=159fee2a171aadf9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72ca59c700 1 -- 172.21.15.47:0/584600778 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c400b9a0 con 0x7f72cc157cf0 2020-07-18T03:30:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72ca59c700 1 -- 172.21.15.47:0/584600778 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c400bb00 con 0x7f72cc157cf0 2020-07-18T03:30:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72ca59c700 1 -- 172.21.15.47:0/584600778 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c400bc70 con 0x7f72cc157cf0 2020-07-18T03:30:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 -- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72cc157cf0 msgr2=0x7f72cc063450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 --2- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72cc157cf0 0x7f72cc063450 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f72c40032b0 tx=0x7f72c40091d0).stop 2020-07-18T03:30:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 -- 172.21.15.47:0/584600778 shutdown_connections 2020-07-18T03:30:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 --2- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72cc157cf0 0x7f72cc063450 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 --2- 172.21.15.47:0/584600778 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc080510 0x7f72cc0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 -- 172.21.15.47:0/584600778 >> 172.21.15.47:0/584600778 conn(0x7f72cc05c620 msgr2=0x7f72cc0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 -- 172.21.15.47:0/584600778 shutdown_connections 2020-07-18T03:30:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.382+0000 7f72d0aad700 1 -- 172.21.15.47:0/584600778 wait complete. 2020-07-18T03:30:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 Processor -- start 2020-07-18T03:30:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 -- start start 2020-07-18T03:30:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc153ad0 0x7f72cc0ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72cc15a050 0x7f72cc12b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc12d1f0 con 0x7f72cc080510 2020-07-18T03:30:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc12d490 con 0x7f72cc15a050 2020-07-18T03:30:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.384+0000 7f72d0aad700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc11cde0 con 0x7f72cc153ad0 2020-07-18T03:30:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72cc15a050 0x7f72cc12b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72cc15a050 0x7f72cc12b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33520/0 (socket says 172.21.15.47:33520) 2020-07-18T03:30:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cad9d700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f72cc080510 0x7f72cc11c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:33372/0 (socket says 172.21.15.47:33372) 2020-07-18T03:30:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 -- 172.21.15.47:0/3040212343 learned_addr learned my addr 172.21.15.47:0/3040212343 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72c259c700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc153ad0 0x7f72cc0ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19220451 0 0) 0x7f72cc12d1f0 con 0x7f72cc080510 2020-07-18T03:30:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 -- 172.21.15.47:0/3040212343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc153ad0 msgr2=0x7f72cc0ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc153ad0 0x7f72cc0ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 -- 172.21.15.47:0/3040212343 >> v1:172.21.15.18:6789/0 conn(0x7f72cc080510 legacy=0x7f72cc11c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 -- 172.21.15.47:0/3040212343 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc1584f0 con 0x7f72cc15a050 2020-07-18T03:30:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72cb59e700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc15a050 0x7f72cc12b5e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f72c40032b0 tx=0x7f72c4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c40107a0 con 0x7f72cc15a050 2020-07-18T03:30:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc12bad0 con 0x7f72cc15a050 2020-07-18T03:30:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4010900 con 0x7f72cc15a050 2020-07-18T03:30:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c4010c60 con 0x7f72cc15a050 2020-07-18T03:30:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.385+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc12bee0 con 0x7f72cc15a050 2020-07-18T03:30:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.386+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc0adec0 con 0x7f72cc15a050 2020-07-18T03:30:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.386+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f72c401e070 con 0x7f72cc15a050 2020-07-18T03:30:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.387+0000 7f72c2ffd700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f729c04fea0 0x7f729c052360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.387+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72c4068c40 con 0x7f72cc15a050 2020-07-18T03:30:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.387+0000 7f72cad9d700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f729c04fea0 0x7f729c052360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.387+0000 7f72cad9d700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f729c04fea0 0x7f729c052360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f72bc002f30 tx=0x7f72bc006e90).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.390+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f72cc0adec0 con 0x7f72cc15a050 2020-07-18T03:30:00.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.603+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f72cc0adec0 con 0x7f72cc15a050 2020-07-18T03:30:00.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.607+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3040212343 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7f72cc0adec0 con 0x7f72cc15a050 2020-07-18T03:30:00.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f729c04fea0 msgr2=0x7f729c052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f729c04fea0 0x7f729c052360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f72bc002f30 tx=0x7f72bc006e90).stop 2020-07-18T03:30:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc15a050 msgr2=0x7f72cc12b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc15a050 0x7f72cc12b5e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f72c40032b0 tx=0x7f72c4016000).stop 2020-07-18T03:30:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 shutdown_connections 2020-07-18T03:30:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f729c04fea0 0x7f729c052360 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc15a050 0x7f72cc12b5e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 --2- 172.21.15.47:0/3040212343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f72cc153ad0 0x7f72cc0ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 >> 172.21.15.47:0/3040212343 conn(0x7f72cc05c620 msgr2=0x7f72cc0b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:00.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.608+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 shutdown_connections 2020-07-18T03:30:00.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.609+0000 7f72d0aad700 1 -- 172.21.15.47:0/3040212343 wait complete. 2020-07-18T03:30:00.626 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.2 2020-07-18T03:30:00.627 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:30:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.747+0000 7ff0a0ffa700 1 Processor -- start 2020-07-18T03:30:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.747+0000 7ff0a0ffa700 1 -- start start 2020-07-18T03:30:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff0a0ffa700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff09c080510 0x7ff09c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff0a0ffa700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153ad0 0x7ff09c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff0a0ffa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c080bb0 con 0x7ff09c153f80 2020-07-18T03:30:00.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff0a0ffa700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c157140 con 0x7ff09c080510 2020-07-18T03:30:00.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff0a0ffa700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0808f0 con 0x7ff09c153ad0 2020-07-18T03:30:00.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff09b59e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff09c153f80 0x7ff09c063450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42848/0 (socket says 172.21.15.47:42848) 2020-07-18T03:30:00.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff09b59e700 1 -- 172.21.15.47:0/3658725336 learned_addr learned my addr 172.21.15.47:0/3658725336 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:00.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff09ad9d700 1 --2- 172.21.15.47:0/3658725336 >> v2:172.21.15.18:3300/0 conn(0x7ff09c080510 0x7ff09c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.748+0000 7ff09a59c700 1 --2- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153ad0 0x7ff09c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff099d9b700 1 -- 172.21.15.47:0/3658725336 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711049788 0 0) 0x7ff09c080bb0 con 0x7ff09c153f80 2020-07-18T03:30:00.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff099d9b700 1 -- 172.21.15.47:0/3658725336 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff088003240 con 0x7ff09c153f80 2020-07-18T03:30:00.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff09ad9d700 1 -- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153ad0 msgr2=0x7ff09c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:00.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff09ad9d700 1 --2- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153ad0 0x7ff09c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff09ad9d700 1 -- 172.21.15.47:0/3658725336 >> v1:172.21.15.47:6789/0 conn(0x7ff09c153f80 legacy=0x7ff09c063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff09ad9d700 1 -- 172.21.15.47:0/3658725336 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c080e50 con 0x7ff09c080510 2020-07-18T03:30:00.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff09ad9d700 1 --2- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c080510 0x7ff09c0b9aa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7ff084002f30 tx=0x7ff084006dd0).ready entity=mon.0 client_cookie=4c2f91555d2017f6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff099d9b700 1 -- 172.21.15.47:0/3658725336 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0840096a0 con 0x7ff09c080510 2020-07-18T03:30:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff099d9b700 1 -- 172.21.15.47:0/3658725336 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff084009800 con 0x7ff09c080510 2020-07-18T03:30:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff099d9b700 1 -- 172.21.15.47:0/3658725336 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff084009b60 con 0x7ff09c080510 2020-07-18T03:30:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c080510 msgr2=0x7ff09c0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c080510 0x7ff09c0b9aa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7ff084002f30 tx=0x7ff084006dd0).stop 2020-07-18T03:30:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/3658725336 shutdown_connections 2020-07-18T03:30:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c080510 0x7ff09c0b9aa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/3658725336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153ad0 0x7ff09c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/3658725336 >> 172.21.15.47:0/3658725336 conn(0x7ff09c05c620 msgr2=0x7ff09c0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/3658725336 shutdown_connections 2020-07-18T03:30:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.749+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/3658725336 wait complete. 2020-07-18T03:30:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.751+0000 7ff0a0ffa700 1 Processor -- start 2020-07-18T03:30:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.751+0000 7ff0a0ffa700 1 -- start start 2020-07-18T03:30:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff09c153ad0 0x7ff09c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153f80 0x7ff09c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff09c0ad940 0x7ff09c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c13ea30 con 0x7ff09c153ad0 2020-07-18T03:30:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c13ecd0 con 0x7ff09c0ad940 2020-07-18T03:30:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0b0110 con 0x7ff09c153f80 2020-07-18T03:30:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff09c0ad940 0x7ff09c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff09c0ad940 0x7ff09c0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33536/0 (socket says 172.21.15.47:33536) 2020-07-18T03:30:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 -- 172.21.15.47:0/2811034435 learned_addr learned my addr 172.21.15.47:0/2811034435 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09ad9d700 1 --2- 172.21.15.47:0/2811034435 >> v2:172.21.15.18:3300/0 conn(0x7ff09c153ad0 0x7ff09c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09a59c700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153f80 0x7ff09c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 -- 172.21.15.47:0/2811034435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153f80 msgr2=0x7ff09c156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153f80 0x7ff09c156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 -- 172.21.15.47:0/2811034435 >> v2:172.21.15.18:3300/0 conn(0x7ff09c153ad0 msgr2=0x7ff09c0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 --2- 172.21.15.47:0/2811034435 >> v2:172.21.15.18:3300/0 conn(0x7ff09c153ad0 0x7ff09c0afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 -- 172.21.15.47:0/2811034435 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c13a7e0 con 0x7ff09c0ad940 2020-07-18T03:30:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff09b59e700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff09c0ad940 0x7ff09c0b1920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7ff09c0b1440 tx=0x7ff094007d20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff094005b80 con 0x7ff09c0ad940 2020-07-18T03:30:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094005ce0 con 0x7ff09c0ad940 2020-07-18T03:30:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c156640 con 0x7ff09c0ad940 2020-07-18T03:30:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff094007300 con 0x7ff09c0ad940 2020-07-18T03:30:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.752+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c0b2050 con 0x7ff09c0ad940 2020-07-18T03:30:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.754+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c0af0e0 con 0x7ff09c0ad940 2020-07-18T03:30:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.754+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff094010030 con 0x7ff09c0ad940 2020-07-18T03:30:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.754+0000 7ff0927fc700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff078050030 0x7ff0780524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:00.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.754+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff094067b90 con 0x7ff09c0ad940 2020-07-18T03:30:00.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.754+0000 7ff09ad9d700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff078050030 0x7ff0780524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:00.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.754+0000 7ff09ad9d700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff078050030 0x7ff0780524f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff08401c000 tx=0x7ff084015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.757+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff094040320 con 0x7ff09c0ad940 2020-07-18T03:30:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.976+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff09c0b8f00 con 0x7ff09c0ad940 2020-07-18T03:30:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.979+0000 7ff0927fc700 1 -- 172.21.15.47:0/2811034435 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff094043940 con 0x7ff09c0ad940 2020-07-18T03:30:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff078050030 msgr2=0x7ff0780524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff078050030 0x7ff0780524f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff08401c000 tx=0x7ff084015000).stop 2020-07-18T03:30:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff09c0ad940 msgr2=0x7ff09c0b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff09c0ad940 0x7ff09c0b1920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7ff09c0b1440 tx=0x7ff094007d20).stop 2020-07-18T03:30:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 shutdown_connections 2020-07-18T03:30:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff078050030 0x7ff0780524f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff09c0ad940 0x7ff09c0b1920 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/2811034435 >> v2:172.21.15.18:3300/0 conn(0x7ff09c153ad0 0x7ff09c0afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 --2- 172.21.15.47:0/2811034435 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff09c153f80 0x7ff09c156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:00.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 >> 172.21.15.47:0/2811034435 conn(0x7ff09c05c620 msgr2=0x7ff09c158160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:00.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 shutdown_connections 2020-07-18T03:30:00.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:00.981+0000 7ff0a0ffa700 1 -- 172.21.15.47:0/2811034435 wait complete. 2020-07-18T03:30:00.994 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.3 2020-07-18T03:30:00.996 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:01.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.109+0000 7f6663f19700 1 Processor -- start 2020-07-18T03:30:01.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.110+0000 7f6663f19700 1 -- start start 2020-07-18T03:30:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.110+0000 7f6663f19700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f665c0b5370 0x7f665c05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.110+0000 7f6663f19700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 0x7f665c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.110+0000 7f6663f19700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c0b3ac0 con 0x7f665c0b41c0 2020-07-18T03:30:01.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.110+0000 7f6663f19700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c0b5a10 con 0x7f665c0b5370 2020-07-18T03:30:01.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.110+0000 7f6663f19700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b0040 con 0x7f665c0ad9e0 2020-07-18T03:30:01.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f66614b4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f665c0b41c0 0x7f665c0ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42862/0 (socket says 172.21.15.47:42862) 2020-07-18T03:30:01.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f66614b4700 1 -- 172.21.15.47:0/2885568802 learned_addr learned my addr 172.21.15.47:0/2885568802 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f66624b6700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 0x7f665c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2885568802 >> v2:172.21.15.18:3300/0 conn(0x7f665c0b5370 0x7f665c05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730681112 0 0) 0x7f665c0b3ac0 con 0x7f665c0b41c0 2020-07-18T03:30:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6644003240 con 0x7f665c0b41c0 2020-07-18T03:30:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2633764141 0 0) 0x7f6644003240 con 0x7f665c0b41c0 2020-07-18T03:30:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 msgr2=0x7f665c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 0x7f665c1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 >> v2:172.21.15.18:3300/0 conn(0x7f665c0b5370 msgr2=0x7f665c05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 --2- 172.21.15.47:0/2885568802 >> v2:172.21.15.18:3300/0 conn(0x7f665c0b5370 0x7f665c05fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0b3d20 con 0x7f665c0b41c0 2020-07-18T03:30:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f664c0016d0 con 0x7f665c0b41c0 2020-07-18T03:30:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2885568802 >> v2:172.21.15.18:3300/0 conn(0x7f665c0b5370 0x7f665c05fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 >> v1:172.21.15.47:6789/0 conn(0x7f665c0b41c0 legacy=0x7f665c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 0x7f6644003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6644003c40 0x7f66440044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440049d0 0x7f6644004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b5a10 con 0x7f6644003c40 2020-07-18T03:30:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b0040 con 0x7f66440049d0 2020-07-18T03:30:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6644004200 con 0x7f665c0ad9e0 2020-07-18T03:30:01.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.111+0000 7f66624b6700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440049d0 0x7f6644004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 0x7f6644003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66614b4700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6644003c40 0x7f66440044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 -- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 msgr2=0x7f6644003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:01.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0ad9e0 0x7f6644003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 -- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6644003c40 msgr2=0x7f66440044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:01.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6644003c40 0x7f66440044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 -- 172.21.15.47:0/2885568802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644003240 con 0x7f66440049d0 2020-07-18T03:30:01.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440049d0 0x7f6644004e50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f665400c000 tx=0x7f66540091d0).ready entity=mon.1 client_cookie=2312d01dc3efe84d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:01.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665400ed70 con 0x7f66440049d0 2020-07-18T03:30:01.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665401a470 con 0x7f66440049d0 2020-07-18T03:30:01.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6660cb3700 1 -- 172.21.15.47:0/2885568802 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665401a7d0 con 0x7f66440049d0 2020-07-18T03:30:01.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6663f19700 1 -- 172.21.15.47:0/2885568802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440049d0 msgr2=0x7f6644004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6663f19700 1 --2- 172.21.15.47:0/2885568802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440049d0 0x7f6644004e50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f665400c000 tx=0x7f66540091d0).stop 2020-07-18T03:30:01.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f66624b6700 1 -- 172.21.15.47:0/2885568802 reap_dead start 2020-07-18T03:30:01.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6663f19700 1 -- 172.21.15.47:0/2885568802 shutdown_connections 2020-07-18T03:30:01.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6663f19700 1 -- 172.21.15.47:0/2885568802 >> 172.21.15.47:0/2885568802 conn(0x7f665c0b9340 msgr2=0x7f665c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:01.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6663f19700 1 -- 172.21.15.47:0/2885568802 shutdown_connections 2020-07-18T03:30:01.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.112+0000 7f6663f19700 1 -- 172.21.15.47:0/2885568802 wait complete. 2020-07-18T03:30:01.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6663f19700 1 Processor -- start 2020-07-18T03:30:01.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6663f19700 1 -- start start 2020-07-18T03:30:01.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6663f19700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0b2880 0x7f665c0655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6663f19700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c065030 con 0x7f665c0b32f0 2020-07-18T03:30:01.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6663f19700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c0652d0 con 0x7f665c140060 2020-07-18T03:30:01.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6663f19700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c062230 con 0x7f665c0b2880 2020-07-18T03:30:01.143 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6661cb5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f665c140060 0x7f665c141e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42872/0 (socket says 172.21.15.47:42872) 2020-07-18T03:30:01.143 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f6661cb5700 1 -- 172.21.15.47:0/2184133062 learned_addr learned my addr 172.21.15.47:0/2184133062 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:01.144 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.114+0000 7f66624b6700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0b2880 0x7f665c0655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.144 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531701775 0 0) 0x7f665c0652d0 con 0x7f665c140060 2020-07-18T03:30:01.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003240 con 0x7f665c140060 2020-07-18T03:30:01.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833257014 0 0) 0x7f665c065030 con 0x7f665c0b32f0 2020-07-18T03:30:01.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665c0652d0 con 0x7f665c0b32f0 2020-07-18T03:30:01.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1258891659 0 0) 0x7f6634003240 con 0x7f665c140060 2020-07-18T03:30:01.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f665c065030 con 0x7f665c140060 2020-07-18T03:30:01.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f665800bbd0 con 0x7f665c140060 2020-07-18T03:30:01.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0b2880 msgr2=0x7f665c0655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0b2880 0x7f665c0655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 >> v1:172.21.15.47:6789/0 conn(0x7f665c140060 legacy=0x7f665c141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 >> v1:172.21.15.18:6789/0 conn(0x7f665c0b32f0 legacy=0x7f665c0626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6634003970 0x7f6634003d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66340044e0 0x7f6634004960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6634004e50 0x7f663400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c062230 con 0x7f6634003970 2020-07-18T03:30:01.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663400bc20 con 0x7f66340044e0 2020-07-18T03:30:01.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663400bd60 con 0x7f6634004e50 2020-07-18T03:30:01.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66624b6700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f665c0b2880 0x7f665c0655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:01.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6634003970 0x7f6634003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66614b4700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66340044e0 0x7f6634004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66624b6700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6634004e50 0x7f663400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66614b4700 1 -- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6634004e50 msgr2=0x7f663400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66614b4700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6634004e50 0x7f663400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66614b4700 1 -- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6634003970 msgr2=0x7f6634003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66614b4700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6634003970 0x7f6634003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.115+0000 7f66614b4700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c142550 con 0x7f66340044e0 2020-07-18T03:30:01.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c062de0 con 0x7f66340044e0 2020-07-18T03:30:01.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6634003970 0x7f6634003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:01.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f66614b4700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66340044e0 0x7f6634004960 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f664c001b80 tx=0x7f665c0b3eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:01.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c140440 con 0x7f66340044e0 2020-07-18T03:30:01.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664c0116b0 con 0x7f66340044e0 2020-07-18T03:30:01.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c011810 con 0x7f66340044e0 2020-07-18T03:30:01.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.116+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664c011b70 con 0x7f66340044e0 2020-07-18T03:30:01.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.117+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f664c011cd0 con 0x7f66340044e0 2020-07-18T03:30:01.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.117+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c064e30 con 0x7f66340044e0 2020-07-18T03:30:01.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.117+0000 7f6649ffb700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66340588e0 0x7f663405ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.117+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f664c067ba0 con 0x7f66340044e0 2020-07-18T03:30:01.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.117+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66340588e0 0x7f663405ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.117+0000 7f6661cb5700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66340588e0 0x7f663405ada0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f6658012000 tx=0x7f6658008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:01.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.121+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f664c0403e0 con 0x7f66340044e0 2020-07-18T03:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.329+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f665c057450 con 0x7f66340044e0 2020-07-18T03:30:01.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.329+0000 7f6649ffb700 1 -- 172.21.15.47:0/2184133062 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f664c043a00 con 0x7f66340044e0 2020-07-18T03:30:01.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66340588e0 msgr2=0x7f663405ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f6663f19700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66340588e0 0x7f663405ada0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f6658012000 tx=0x7f6658008040).stop 2020-07-18T03:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66340044e0 msgr2=0x7f6634004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f6663f19700 1 --2- 172.21.15.47:0/2184133062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66340044e0 0x7f6634004960 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f664c001b80 tx=0x7f665c0b3eb0).stop 2020-07-18T03:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f66624b6700 1 -- 172.21.15.47:0/2184133062 reap_dead start 2020-07-18T03:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f66624b6700 1 -- 172.21.15.47:0/2184133062 reap_dead start 2020-07-18T03:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 shutdown_connections 2020-07-18T03:30:01.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.330+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 >> 172.21.15.47:0/2184133062 conn(0x7f665c0b9340 msgr2=0x7f665c0b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:01.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.331+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 shutdown_connections 2020-07-18T03:30:01.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.331+0000 7f6663f19700 1 -- 172.21.15.47:0/2184133062 wait complete. 2020-07-18T03:30:01.340 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:01.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:01.341 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:01.545 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:57.859680+0000', 'modified': '2020-07-18T03:29:58.861271+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4973': {'gid': 4973, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/3198086169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 3198086169}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 3198086169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:30:01.546 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:30:01.547 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-07-18T03:30:01.698 INFO:teuthology.orchestra.run.smithi018.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4973,"want_state":"up:active","state":"up:active","rank_uptime":3.827313899,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":53,"uptime":9.2378574960000002} 2020-07-18T03:30:01.698 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4973, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 53, "rank_uptime": 3.827313899, "state": "up:active", "uptime": 9.237857496, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:30:01.699 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:30:01.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.815+0000 7f7140e81700 1 Processor -- start 2020-07-18T03:30:01.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.815+0000 7f7140e81700 1 -- start start 2020-07-18T03:30:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f7140e81700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f713c063d20 0x7f713c0b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f7140e81700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f713c0641d0 0x7f713c0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f7140e81700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713c060ff0 con 0x7f713c147150 2020-07-18T03:30:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f7140e81700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c147530 con 0x7f713c063d20 2020-07-18T03:30:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f7140e81700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c147740 con 0x7f713c0641d0 2020-07-18T03:30:01.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f713a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f713c147150 0x7f713c1445c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42886/0 (socket says 172.21.15.47:42886) 2020-07-18T03:30:01.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f713a59c700 1 -- 172.21.15.47:0/3976334650 learned_addr learned my addr 172.21.15.47:0/3976334650 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:01.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.816+0000 7f713ad9d700 1 --2- 172.21.15.47:0/3976334650 >> v2:172.21.15.47:3300/0 conn(0x7f713c0641d0 0x7f713c0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7139d9b700 1 --2- 172.21.15.47:0/3976334650 >> v2:172.21.15.18:3300/0 conn(0x7f713c063d20 0x7f713c0b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713ad9d700 1 -- 172.21.15.47:0/3976334650 >> v2:172.21.15.18:3300/0 conn(0x7f713c063d20 msgr2=0x7f713c0b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:01.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713ad9d700 1 --2- 172.21.15.47:0/3976334650 >> v2:172.21.15.18:3300/0 conn(0x7f713c063d20 0x7f713c0b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713ad9d700 1 -- 172.21.15.47:0/3976334650 >> v1:172.21.15.47:6789/0 conn(0x7f713c147150 legacy=0x7f713c1445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713ad9d700 1 -- 172.21.15.47:0/3976334650 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c1479e0 con 0x7f713c0641d0 2020-07-18T03:30:01.823 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713ad9d700 1 --2- 172.21.15.47:0/3976334650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c0641d0 0x7f713c0b8400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f7134002f30 tx=0x7f7134006dd0).ready entity=mon.1 client_cookie=3f88a1bac8423586 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:01.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713959a700 1 -- 172.21.15.47:0/3976334650 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71340097f0 con 0x7f713c0641d0 2020-07-18T03:30:01.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713959a700 1 -- 172.21.15.47:0/3976334650 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7134009950 con 0x7f713c0641d0 2020-07-18T03:30:01.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f713959a700 1 -- 172.21.15.47:0/3976334650 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7134009cb0 con 0x7f713c0641d0 2020-07-18T03:30:01.824 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 -- 172.21.15.47:0/3976334650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c0641d0 msgr2=0x7f713c0b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 --2- 172.21.15.47:0/3976334650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c0641d0 0x7f713c0b8400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f7134002f30 tx=0x7f7134006dd0).stop 2020-07-18T03:30:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 -- 172.21.15.47:0/3976334650 shutdown_connections 2020-07-18T03:30:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 --2- 172.21.15.47:0/3976334650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c0641d0 0x7f713c0b8400 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 --2- 172.21.15.47:0/3976334650 >> v2:172.21.15.18:3300/0 conn(0x7f713c063d20 0x7f713c0b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 -- 172.21.15.47:0/3976334650 >> 172.21.15.47:0/3976334650 conn(0x7f713c05f5a0 msgr2=0x7f713c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 -- 172.21.15.47:0/3976334650 shutdown_connections 2020-07-18T03:30:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.817+0000 7f7140e81700 1 -- 172.21.15.47:0/3976334650 wait complete. 2020-07-18T03:30:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 Processor -- start 2020-07-18T03:30:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 -- start start 2020-07-18T03:30:01.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f713c063d20 0x7f713c1419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f713c147150 0x7f713c140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f713c1414f0 0x7f713c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c14e350 con 0x7f713c147150 2020-07-18T03:30:01.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c140d80 con 0x7f713c063d20 2020-07-18T03:30:01.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.819+0000 7f7140e81700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c140ef0 con 0x7f713c1414f0 2020-07-18T03:30:01.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f713c063d20 0x7f713c1419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f713c063d20 0x7f713c1419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33572/0 (socket says 172.21.15.47:33572) 2020-07-18T03:30:01.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 -- 172.21.15.47:0/2746286976 learned_addr learned my addr 172.21.15.47:0/2746286976 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:01.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f7139d9b700 1 --2- 172.21.15.47:0/2746286976 >> v2:172.21.15.18:3300/0 conn(0x7f713c147150 0x7f713c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713ad9d700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f713c1414f0 0x7f713c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 -- 172.21.15.47:0/2746286976 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f713c1414f0 msgr2=0x7f713c148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f713c1414f0 0x7f713c148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 -- 172.21.15.47:0/2746286976 >> v2:172.21.15.18:3300/0 conn(0x7f713c147150 msgr2=0x7f713c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 --2- 172.21.15.47:0/2746286976 >> v2:172.21.15.18:3300/0 conn(0x7f713c147150 0x7f713c140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 -- 172.21.15.47:0/2746286976 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c141060 con 0x7f713c063d20 2020-07-18T03:30:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f713a59c700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c063d20 0x7f713c1419f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f713c063a80 tx=0x7f7130007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:01.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71300072e0 con 0x7f713c063d20 2020-07-18T03:30:01.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7130007440 con 0x7f713c063d20 2020-07-18T03:30:01.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c1493e0 con 0x7f713c063d20 2020-07-18T03:30:01.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71300057e0 con 0x7f713c063d20 2020-07-18T03:30:01.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.820+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c149930 con 0x7f713c063d20 2020-07-18T03:30:01.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.821+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c057450 con 0x7f713c063d20 2020-07-18T03:30:01.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.821+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f713001a060 con 0x7f713c063d20 2020-07-18T03:30:01.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.822+0000 7f712a7fc700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7110050030 0x7f71100524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:01.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.822+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7130066f40 con 0x7f713c063d20 2020-07-18T03:30:01.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.822+0000 7f7139d9b700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7110050030 0x7f71100524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:01.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.822+0000 7f7139d9b700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7110050030 0x7f71100524f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f712c00c000 tx=0x7f712c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:01.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.825+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f713003f6d0 con 0x7f713c063d20 2020-07-18T03:30:01.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.982+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f713c064ad0 con 0x7f713c063d20 2020-07-18T03:30:01.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.983+0000 7f712a7fc700 1 -- 172.21.15.47:0/2746286976 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16276 (secure 0 0 0) 0x7f7130018020 con 0x7f713c063d20 2020-07-18T03:30:01.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7110050030 msgr2=0x7f71100524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7110050030 0x7f71100524f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f712c00c000 tx=0x7f712c008040).stop 2020-07-18T03:30:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c063d20 msgr2=0x7f713c1419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c063d20 0x7f713c1419f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f713c063a80 tx=0x7f7130007c20).stop 2020-07-18T03:30:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 shutdown_connections 2020-07-18T03:30:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7110050030 0x7f71100524f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f713c1414f0 0x7f713c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 --2- 172.21.15.47:0/2746286976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f713c063d20 0x7f713c1419f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 --2- 172.21.15.47:0/2746286976 >> v2:172.21.15.18:3300/0 conn(0x7f713c147150 0x7f713c140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:01.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 >> 172.21.15.47:0/2746286976 conn(0x7f713c05f5a0 msgr2=0x7f713c0622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:01.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.984+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 shutdown_connections 2020-07-18T03:30:01.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:01.985+0000 7f7140e81700 1 -- 172.21.15.47:0/2746286976 wait complete. 2020-07-18T03:30:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:59.865840+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:55.419365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:56.654688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:30:02.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:29:55.419365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:29:56.654688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:30:02.002 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:30:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.124+0000 7fac34492700 1 Processor -- start 2020-07-18T03:30:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.124+0000 7fac34492700 1 -- start start 2020-07-18T03:30:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.125+0000 7fac34492700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fac2c0ad330 0x7fac2c143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.125+0000 7fac34492700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2c0b3a10 0x7fac2c0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.125+0000 7fac34492700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c147de0 con 0x7fac2c0b59b0 2020-07-18T03:30:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.125+0000 7fac34492700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c0ae000 con 0x7fac2c0ad330 2020-07-18T03:30:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.125+0000 7fac34492700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c148010 con 0x7fac2c0b3a10 2020-07-18T03:30:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.125+0000 7fac32a2f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2c0b3a10 0x7fac2c0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac31a2d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac2c0b59b0 0x7fac2c0b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42900/0 (socket says 172.21.15.47:42900) 2020-07-18T03:30:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac32a2f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2c0b3a10 0x7fac2c0b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33584/0 (socket says 172.21.15.47:33584) 2020-07-18T03:30:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac31a2d700 1 -- 172.21.15.47:0/3709048362 learned_addr learned my addr 172.21.15.47:0/3709048362 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac3222e700 1 --2- 172.21.15.47:0/3709048362 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0ad330 0x7fac2c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac32a2f700 1 -- 172.21.15.47:0/3709048362 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0ad330 msgr2=0x7fac2c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac32a2f700 1 --2- 172.21.15.47:0/3709048362 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0ad330 0x7fac2c143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac32a2f700 1 -- 172.21.15.47:0/3709048362 >> v1:172.21.15.47:6789/0 conn(0x7fac2c0b59b0 legacy=0x7fac2c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac3122c700 1 -- 172.21.15.47:0/3709048362 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016045277 0 0) 0x7fac2c147de0 con 0x7fac2c0b59b0 2020-07-18T03:30:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac32a2f700 1 -- 172.21.15.47:0/3709048362 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c0ad710 con 0x7fac2c0b3a10 2020-07-18T03:30:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac32a2f700 1 --2- 172.21.15.47:0/3709048362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c0b3a10 0x7fac2c0b3e80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fac240032b0 tx=0x7fac240091d0).ready entity=mon.1 client_cookie=689330600c4ecf43 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac3122c700 1 -- 172.21.15.47:0/3709048362 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac2400ba50 con 0x7fac2c0b3a10 2020-07-18T03:30:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac3122c700 1 -- 172.21.15.47:0/3709048362 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2400bbb0 con 0x7fac2c0b3a10 2020-07-18T03:30:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac3122c700 1 -- 172.21.15.47:0/3709048362 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac2400bd20 con 0x7fac2c0b3a10 2020-07-18T03:30:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 -- 172.21.15.47:0/3709048362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c0b3a10 msgr2=0x7fac2c0b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 --2- 172.21.15.47:0/3709048362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c0b3a10 0x7fac2c0b3e80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fac240032b0 tx=0x7fac240091d0).stop 2020-07-18T03:30:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 -- 172.21.15.47:0/3709048362 shutdown_connections 2020-07-18T03:30:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 --2- 172.21.15.47:0/3709048362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c0b3a10 0x7fac2c0b3e80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 --2- 172.21.15.47:0/3709048362 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0ad330 0x7fac2c143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 -- 172.21.15.47:0/3709048362 >> 172.21.15.47:0/3709048362 conn(0x7fac2c05f5a0 msgr2=0x7fac2c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 -- 172.21.15.47:0/3709048362 shutdown_connections 2020-07-18T03:30:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.126+0000 7fac34492700 1 -- 172.21.15.47:0/3709048362 wait complete. 2020-07-18T03:30:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 Processor -- start 2020-07-18T03:30:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 -- start start 2020-07-18T03:30:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fac2c0b59b0 0x7fac2c144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2c1465a0 0x7fac2c063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c144510 con 0x7fac2c0ad330 2020-07-18T03:30:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c143640 con 0x7fac2c0b59b0 2020-07-18T03:30:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac34492700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c1438e0 con 0x7fac2c1465a0 2020-07-18T03:30:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.128+0000 7fac32a2f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2c1465a0 0x7fac2c063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac2c1465a0 0x7fac2c063da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33590/0 (socket says 172.21.15.47:33590) 2020-07-18T03:30:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 -- 172.21.15.47:0/3548866311 learned_addr learned my addr 172.21.15.47:0/3548866311 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac31a2d700 1 --2- 172.21.15.47:0/3548866311 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0b59b0 0x7fac2c144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 -- 172.21.15.47:0/3548866311 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0b59b0 msgr2=0x7fac2c144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 --2- 172.21.15.47:0/3548866311 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0b59b0 0x7fac2c144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 -- 172.21.15.47:0/3548866311 >> v1:172.21.15.18:6789/0 conn(0x7fac2c0ad330 legacy=0x7fac2c061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578239139 0 0) 0x7fac2c144510 con 0x7fac2c0ad330 2020-07-18T03:30:02.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 -- 172.21.15.47:0/3548866311 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c143b80 con 0x7fac2c1465a0 2020-07-18T03:30:02.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac32a2f700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c1465a0 0x7fac2c063da0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fac240032b0 tx=0x7fac24016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac240108c0 con 0x7fac2c1465a0 2020-07-18T03:30:02.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac24010a20 con 0x7fac2c1465a0 2020-07-18T03:30:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c062290 con 0x7fac2c1465a0 2020-07-18T03:30:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac24010d80 con 0x7fac2c1465a0 2020-07-18T03:30:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.129+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c0644d0 con 0x7fac2c1465a0 2020-07-18T03:30:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.130+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c143d10 con 0x7fac2c1465a0 2020-07-18T03:30:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.130+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fac2401e070 con 0x7fac2c1465a0 2020-07-18T03:30:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.131+0000 7fac1a7fc700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac0404fe90 0x7fac04052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.131+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fac24068e80 con 0x7fac2c1465a0 2020-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.131+0000 7fac3222e700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac0404fe90 0x7fac04052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.131+0000 7fac3222e700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac0404fe90 0x7fac04052350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fac1c006920 tx=0x7fac1c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.134+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fac24041590 con 0x7fac2c1465a0 2020-07-18T03:30:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.293+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fac2c0690d0 con 0x7fac2c1465a0 2020-07-18T03:30:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.294+0000 7fac1a7fc700 1 -- 172.21.15.47:0/3548866311 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v54) v1 ==== 110+0+10 (secure 0 0 0) 0x7fac240150b0 con 0x7fac2c1465a0 2020-07-18T03:30:02.296 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:30:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac0404fe90 msgr2=0x7fac04052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac0404fe90 0x7fac04052350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fac1c006920 tx=0x7fac1c008040).stop 2020-07-18T03:30:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c1465a0 msgr2=0x7fac2c063da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c1465a0 0x7fac2c063da0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fac240032b0 tx=0x7fac24016000).stop 2020-07-18T03:30:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 shutdown_connections 2020-07-18T03:30:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fac0404fe90 0x7fac04052350 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 --2- 172.21.15.47:0/3548866311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac2c1465a0 0x7fac2c063da0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 --2- 172.21.15.47:0/3548866311 >> v2:172.21.15.18:3300/0 conn(0x7fac2c0b59b0 0x7fac2c144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 >> 172.21.15.47:0/3548866311 conn(0x7fac2c05f5a0 msgr2=0x7fac2c0b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 shutdown_connections 2020-07-18T03:30:02.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.295+0000 7fac34492700 1 -- 172.21.15.47:0/3548866311 wait complete. 2020-07-18T03:30:02.308 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:30:02.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.423+0000 7f8b650ec700 1 Processor -- start 2020-07-18T03:30:02.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.423+0000 7f8b650ec700 1 -- start start 2020-07-18T03:30:02.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b650ec700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b60064250 0x7f8b600b8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b650ec700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b60066fa0 con 0x7f8b60063e70 2020-07-18T03:30:02.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b650ec700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b60067240 con 0x7f8b600b8780 2020-07-18T03:30:02.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b650ec700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600b7df0 con 0x7f8b60064250 2020-07-18T03:30:02.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5f59e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b600b8780 0x7f8b601472f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42918/0 (socket says 172.21.15.47:42918) 2020-07-18T03:30:02.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5f59e700 1 -- 172.21.15.47:0/2943692207 learned_addr learned my addr 172.21.15.47:0/2943692207 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:02.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5e59c700 1 --2- 172.21.15.47:0/2943692207 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b60064250 0x7f8b600b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907020249 0 0) 0x7f8b60067240 con 0x7f8b600b8780 2020-07-18T03:30:02.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b4c003240 con 0x7f8b600b8780 2020-07-18T03:30:02.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341148060 0 0) 0x7f8b60066fa0 con 0x7f8b60063e70 2020-07-18T03:30:02.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b60067240 con 0x7f8b60063e70 2020-07-18T03:30:02.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5e59c700 1 -- 172.21.15.47:0/2943692207 >> v1:172.21.15.47:6789/0 conn(0x7f8b600b8780 legacy=0x7f8b601472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5e59c700 1 -- 172.21.15.47:0/2943692207 >> v1:172.21.15.18:6789/0 conn(0x7f8b60063e70 legacy=0x7f8b60066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.424+0000 7f8b5e59c700 1 -- 172.21.15.47:0/2943692207 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b600612f0 con 0x7f8b60064250 2020-07-18T03:30:02.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b5e59c700 1 --2- 172.21.15.47:0/2943692207 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b60064250 0x7f8b600b8290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f8b54002f30 tx=0x7f8b54006dd0).ready entity=mon.2 client_cookie=1d25412c7d1efbe9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b540096e0 con 0x7f8b60064250 2020-07-18T03:30:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54009840 con 0x7f8b60064250 2020-07-18T03:30:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b5dd9b700 1 -- 172.21.15.47:0/2943692207 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b54009ba0 con 0x7f8b60064250 2020-07-18T03:30:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 -- 172.21.15.47:0/2943692207 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b60064250 msgr2=0x7f8b600b8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 --2- 172.21.15.47:0/2943692207 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b60064250 0x7f8b600b8290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f8b54002f30 tx=0x7f8b54006dd0).stop 2020-07-18T03:30:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 -- 172.21.15.47:0/2943692207 shutdown_connections 2020-07-18T03:30:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 --2- 172.21.15.47:0/2943692207 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b60064250 0x7f8b600b8290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 -- 172.21.15.47:0/2943692207 >> 172.21.15.47:0/2943692207 conn(0x7f8b6005f5a0 msgr2=0x7f8b6005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 -- 172.21.15.47:0/2943692207 shutdown_connections 2020-07-18T03:30:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.425+0000 7f8b650ec700 1 -- 172.21.15.47:0/2943692207 wait complete. 2020-07-18T03:30:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.426+0000 7f8b650ec700 1 Processor -- start 2020-07-18T03:30:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.426+0000 7f8b650ec700 1 -- start start 2020-07-18T03:30:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b650ec700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b60063e70 0x7f8b601477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b650ec700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6012d9a0 con 0x7f8b60148010 2020-07-18T03:30:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b650ec700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6012dbd0 con 0x7f8b600b8780 2020-07-18T03:30:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b650ec700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6012de70 con 0x7f8b60063e70 2020-07-18T03:30:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b5ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b60063e70 0x7f8b601477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b5e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b600b8780 0x7f8b600b9b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42922/0 (socket says 172.21.15.47:42922) 2020-07-18T03:30:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b5ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b60063e70 0x7f8b601477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33600/0 (socket says 172.21.15.47:33600) 2020-07-18T03:30:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b5e59c700 1 -- 172.21.15.47:0/3105217502 learned_addr learned my addr 172.21.15.47:0/3105217502 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302219910 0 0) 0x7f8b6012dbd0 con 0x7f8b600b8780 2020-07-18T03:30:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.427+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b3c003240 con 0x7f8b600b8780 2020-07-18T03:30:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b5ed9d700 1 -- 172.21.15.47:0/3105217502 >> v1:172.21.15.47:6789/0 conn(0x7f8b600b8780 legacy=0x7f8b600b9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b5ed9d700 1 -- 172.21.15.47:0/3105217502 >> v1:172.21.15.18:6789/0 conn(0x7f8b60148010 legacy=0x7f8b6012c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b5ed9d700 1 -- 172.21.15.47:0/3105217502 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6012e110 con 0x7f8b60063e70 2020-07-18T03:30:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b5ed9d700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b60063e70 0x7f8b601477b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f8b60146cb0 tx=0x7f8b50007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5000f470 con 0x7f8b60063e70 2020-07-18T03:30:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5000f5d0 con 0x7f8b60063e70 2020-07-18T03:30:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b60147ca0 con 0x7f8b60063e70 2020-07-18T03:30:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5000f930 con 0x7f8b60063e70 2020-07-18T03:30:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.428+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6012cde0 con 0x7f8b60063e70 2020-07-18T03:30:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.429+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6012e2a0 con 0x7f8b60063e70 2020-07-18T03:30:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.429+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8b50019030 con 0x7f8b60063e70 2020-07-18T03:30:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.429+0000 7f8b46ffd700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b3c050330 0x7f8b3c0527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.429+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b500670e0 con 0x7f8b60063e70 2020-07-18T03:30:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.429+0000 7f8b5e59c700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b3c050330 0x7f8b3c0527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.430+0000 7f8b5e59c700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b3c050330 0x7f8b3c0527f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8b54007000 tx=0x7f8b54016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.433+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8b5003f870 con 0x7f8b60063e70 2020-07-18T03:30:02.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.589+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b600b7bd0 con 0x7f8b60063e70 2020-07-18T03:30:02.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.589+0000 7f8b46ffd700 1 -- 172.21.15.47:0/3105217502 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b50042e90 con 0x7f8b60063e70 2020-07-18T03:30:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:30:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.590+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b3c050330 msgr2=0x7f8b3c0527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.590+0000 7f8b650ec700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b3c050330 0x7f8b3c0527f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8b54007000 tx=0x7f8b54016000).stop 2020-07-18T03:30:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.590+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b60063e70 msgr2=0x7f8b601477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.590+0000 7f8b650ec700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b60063e70 0x7f8b601477b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f8b60146cb0 tx=0x7f8b50007a00).stop 2020-07-18T03:30:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.591+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 shutdown_connections 2020-07-18T03:30:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.591+0000 7f8b650ec700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b60063e70 0x7f8b601477b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.591+0000 7f8b650ec700 1 --2- 172.21.15.47:0/3105217502 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b3c050330 0x7f8b3c0527f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.591+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 >> 172.21.15.47:0/3105217502 conn(0x7f8b6005f5a0 msgr2=0x7f8b6005f980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.591+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 shutdown_connections 2020-07-18T03:30:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.591+0000 7f8b650ec700 1 -- 172.21.15.47:0/3105217502 wait complete. 2020-07-18T03:30:02.603 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:30:02.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.718+0000 7f8ff1787700 1 Processor -- start 2020-07-18T03:30:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.718+0000 7f8ff1787700 1 -- start start 2020-07-18T03:30:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8ff1787700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec0ad330 0x7f8fec143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8ff1787700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8fec0b59b0 0x7f8fec0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8ff1787700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec147de0 con 0x7f8fec0b6300 2020-07-18T03:30:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8ff1787700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec0ae000 con 0x7f8fec0b59b0 2020-07-18T03:30:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8ff1787700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec148010 con 0x7f8fec0ad330 2020-07-18T03:30:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8feb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8fec0b6300 0x7f8fec0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42928/0 (socket says 172.21.15.47:42928) 2020-07-18T03:30:02.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8feb7fe700 1 -- 172.21.15.47:0/4216289163 learned_addr learned my addr 172.21.15.47:0/4216289163 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:02.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8feaffd700 1 --2- 172.21.15.47:0/4216289163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec0ad330 0x7f8fec143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.719+0000 7f8fea7fc700 1 --2- 172.21.15.47:0/4216289163 >> v2:172.21.15.18:3300/0 conn(0x7f8fec0b59b0 0x7f8fec0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8fe9ffb700 1 -- 172.21.15.47:0/4216289163 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543937184 0 0) 0x7f8fec147de0 con 0x7f8fec0b6300 2020-07-18T03:30:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8fe9ffb700 1 -- 172.21.15.47:0/4216289163 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc003240 con 0x7f8fec0b6300 2020-07-18T03:30:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8feaffd700 1 -- 172.21.15.47:0/4216289163 >> v2:172.21.15.18:3300/0 conn(0x7f8fec0b59b0 msgr2=0x7f8fec0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:02.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8feaffd700 1 --2- 172.21.15.47:0/4216289163 >> v2:172.21.15.18:3300/0 conn(0x7f8fec0b59b0 0x7f8fec0b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8feaffd700 1 -- 172.21.15.47:0/4216289163 >> v1:172.21.15.47:6789/0 conn(0x7f8fec0b6300 legacy=0x7f8fec0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8feaffd700 1 -- 172.21.15.47:0/4216289163 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0ad710 con 0x7f8fec0ad330 2020-07-18T03:30:02.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8feaffd700 1 --2- 172.21.15.47:0/4216289163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec0ad330 0x7f8fec143640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f8fd4002f30 tx=0x7f8fd4006dd0).ready entity=mon.2 client_cookie=2f2b459c4721e86c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8fe9ffb700 1 -- 172.21.15.47:0/4216289163 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd400f9c0 con 0x7f8fec0ad330 2020-07-18T03:30:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8fe9ffb700 1 -- 172.21.15.47:0/4216289163 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd400fb20 con 0x7f8fec0ad330 2020-07-18T03:30:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8fe9ffb700 1 -- 172.21.15.47:0/4216289163 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd400fe80 con 0x7f8fec0ad330 2020-07-18T03:30:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 -- 172.21.15.47:0/4216289163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec0ad330 msgr2=0x7f8fec143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 --2- 172.21.15.47:0/4216289163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec0ad330 0x7f8fec143640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f8fd4002f30 tx=0x7f8fd4006dd0).stop 2020-07-18T03:30:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 -- 172.21.15.47:0/4216289163 shutdown_connections 2020-07-18T03:30:02.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 --2- 172.21.15.47:0/4216289163 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec0ad330 0x7f8fec143640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 --2- 172.21.15.47:0/4216289163 >> v2:172.21.15.18:3300/0 conn(0x7f8fec0b59b0 0x7f8fec0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 -- 172.21.15.47:0/4216289163 >> 172.21.15.47:0/4216289163 conn(0x7f8fec05f5a0 msgr2=0x7f8fec05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:02.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 -- 172.21.15.47:0/4216289163 shutdown_connections 2020-07-18T03:30:02.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.720+0000 7f8ff1787700 1 -- 172.21.15.47:0/4216289163 wait complete. 2020-07-18T03:30:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.722+0000 7f8ff1787700 1 Processor -- start 2020-07-18T03:30:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.722+0000 7f8ff1787700 1 -- start start 2020-07-18T03:30:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.722+0000 7f8ff1787700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec1583b0 0x7f8fec13e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.722+0000 7f8ff1787700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec064360 con 0x7f8fec0b59b0 2020-07-18T03:30:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.722+0000 7f8ff1787700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec064600 con 0x7f8fec0b6300 2020-07-18T03:30:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.722+0000 7f8ff1787700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0648a0 con 0x7f8fec1583b0 2020-07-18T03:30:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fea7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8fec0b6300 0x7f8fec133c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42936/0 (socket says 172.21.15.47:42936) 2020-07-18T03:30:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fea7fc700 1 -- 172.21.15.47:0/925993645 learned_addr learned my addr 172.21.15.47:0/925993645 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8feb7fe700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec1583b0 0x7f8fec13e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139136711 0 0) 0x7f8fec064600 con 0x7f8fec0b6300 2020-07-18T03:30:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc8003240 con 0x7f8fec0b6300 2020-07-18T03:30:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248120999 0 0) 0x7f8fec064360 con 0x7f8fec0b59b0 2020-07-18T03:30:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fec064600 con 0x7f8fec0b59b0 2020-07-18T03:30:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2819085747 0 0) 0x7f8fc8003240 con 0x7f8fec0b6300 2020-07-18T03:30:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fec064360 con 0x7f8fec0b6300 2020-07-18T03:30:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8fdc00b7f0 con 0x7f8fec0b6300 2020-07-18T03:30:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8feb7fe700 1 -- 172.21.15.47:0/925993645 >> v1:172.21.15.47:6789/0 conn(0x7f8fec0b6300 legacy=0x7f8fec133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8feb7fe700 1 -- 172.21.15.47:0/925993645 >> v1:172.21.15.18:6789/0 conn(0x7f8fec0b59b0 legacy=0x7f8fec158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8feb7fe700 1 -- 172.21.15.47:0/925993645 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec140ae0 con 0x7f8fec1583b0 2020-07-18T03:30:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8feb7fe700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec1583b0 0x7f8fec13e710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f8fe4001450 tx=0x7f8fe4007c80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fe4005970 con 0x7f8fec1583b0 2020-07-18T03:30:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4005ad0 con 0x7f8fec1583b0 2020-07-18T03:30:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec134310 con 0x7f8fec1583b0 2020-07-18T03:30:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fe4005e30 con 0x7f8fec1583b0 2020-07-18T03:30:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.723+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1347b0 con 0x7f8fec1583b0 2020-07-18T03:30:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.724+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec05b120 con 0x7f8fec1583b0 2020-07-18T03:30:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.725+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8fe4010030 con 0x7f8fec1583b0 2020-07-18T03:30:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.725+0000 7f8fe2ffd700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8fc80506d0 0x7f8fc8052b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.725+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8fe40676a0 con 0x7f8fec1583b0 2020-07-18T03:30:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.725+0000 7f8feaffd700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8fc80506d0 0x7f8fc8052b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.725+0000 7f8feaffd700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8fc80506d0 0x7f8fc8052b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f8fec065e00 tx=0x7f8fd4017000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.729+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8fe403fdd0 con 0x7f8fec1583b0 2020-07-18T03:30:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.894+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fec0690d0 con 0x7f8fec1583b0 2020-07-18T03:30:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.895+0000 7f8fe2ffd700 1 -- 172.21.15.47:0/925993645 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8fe40433f0 con 0x7f8fec1583b0 2020-07-18T03:30:02.897 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:30:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8fc80506d0 msgr2=0x7f8fc8052b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8fc80506d0 0x7f8fc8052b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f8fec065e00 tx=0x7f8fd4017000).stop 2020-07-18T03:30:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec1583b0 msgr2=0x7f8fec13e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec1583b0 0x7f8fec13e710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f8fe4001450 tx=0x7f8fe4007c80).stop 2020-07-18T03:30:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 shutdown_connections 2020-07-18T03:30:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fec1583b0 0x7f8fec13e710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 --2- 172.21.15.47:0/925993645 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8fc80506d0 0x7f8fc8052b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 >> 172.21.15.47:0/925993645 conn(0x7f8fec05f5a0 msgr2=0x7f8fec0af300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 shutdown_connections 2020-07-18T03:30:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:02.896+0000 7f8ff1787700 1 -- 172.21.15.47:0/925993645 wait complete. 2020-07-18T03:30:02.909 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:30:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.025+0000 7f7422749700 1 Processor -- start 2020-07-18T03:30:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.025+0000 7f7422749700 1 -- start start 2020-07-18T03:30:03.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f7422749700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f741c0b59b0 0x7f741c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f7422749700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f741c0b6300 0x7f741c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f7422749700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c147de0 con 0x7f741c0ad330 2020-07-18T03:30:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f7422749700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c0ae000 con 0x7f741c0b6300 2020-07-18T03:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f7422749700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c148010 con 0x7f741c0b59b0 2020-07-18T03:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f741c0b59b0 0x7f741c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f741c0ad330 0x7f741c143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42942/0 (socket says 172.21.15.47:42942) 2020-07-18T03:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741bfff700 1 -- 172.21.15.47:0/1075016303 learned_addr learned my addr 172.21.15.47:0/1075016303 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f7420ce6700 1 --2- 172.21.15.47:0/1075016303 >> v2:172.21.15.18:3300/0 conn(0x7f741c0b6300 0x7f741c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741b7fe700 1 -- 172.21.15.47:0/1075016303 >> v2:172.21.15.18:3300/0 conn(0x7f741c0b6300 msgr2=0x7f741c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741b7fe700 1 --2- 172.21.15.47:0/1075016303 >> v2:172.21.15.18:3300/0 conn(0x7f741c0b6300 0x7f741c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741b7fe700 1 -- 172.21.15.47:0/1075016303 >> v1:172.21.15.47:6789/0 conn(0x7f741c0ad330 legacy=0x7f741c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741b7fe700 1 -- 172.21.15.47:0/1075016303 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0ad710 con 0x7f741c0b59b0 2020-07-18T03:30:03.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741affd700 1 -- 172.21.15.47:0/1075016303 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208421255 0 0) 0x7f741c147de0 con 0x7f741c0ad330 2020-07-18T03:30:03.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.026+0000 7f741b7fe700 1 --2- 172.21.15.47:0/1075016303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f741c0b59b0 0x7f741c0b5e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f7410002f30 tx=0x7f7410006dd0).ready entity=mon.1 client_cookie=c36b01fcbef6216d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:03.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f741affd700 1 -- 172.21.15.47:0/1075016303 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74100097b0 con 0x7f741c0b59b0 2020-07-18T03:30:03.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f741affd700 1 -- 172.21.15.47:0/1075016303 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7410009910 con 0x7f741c0b59b0 2020-07-18T03:30:03.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f741affd700 1 -- 172.21.15.47:0/1075016303 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7410009c70 con 0x7f741c0b59b0 2020-07-18T03:30:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 -- 172.21.15.47:0/1075016303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f741c0b59b0 msgr2=0x7f741c0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 --2- 172.21.15.47:0/1075016303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f741c0b59b0 0x7f741c0b5e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f7410002f30 tx=0x7f7410006dd0).stop 2020-07-18T03:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 -- 172.21.15.47:0/1075016303 shutdown_connections 2020-07-18T03:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 --2- 172.21.15.47:0/1075016303 >> v2:172.21.15.18:3300/0 conn(0x7f741c0b6300 0x7f741c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 --2- 172.21.15.47:0/1075016303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f741c0b59b0 0x7f741c0b5e10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 -- 172.21.15.47:0/1075016303 >> 172.21.15.47:0/1075016303 conn(0x7f741c05f5a0 msgr2=0x7f741c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 -- 172.21.15.47:0/1075016303 shutdown_connections 2020-07-18T03:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.027+0000 7f7422749700 1 -- 172.21.15.47:0/1075016303 wait complete. 2020-07-18T03:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.028+0000 7f7422749700 1 Processor -- start 2020-07-18T03:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.028+0000 7f7422749700 1 -- start start 2020-07-18T03:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f7422749700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f741c146c00 0x7f741c138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f7422749700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c0b9860 con 0x7f741c0b6300 2020-07-18T03:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f7422749700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c0b9a90 con 0x7f741c0ad330 2020-07-18T03:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f7422749700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c146810 con 0x7f741c146c00 2020-07-18T03:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f741bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f741c0ad330 0x7f741c149920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42948/0 (socket says 172.21.15.47:42948) 2020-07-18T03:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f741bfff700 1 -- 172.21.15.47:0/3086310343 learned_addr learned my addr 172.21.15.47:0/3086310343 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f7420ce6700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f741c146c00 0x7f741c138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323000080 0 0) 0x7f741c0b9a90 con 0x7f741c0ad330 2020-07-18T03:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.029+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f4003240 con 0x7f741c0ad330 2020-07-18T03:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814760045 0 0) 0x7f741c0b9860 con 0x7f741c0b6300 2020-07-18T03:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f741c0b9a90 con 0x7f741c0b6300 2020-07-18T03:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2783542784 0 0) 0x7f73f4003240 con 0x7f741c0ad330 2020-07-18T03:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f741c0b9860 con 0x7f741c0ad330 2020-07-18T03:30:03.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f740c0018e0 con 0x7f741c0ad330 2020-07-18T03:30:03.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f741c146c00 msgr2=0x7f741c138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f741c146c00 0x7f741c138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 >> v1:172.21.15.47:6789/0 conn(0x7f741c0ad330 legacy=0x7f741c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 >> v1:172.21.15.18:6789/0 conn(0x7f741c0b6300 legacy=0x7f741c144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f4003970 0x7f73f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73f40044e0 0x7f73f4004960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f73f4004e50 0x7f73f400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40052d0 con 0x7f73f40044e0 2020-07-18T03:30:03.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f400bc20 con 0x7f73f4003970 2020-07-18T03:30:03.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f400bd60 con 0x7f73f4004e50 2020-07-18T03:30:03.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f7420ce6700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f741c146c00 0x7f741c138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:03.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f4003970 0x7f73f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741b7fe700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73f40044e0 0x7f73f4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f7420ce6700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f73f4004e50 0x7f73f400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 -- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f73f4004e50 msgr2=0x7f73f400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f73f4004e50 0x7f73f400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 -- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73f40044e0 msgr2=0x7f73f4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73f40044e0 0x7f73f4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0b9d30 con 0x7f73f4003970 2020-07-18T03:30:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c0b9240 con 0x7f73f4003970 2020-07-18T03:30:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f7420ce6700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f73f4004e50 0x7f73f400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:30:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c138140 con 0x7f73f4003970 2020-07-18T03:30:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f741bfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f4003970 0x7f73f4003d90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f740c0010c0 tx=0x7f740c007920).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.030+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f740c00aea0 con 0x7f73f4003970 2020-07-18T03:30:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.031+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c007110 con 0x7f73f4003970 2020-07-18T03:30:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.031+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f740c007470 con 0x7f73f4003970 2020-07-18T03:30:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.031+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c0b9ec0 con 0x7f73f4003970 2020-07-18T03:30:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.032+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f740c010030 con 0x7f73f4003970 2020-07-18T03:30:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.032+0000 7f73fbfff700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f73f4058a10 0x7f73f405aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.032+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f740c0681d0 con 0x7f73f4003970 2020-07-18T03:30:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.032+0000 7f741b7fe700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f73f4058a10 0x7f73f405aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.032+0000 7f741b7fe700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f73f4058a10 0x7f73f405aed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f741001b000 tx=0x7f7410015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.035+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f740c040900 con 0x7f73f4003970 2020-07-18T03:30:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.195+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f741c138420 con 0x7f73f4003970 2020-07-18T03:30:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.196+0000 7f73fbfff700 1 -- 172.21.15.47:0/3086310343 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+10 (secure 0 0 0) 0x7f740c043f20 con 0x7f73f4003970 2020-07-18T03:30:03.199 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:30:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f73f4058a10 msgr2=0x7f73f405aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f73f4058a10 0x7f73f405aed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f741001b000 tx=0x7f7410015000).stop 2020-07-18T03:30:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f4003970 msgr2=0x7f73f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 --2- 172.21.15.47:0/3086310343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f4003970 0x7f73f4003d90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f740c0010c0 tx=0x7f740c007920).stop 2020-07-18T03:30:03.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7420ce6700 1 -- 172.21.15.47:0/3086310343 reap_dead start 2020-07-18T03:30:03.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 shutdown_connections 2020-07-18T03:30:03.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 >> 172.21.15.47:0/3086310343 conn(0x7f741c05f5a0 msgr2=0x7f741c14ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:03.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 shutdown_connections 2020-07-18T03:30:03.203 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.197+0000 7f7422749700 1 -- 172.21.15.47:0/3086310343 wait complete. 2020-07-18T03:30:03.210 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:30:03.211 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:03.294 INFO:tasks.ceph.mds.b.smithi018.stderr:2020-07-18T03:30:03.292+0000 7ff632655700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:30:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.327+0000 7f31d7dee700 1 Processor -- start 2020-07-18T03:30:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.327+0000 7f31d7dee700 1 -- start start 2020-07-18T03:30:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d7dee700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31d00ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d7dee700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d00b3ac0 con 0x7f31d00b4620 2020-07-18T03:30:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d7dee700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d00b3d20 con 0x7f31d00b5370 2020-07-18T03:30:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d7dee700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d00b5a10 con 0x7f31d00b41c0 2020-07-18T03:30:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d5b8a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f31d00b5370 0x7f31d005fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42962/0 (socket says 172.21.15.47:42962) 2020-07-18T03:30:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d5b8a700 1 -- 172.21.15.47:0/4268686451 learned_addr learned my addr 172.21.15.47:0/4268686451 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d5389700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31d00ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408095389 0 0) 0x7f31d00b3d20 con 0x7f31d00b5370 2020-07-18T03:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b8003240 con 0x7f31d00b5370 2020-07-18T03:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998732981 0 0) 0x7f31d00b3ac0 con 0x7f31d00b4620 2020-07-18T03:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d00b3d20 con 0x7f31d00b4620 2020-07-18T03:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1250098259 0 0) 0x7f31b8003240 con 0x7f31d00b5370 2020-07-18T03:30:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 msgr2=0x7f31d00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31d00ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 >> v1:172.21.15.18:6789/0 conn(0x7f31d00b4620 legacy=0x7f31d01428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d00b5750 con 0x7f31d00b5370 2020-07-18T03:30:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f31c00016d0 con 0x7f31d00b5370 2020-07-18T03:30:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d5389700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31d00ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.328+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 >> v1:172.21.15.47:6789/0 conn(0x7f31d00b5370 legacy=0x7f31d005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31b80038c0 0x7f31b8003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31b80044e0 0x7f31b8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31b8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d00b5a10 con 0x7f31b80044e0 2020-07-18T03:30:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8005340 con 0x7f31b80038c0 2020-07-18T03:30:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b800b760 con 0x7f31d00b41c0 2020-07-18T03:30:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31b80038c0 0x7f31b8003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5389700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31b80044e0 0x7f31b8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d638b700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31b8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 -- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 msgr2=0x7f31b8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d00b41c0 0x7f31b8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 -- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31b80044e0 msgr2=0x7f31b8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31b80044e0 0x7f31b8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 -- 172.21.15.47:0/4268686451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8003240 con 0x7f31b80038c0 2020-07-18T03:30:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31b80038c0 0x7f31b8003cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f31c00014a0 tx=0x7f31d00b3eb0).ready entity=mon.1 client_cookie=4190a76379795577 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c000ce60 con 0x7f31b80038c0 2020-07-18T03:30:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c00070d0 con 0x7f31b80038c0 2020-07-18T03:30:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d4b88700 1 -- 172.21.15.47:0/4268686451 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c0007430 con 0x7f31b80038c0 2020-07-18T03:30:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d7dee700 1 -- 172.21.15.47:0/4268686451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31b80038c0 msgr2=0x7f31b8003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d7dee700 1 --2- 172.21.15.47:0/4268686451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31b80038c0 0x7f31b8003cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f31c00014a0 tx=0x7f31d00b3eb0).stop 2020-07-18T03:30:03.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d638b700 1 -- 172.21.15.47:0/4268686451 reap_dead start 2020-07-18T03:30:03.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d7dee700 1 -- 172.21.15.47:0/4268686451 shutdown_connections 2020-07-18T03:30:03.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d7dee700 1 -- 172.21.15.47:0/4268686451 >> 172.21.15.47:0/4268686451 conn(0x7f31d00b9340 msgr2=0x7f31d00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:03.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d7dee700 1 -- 172.21.15.47:0/4268686451 shutdown_connections 2020-07-18T03:30:03.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.329+0000 7f31d7dee700 1 -- 172.21.15.47:0/4268686451 wait complete. 2020-07-18T03:30:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 Processor -- start 2020-07-18T03:30:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 -- start start 2020-07-18T03:30:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31d0140380 0x7f31d0065a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d0151970 0x7f31d0065f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f31d00663f0 0x7f31d0154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d0151e60 con 0x7f31d00663f0 2020-07-18T03:30:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d01443f0 con 0x7f31d0140380 2020-07-18T03:30:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d7dee700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0144690 con 0x7f31d0151970 2020-07-18T03:30:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d5b8a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31d0140380 0x7f31d0065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d5b8a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31d0140380 0x7f31d0065a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33654/0 (socket says 172.21.15.47:33654) 2020-07-18T03:30:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.331+0000 7f31d5b8a700 1 -- 172.21.15.47:0/289924527 learned_addr learned my addr 172.21.15.47:0/289924527 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5389700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d0151970 0x7f31d0065f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d638b700 1 --2- 172.21.15.47:0/289924527 >> v2:172.21.15.18:3300/0 conn(0x7f31d00663f0 0x7f31d0154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5b8a700 1 -- 172.21.15.47:0/289924527 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d0151970 msgr2=0x7f31d0065f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d0151970 0x7f31d0065f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.366 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5b8a700 1 -- 172.21.15.47:0/289924527 >> v2:172.21.15.18:3300/0 conn(0x7f31d00663f0 msgr2=0x7f31d0154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/289924527 >> v2:172.21.15.18:3300/0 conn(0x7f31d00663f0 0x7f31d0154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5b8a700 1 -- 172.21.15.47:0/289924527 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0064570 con 0x7f31d0140380 2020-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d638b700 1 --2- 172.21.15.47:0/289924527 >> v2:172.21.15.18:3300/0 conn(0x7f31d00663f0 0x7f31d0154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d5b8a700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d0140380 0x7f31d0065a10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f31c00014a0 tx=0x7f31c0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c000ce60 con 0x7f31d0140380 2020-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0013850 con 0x7f31d0140380 2020-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c0013bb0 con 0x7f31d0140380 2020-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d0061d90 con 0x7f31d0140380 2020-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.332+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d0140920 con 0x7f31d0140380 2020-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.333+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d006a5f0 con 0x7f31d0140380 2020-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.333+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f31c001d070 con 0x7f31d0140380 2020-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.334+0000 7f31bdffb700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f31a8050060 0x7f31a8052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.334+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f31c0068e70 con 0x7f31d0140380 2020-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.334+0000 7f31d5389700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f31a8050060 0x7f31a8052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.334+0000 7f31d5389700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f31a8050060 0x7f31a8052520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f31cc008000 tx=0x7f31cc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.337+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f31c0041600 con 0x7f31d0140380 2020-07-18T03:30:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.547+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31d0057450 con 0x7f31d0140380 2020-07-18T03:30:03.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.547+0000 7f31bdffb700 1 -- 172.21.15.47:0/289924527 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f31c001a030 con 0x7f31d0140380 2020-07-18T03:30:03.550 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:03.550 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f31a8050060 msgr2=0x7f31a8052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f31a8050060 0x7f31a8052520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f31cc008000 tx=0x7f31cc008040).stop 2020-07-18T03:30:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d0140380 msgr2=0x7f31d0065a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d0140380 0x7f31d0065a10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f31c00014a0 tx=0x7f31c0015000).stop 2020-07-18T03:30:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 shutdown_connections 2020-07-18T03:30:03.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f31a8050060 0x7f31a8052520 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 --2- 172.21.15.47:0/289924527 >> v2:172.21.15.18:3300/0 conn(0x7f31d00663f0 0x7f31d0154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d0140380 0x7f31d0065a10 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 --2- 172.21.15.47:0/289924527 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f31d0151970 0x7f31d0065f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 >> 172.21.15.47:0/289924527 conn(0x7f31d00b9340 msgr2=0x7f31d00b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:03.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 shutdown_connections 2020-07-18T03:30:03.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:03.549+0000 7f31d7dee700 1 -- 172.21.15.47:0/289924527 wait complete. 2020-07-18T03:30:03.554 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:03.763 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:57.859680+0000', 'modified': '2020-07-18T03:29:58.861271+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4973': {'gid': 4973, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/3198086169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 3198086169}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 3198086169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:30:03.764 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:30:03.765 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-07-18T03:30:03.917 INFO:teuthology.orchestra.run.smithi018.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4973,"want_state":"up:active","state":"up:active","rank_uptime":6.0460447290000001,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":53,"uptime":11.456587943000001} 2020-07-18T03:30:03.917 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4973, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 53, "rank_uptime": 6.046044729, "state": "up:active", "uptime": 11.456587943, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:30:03.918 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:30:03.918 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:30:03.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:03.921 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:30:03.938 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:30:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:30:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:30:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:30:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:03.940 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:30:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:03.940 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:30:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:30:03.941 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42587sec preferred_lft 42587sec 2020-07-18T03:30:03.943 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:30:03.944 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:03.944 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:30:03.944 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:03.945 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:30:03.945 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:03.945 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:30:03.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:03.946 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:30:04.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.028 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:30:04.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.101 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:30:04.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.174 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:30:04.245 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:30:04.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.247 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:30:04.302 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:30:04.302 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:30:04.303 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:30:04.303 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:30:04.303 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:30:04.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.304 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:30:04.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.377 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:30:04.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.451 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:30:04.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.523 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:30:04.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.578 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:30:04.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.822 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:30:04.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.858 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-07-18T03:30:04.931 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:30:04.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:04.933 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-07-18T03:30:05.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:05.010 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:30:05.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:05.109 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:30:05.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:05.175 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:30:05.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:05.224 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:30:05.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:05.272 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:30:05.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:05.353 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:30:05.454 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:30:05.456 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:30:05.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:05.457 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:30:05.473 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:30:05.544 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:30:05.545 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:30:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:30:05.625 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:30:05.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:05.627 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:30:05.679 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:30:05.680 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:30:05.722 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:30:05.754 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:30:05.755 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:30:05.835 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:30:05.816+0000 7f2a7b0851c0 -1 init, newargv = 0x56132c60fe40 newargc=15 2020-07-18T03:30:05.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[42196]: starting ceph client 2020-07-18T03:30:05.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[42196]: starting fuse 2020-07-18T03:30:05.836 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:30:05.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:30:05.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:05.839 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:30:05.891 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:30:05.892 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:30:05.892 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:30:05.893 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:30:05.894 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:30:05.894 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:30:05.895 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:30:05.895 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:05.895 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:30:05.896 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:30:05.896 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:30:05.897 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:30:05.898 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:05.898 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:30:05.899 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:30:05.899 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:30:05.900 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:30:05.900 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:05.901 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:30:05.901 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:30:05.902 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:30:05.902 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:30:05.903 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:30:05.903 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:30:05.903 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:30:05.904 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:30:05.904 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:05.905 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:30:05.905 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:30:06.028 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.42196.asok 2020-07-18T03:30:06.203 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42196.asok 2020-07-18T03:30:06.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:06.204 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42196.asok status 2020-07-18T03:30:06.355 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:30:06.356 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:30:06.356 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:30:06.356 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:30:06.356 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:30:06.357 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:30:06.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:30:06.357 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "42196", 2020-07-18T03:30:06.357 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:30:06.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:06.358 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:30:06.358 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:30:06.358 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6109, 2020-07-18T03:30:06.358 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:30:06.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:30:06.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:30:06.359 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6109 2020-07-18T03:30:06.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:06.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:30:06.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:30:06.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:30:06.360 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1554713892 2020-07-18T03:30:06.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:30:06.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:06.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:30:06.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:30:06.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:30:06.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1554713892 2020-07-18T03:30:06.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:06.361 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6109 192.168.0.1:0/1554713892", 2020-07-18T03:30:06.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1554713892", 2020-07-18T03:30:06.363 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:30:06.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 23, 2020-07-18T03:30:06.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 54, 2020-07-18T03:30:06.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:30:06.364 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:30:06.364 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:30:06.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:06.367 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:30:06.367 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:30:06.382 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:30:06.383 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:30:06.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:06.384 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:30:06.455 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:30:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.607+0000 7f23726dd700 1 Processor -- start 2020-07-18T03:30:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.607+0000 7f23726dd700 1 -- start start 2020-07-18T03:30:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f23726dd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f236c0ad100 0x7f236c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f23726dd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f236c1443f0 0x7f236c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f23726dd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c147330 con 0x7f236c147a00 2020-07-18T03:30:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f23726dd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0ae530 con 0x7f236c0ad100 2020-07-18T03:30:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f23726dd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0b57e0 con 0x7f236c1443f0 2020-07-18T03:30:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2370c7a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f236c1443f0 0x7f236c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2363fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f236c147a00 0x7f236c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43000/0 (socket says 172.21.15.47:43000) 2020-07-18T03:30:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2363fff700 1 -- 172.21.15.47:0/2758531005 learned_addr learned my addr 172.21.15.47:0/2758531005 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f236bfff700 1 --2- 172.21.15.47:0/2758531005 >> v2:172.21.15.18:3300/0 conn(0x7f236c0ad100 0x7f236c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2370c7a700 1 -- 172.21.15.47:0/2758531005 >> v2:172.21.15.18:3300/0 conn(0x7f236c0ad100 msgr2=0x7f236c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2370c7a700 1 --2- 172.21.15.47:0/2758531005 >> v2:172.21.15.18:3300/0 conn(0x7f236c0ad100 0x7f236c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2370c7a700 1 -- 172.21.15.47:0/2758531005 >> v1:172.21.15.47:6789/0 conn(0x7f236c147a00 legacy=0x7f236c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f236b7fe700 1 -- 172.21.15.47:0/2758531005 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333163015 0 0) 0x7f236c147330 con 0x7f236c147a00 2020-07-18T03:30:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2370c7a700 1 -- 172.21.15.47:0/2758531005 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0b5a80 con 0x7f236c1443f0 2020-07-18T03:30:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f236bfff700 1 --2- 172.21.15.47:0/2758531005 >> v2:172.21.15.18:3300/0 conn(0x7f236c0ad100 0x7f236c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:30:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.608+0000 7f2370c7a700 1 --2- 172.21.15.47:0/2758531005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f236c1443f0 0x7f236c147e40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f23640032b0 tx=0x7f23640091d0).ready entity=mon.1 client_cookie=7d3abdd94222f612 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f236b7fe700 1 -- 172.21.15.47:0/2758531005 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236400ba50 con 0x7f236c1443f0 2020-07-18T03:30:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f236b7fe700 1 -- 172.21.15.47:0/2758531005 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400bbb0 con 0x7f236c1443f0 2020-07-18T03:30:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f236b7fe700 1 -- 172.21.15.47:0/2758531005 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236400bd20 con 0x7f236c1443f0 2020-07-18T03:30:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 -- 172.21.15.47:0/2758531005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f236c1443f0 msgr2=0x7f236c147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 --2- 172.21.15.47:0/2758531005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f236c1443f0 0x7f236c147e40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f23640032b0 tx=0x7f23640091d0).stop 2020-07-18T03:30:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 -- 172.21.15.47:0/2758531005 shutdown_connections 2020-07-18T03:30:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 --2- 172.21.15.47:0/2758531005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f236c1443f0 0x7f236c147e40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 --2- 172.21.15.47:0/2758531005 >> v2:172.21.15.18:3300/0 conn(0x7f236c0ad100 0x7f236c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 -- 172.21.15.47:0/2758531005 >> 172.21.15.47:0/2758531005 conn(0x7f236c0b9340 msgr2=0x7f236c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 -- 172.21.15.47:0/2758531005 shutdown_connections 2020-07-18T03:30:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.609+0000 7f23726dd700 1 -- 172.21.15.47:0/2758531005 wait complete. 2020-07-18T03:30:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.610+0000 7f23726dd700 1 Processor -- start 2020-07-18T03:30:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.610+0000 7f23726dd700 1 -- start start 2020-07-18T03:30:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f23726dd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 0x7f236c141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f23726dd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f236c0b1ac0 0x7f236c1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f23726dd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c1422f0 con 0x7f236c147a00 2020-07-18T03:30:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f23726dd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c159580 con 0x7f236c0b1ac0 2020-07-18T03:30:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f23726dd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c155410 con 0x7f236c0ad100 2020-07-18T03:30:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f236bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 0x7f236c141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f2370c7a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f236c0b1ac0 0x7f236c1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f236bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 0x7f236c141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54514/0 (socket says 172.21.15.47:54514) 2020-07-18T03:30:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.611+0000 7f236bfff700 1 -- 172.21.15.47:0/3831390621 learned_addr learned my addr 172.21.15.47:0/3831390621 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2370c7a700 1 -- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 msgr2=0x7f236c141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2370c7a700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 0x7f236c141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2370c7a700 1 -- 172.21.15.47:0/3831390621 >> v1:172.21.15.18:6789/0 conn(0x7f236c147a00 legacy=0x7f236c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542841751 0 0) 0x7f236c1422f0 con 0x7f236c147a00 2020-07-18T03:30:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2370c7a700 1 -- 172.21.15.47:0/3831390621 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c13fc50 con 0x7f236c0b1ac0 2020-07-18T03:30:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f236bfff700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 0x7f236c141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2370c7a700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f236c0b1ac0 0x7f236c1558b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f23640032b0 tx=0x7f2364016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23640107a0 con 0x7f236c0b1ac0 2020-07-18T03:30:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364010900 con 0x7f236c0b1ac0 2020-07-18T03:30:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c1591d0 con 0x7f236c0b1ac0 2020-07-18T03:30:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2364010c60 con 0x7f236c0b1ac0 2020-07-18T03:30:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.612+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c0b2190 con 0x7f236c0b1ac0 2020-07-18T03:30:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.613+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c155270 con 0x7f236c0b1ac0 2020-07-18T03:30:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.613+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2364027030 con 0x7f236c0b1ac0 2020-07-18T03:30:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.614+0000 7f2368ff9700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f234804fd80 0x7f2348052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.614+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2364068fe0 con 0x7f236c0b1ac0 2020-07-18T03:30:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.614+0000 7f236bfff700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f234804fd80 0x7f2348052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.614+0000 7f236bfff700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f234804fd80 0x7f2348052240 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f235c008000 tx=0x7f235c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.618+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f23640417f0 con 0x7f236c0b1ac0 2020-07-18T03:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.776+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236c155270 con 0x7f236c0b1ac0 2020-07-18T03:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.778+0000 7f2368ff9700 1 -- 172.21.15.47:0/3831390621 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16276 (secure 0 0 0) 0x7f236c155270 con 0x7f236c0b1ac0 2020-07-18T03:30:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:06.780 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:29:59.865840+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:29:55.419365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-07-18T03:29:56.654688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:30:06.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f234804fd80 msgr2=0x7f2348052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f234804fd80 0x7f2348052240 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f235c008000 tx=0x7f235c008040).stop 2020-07-18T03:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f236c0b1ac0 msgr2=0x7f236c1558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f236c0b1ac0 0x7f236c1558b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f23640032b0 tx=0x7f2364016000).stop 2020-07-18T03:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 shutdown_connections 2020-07-18T03:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f234804fd80 0x7f2348052240 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f236c0b1ac0 0x7f236c1558b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 --2- 172.21.15.47:0/3831390621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f236c0ad100 0x7f236c141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 >> 172.21.15.47:0/3831390621 conn(0x7f236c0b9340 msgr2=0x7f236c143530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 shutdown_connections 2020-07-18T03:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.779+0000 7f23726dd700 1 -- 172.21.15.47:0/3831390621 wait complete. 2020-07-18T03:30:06.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:29:55.419365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:29:56.654688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:30:06.792 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:30:06.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.908+0000 7face3143700 1 Processor -- start 2020-07-18T03:30:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.908+0000 7face3143700 1 -- start start 2020-07-18T03:30:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face3143700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7facdc061c60 0x7facdc0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face3143700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7facdc0b3af0 0x7facdc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face3143700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc0af4b0 con 0x7facdc142a20 2020-07-18T03:30:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face3143700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc0b6d80 con 0x7facdc0b3af0 2020-07-18T03:30:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face3143700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0af710 con 0x7facdc061c60 2020-07-18T03:30:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face0edf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7facdc142a20 0x7facdc142e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43010/0 (socket says 172.21.15.47:43010) 2020-07-18T03:30:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face0edf700 1 -- 172.21.15.47:0/3703323005 learned_addr learned my addr 172.21.15.47:0/3703323005 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7face16e0700 1 --2- 172.21.15.47:0/3703323005 >> v2:172.21.15.18:3300/0 conn(0x7facdc0b3af0 0x7facdc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7facdbfff700 1 --2- 172.21.15.47:0/3703323005 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7facdc061c60 0x7facdc0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7facdb7fe700 1 -- 172.21.15.47:0/3703323005 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479284241 0 0) 0x7facdc0af4b0 con 0x7facdc142a20 2020-07-18T03:30:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.909+0000 7facdb7fe700 1 -- 172.21.15.47:0/3703323005 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc8003240 con 0x7facdc142a20 2020-07-18T03:30:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdbfff700 1 -- 172.21.15.47:0/3703323005 >> v2:172.21.15.18:3300/0 conn(0x7facdc0b3af0 msgr2=0x7facdc0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdbfff700 1 --2- 172.21.15.47:0/3703323005 >> v2:172.21.15.18:3300/0 conn(0x7facdc0b3af0 0x7facdc0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdbfff700 1 -- 172.21.15.47:0/3703323005 >> v1:172.21.15.47:6789/0 conn(0x7facdc142a20 legacy=0x7facdc142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdb7fe700 1 -- 172.21.15.47:0/3703323005 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3519894879 0 0) 0x7facc8003240 con 0x7facdc142a20 2020-07-18T03:30:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdbfff700 1 -- 172.21.15.47:0/3703323005 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc0af9b0 con 0x7facdc061c60 2020-07-18T03:30:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face16e0700 1 --2- 172.21.15.47:0/3703323005 >> v2:172.21.15.18:3300/0 conn(0x7facdc0b3af0 0x7facdc0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdbfff700 1 --2- 172.21.15.47:0/3703323005 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7facdc061c60 0x7facdc0620c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7facd0002f30 tx=0x7facd0006dd0).ready entity=mon.2 client_cookie=8bbae1ff05ad2b95 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdb7fe700 1 -- 172.21.15.47:0/3703323005 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd00096a0 con 0x7facdc061c60 2020-07-18T03:30:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdb7fe700 1 -- 172.21.15.47:0/3703323005 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd0009800 con 0x7facdc061c60 2020-07-18T03:30:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7facdb7fe700 1 -- 172.21.15.47:0/3703323005 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd0009b60 con 0x7facdc061c60 2020-07-18T03:30:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 -- 172.21.15.47:0/3703323005 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7facdc061c60 msgr2=0x7facdc0620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 --2- 172.21.15.47:0/3703323005 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7facdc061c60 0x7facdc0620c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7facd0002f30 tx=0x7facd0006dd0).stop 2020-07-18T03:30:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 -- 172.21.15.47:0/3703323005 shutdown_connections 2020-07-18T03:30:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 --2- 172.21.15.47:0/3703323005 >> v2:172.21.15.18:3300/0 conn(0x7facdc0b3af0 0x7facdc0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 --2- 172.21.15.47:0/3703323005 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7facdc061c60 0x7facdc0620c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 -- 172.21.15.47:0/3703323005 >> 172.21.15.47:0/3703323005 conn(0x7facdc05f5a0 msgr2=0x7facdc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 -- 172.21.15.47:0/3703323005 shutdown_connections 2020-07-18T03:30:06.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.910+0000 7face3143700 1 -- 172.21.15.47:0/3703323005 wait complete. 2020-07-18T03:30:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 Processor -- start 2020-07-18T03:30:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 -- start start 2020-07-18T03:30:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7facdc142a20 0x7facdc066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7facdc063850 0x7facdc13ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc063da0 con 0x7facdc0b3af0 2020-07-18T03:30:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc064040 con 0x7facdc063850 2020-07-18T03:30:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.912+0000 7face3143700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc0642e0 con 0x7facdc142a20 2020-07-18T03:30:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facdbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7facdc142a20 0x7facdc066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7face0edf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7facdc0b3af0 0x7facdc13b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43016/0 (socket says 172.21.15.47:43016) 2020-07-18T03:30:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7face0edf700 1 -- 172.21.15.47:0/2782319526 learned_addr learned my addr 172.21.15.47:0/2782319526 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7face16e0700 1 --2- 172.21.15.47:0/2782319526 >> v2:172.21.15.18:3300/0 conn(0x7facdc063850 0x7facdc13ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facdbfff700 1 -- 172.21.15.47:0/2782319526 >> v2:172.21.15.18:3300/0 conn(0x7facdc063850 msgr2=0x7facdc13ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facdbfff700 1 --2- 172.21.15.47:0/2782319526 >> v2:172.21.15.18:3300/0 conn(0x7facdc063850 0x7facdc13ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facdbfff700 1 -- 172.21.15.47:0/2782319526 >> v1:172.21.15.47:6789/0 conn(0x7facdc0b3af0 legacy=0x7facdc13b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279800771 0 0) 0x7facdc063da0 con 0x7facdc0b3af0 2020-07-18T03:30:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facdbfff700 1 -- 172.21.15.47:0/2782319526 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc065620 con 0x7facdc142a20 2020-07-18T03:30:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facdbfff700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc142a20 0x7facdc066a50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7facd0002f30 tx=0x7facd0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd0011e70 con 0x7facdc142a20 2020-07-18T03:30:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc13b000 con 0x7facdc142a20 2020-07-18T03:30:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd001d5d0 con 0x7facdc142a20 2020-07-18T03:30:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd001d930 con 0x7facdc142a20 2020-07-18T03:30:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.913+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc144570 con 0x7facdc142a20 2020-07-18T03:30:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.914+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc064450 con 0x7facdc142a20 2020-07-18T03:30:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.914+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7facd0025030 con 0x7facdc142a20 2020-07-18T03:30:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.915+0000 7facd8ff9700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7facb404fe40 0x7facb4052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.915+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7facd0069280 con 0x7facdc142a20 2020-07-18T03:30:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.915+0000 7face0edf700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7facb404fe40 0x7facb4052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.915+0000 7face0edf700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7facb404fe40 0x7facb4052300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7faccc0011a0 tx=0x7faccc001620).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:06.918+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7facd00453c0 con 0x7facdc142a20 2020-07-18T03:30:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.075+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7facdc05b120 con 0x7facdc142a20 2020-07-18T03:30:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.076+0000 7facd8ff9700 1 -- 172.21.15.47:0/2782319526 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v54) v1 ==== 110+0+10 (secure 0 0 0) 0x7facd0023020 con 0x7facdc142a20 2020-07-18T03:30:07.078 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:30:07.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7facb404fe40 msgr2=0x7facb4052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7facb404fe40 0x7facb4052300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7faccc0011a0 tx=0x7faccc001620).stop 2020-07-18T03:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc142a20 msgr2=0x7facdc066a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc142a20 0x7facdc066a50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7facd0002f30 tx=0x7facd0015000).stop 2020-07-18T03:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 shutdown_connections 2020-07-18T03:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 --2- 172.21.15.47:0/2782319526 >> v2:172.21.15.18:3300/0 conn(0x7facdc063850 0x7facdc13ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7facb404fe40 0x7facb4052300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 --2- 172.21.15.47:0/2782319526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc142a20 0x7facdc066a50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 >> 172.21.15.47:0/2782319526 conn(0x7facdc05f5a0 msgr2=0x7facdc0ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 shutdown_connections 2020-07-18T03:30:07.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.077+0000 7face3143700 1 -- 172.21.15.47:0/2782319526 wait complete. 2020-07-18T03:30:07.089 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:30:07.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.205+0000 7fa2f047d700 1 Processor -- start 2020-07-18T03:30:07.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.205+0000 7fa2f047d700 1 -- start start 2020-07-18T03:30:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2f047d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80b3a10 0x7fa2e80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2f047d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e8147de0 con 0x7fa2e80ad330 2020-07-18T03:30:07.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2f047d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80ae000 con 0x7fa2e80b59b0 2020-07-18T03:30:07.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2f047d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8148010 con 0x7fa2e80b3a10 2020-07-18T03:30:07.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2eda18700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa2e80b59b0 0x7fa2e80b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43024/0 (socket says 172.21.15.47:43024) 2020-07-18T03:30:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2eda18700 1 -- 172.21.15.47:0/197403809 learned_addr learned my addr 172.21.15.47:0/197403809 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2eea1a700 1 --2- 172.21.15.47:0/197403809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80b3a10 0x7fa2e80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190199744 0 0) 0x7fa2e80ae000 con 0x7fa2e80b59b0 2020-07-18T03:30:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d0003240 con 0x7fa2e80b59b0 2020-07-18T03:30:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206718586 0 0) 0x7fa2e8147de0 con 0x7fa2e80ad330 2020-07-18T03:30:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.206+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e80ae000 con 0x7fa2e80ad330 2020-07-18T03:30:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2eea1a700 1 -- 172.21.15.47:0/197403809 >> v1:172.21.15.47:6789/0 conn(0x7fa2e80b59b0 legacy=0x7fa2e80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2eea1a700 1 -- 172.21.15.47:0/197403809 >> v1:172.21.15.18:6789/0 conn(0x7fa2e80ad330 legacy=0x7fa2e8143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2eea1a700 1 -- 172.21.15.47:0/197403809 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80ad710 con 0x7fa2e80b3a10 2020-07-18T03:30:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2eea1a700 1 --2- 172.21.15.47:0/197403809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80b3a10 0x7fa2e80b3e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fa2e00032b0 tx=0x7fa2e00091d0).ready entity=mon.2 client_cookie=fd3f57221de9ad61 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e000b920 con 0x7fa2e80b3a10 2020-07-18T03:30:07.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e000ba80 con 0x7fa2e80b3a10 2020-07-18T03:30:07.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2ed217700 1 -- 172.21.15.47:0/197403809 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e000bbf0 con 0x7fa2e80b3a10 2020-07-18T03:30:07.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 -- 172.21.15.47:0/197403809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80b3a10 msgr2=0x7fa2e80b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 --2- 172.21.15.47:0/197403809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80b3a10 0x7fa2e80b3e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fa2e00032b0 tx=0x7fa2e00091d0).stop 2020-07-18T03:30:07.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 -- 172.21.15.47:0/197403809 shutdown_connections 2020-07-18T03:30:07.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 --2- 172.21.15.47:0/197403809 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80b3a10 0x7fa2e80b3e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 -- 172.21.15.47:0/197403809 >> 172.21.15.47:0/197403809 conn(0x7fa2e805f5a0 msgr2=0x7fa2e805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 -- 172.21.15.47:0/197403809 shutdown_connections 2020-07-18T03:30:07.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.207+0000 7fa2f047d700 1 -- 172.21.15.47:0/197403809 wait complete. 2020-07-18T03:30:07.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.208+0000 7fa2f047d700 1 Processor -- start 2020-07-18T03:30:07.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2f047d700 1 -- start start 2020-07-18T03:30:07.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2f047d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80ad330 0x7fa2e8155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2f047d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2e813bbe0 0x7fa2e813c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2f047d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e813c560 con 0x7fa2e80b59b0 2020-07-18T03:30:07.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2f047d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e80b9000 con 0x7fa2e813bbe0 2020-07-18T03:30:07.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2f047d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80b92a0 con 0x7fa2e80ad330 2020-07-18T03:30:07.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2eda18700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa2e80b59b0 0x7fa2e8061d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43032/0 (socket says 172.21.15.47:43032) 2020-07-18T03:30:07.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2eea1a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2e813bbe0 0x7fa2e813c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2eda18700 1 -- 172.21.15.47:0/2460841409 learned_addr learned my addr 172.21.15.47:0/2460841409 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:07.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.209+0000 7fa2ee219700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80ad330 0x7fa2e8155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2eea1a700 1 -- 172.21.15.47:0/2460841409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80ad330 msgr2=0x7fa2e8155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2eea1a700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80ad330 0x7fa2e8155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2eea1a700 1 -- 172.21.15.47:0/2460841409 >> v1:172.21.15.47:6789/0 conn(0x7fa2e80b59b0 legacy=0x7fa2e8061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393868625 0 0) 0x7fa2e813c560 con 0x7fa2e80b59b0 2020-07-18T03:30:07.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2eea1a700 1 -- 172.21.15.47:0/2460841409 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80b8c80 con 0x7fa2e813bbe0 2020-07-18T03:30:07.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2eea1a700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e813bbe0 0x7fa2e813c070 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fa2e00032b0 tx=0x7fa2e0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e00107a0 con 0x7fa2e813bbe0 2020-07-18T03:30:07.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e0010900 con 0x7fa2e813bbe0 2020-07-18T03:30:07.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e8146bc0 con 0x7fa2e813bbe0 2020-07-18T03:30:07.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e0010c60 con 0x7fa2e813bbe0 2020-07-18T03:30:07.221 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.210+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e8146de0 con 0x7fa2e813bbe0 2020-07-18T03:30:07.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.211+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e80b8e70 con 0x7fa2e813bbe0 2020-07-18T03:30:07.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.211+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa2e001e070 con 0x7fa2e813bbe0 2020-07-18T03:30:07.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.211+0000 7fa2d67fc700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2c004fd70 0x7fa2c0052230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.211+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa2e0068e20 con 0x7fa2e813bbe0 2020-07-18T03:30:07.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.211+0000 7fa2ee219700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2c004fd70 0x7fa2c0052230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.212+0000 7fa2ee219700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2c004fd70 0x7fa2c0052230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa2d800b000 tx=0x7fa2d8009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.215+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa2e0041530 con 0x7fa2e813bbe0 2020-07-18T03:30:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.372+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2e80b9410 con 0x7fa2e813bbe0 2020-07-18T03:30:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.373+0000 7fa2d67fc700 1 -- 172.21.15.47:0/2460841409 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2e0015020 con 0x7fa2e813bbe0 2020-07-18T03:30:07.376 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:30:07.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2c004fd70 msgr2=0x7fa2c0052230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2c004fd70 0x7fa2c0052230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa2d800b000 tx=0x7fa2d8009040).stop 2020-07-18T03:30:07.378 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e813bbe0 msgr2=0x7fa2e813c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e813bbe0 0x7fa2e813c070 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fa2e00032b0 tx=0x7fa2e0016000).stop 2020-07-18T03:30:07.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 shutdown_connections 2020-07-18T03:30:07.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e813bbe0 0x7fa2e813c070 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa2e80ad330 0x7fa2e8155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 --2- 172.21.15.47:0/2460841409 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2c004fd70 0x7fa2c0052230 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 >> 172.21.15.47:0/2460841409 conn(0x7fa2e805f5a0 msgr2=0x7fa2e80b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 shutdown_connections 2020-07-18T03:30:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.374+0000 7fa2f047d700 1 -- 172.21.15.47:0/2460841409 wait complete. 2020-07-18T03:30:07.396 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:30:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.514+0000 7f80381bb700 1 Processor -- start 2020-07-18T03:30:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.514+0000 7f80381bb700 1 -- start start 2020-07-18T03:30:07.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f80381bb700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f803005fce0 0x7f8030066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f80381bb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300670c0 0x7f80301441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f80381bb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803005f5a0 con 0x7f80300600c0 2020-07-18T03:30:07.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f80381bb700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803005f840 con 0x7f803005fce0 2020-07-18T03:30:07.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f80381bb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8030065360 con 0x7f80300670c0 2020-07-18T03:30:07.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f8036758700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f80300600c0 0x7f803014a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43038/0 (socket says 172.21.15.47:43038) 2020-07-18T03:30:07.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f8036758700 1 -- 172.21.15.47:0/447802412 learned_addr learned my addr 172.21.15.47:0/447802412 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:07.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f8035756700 1 --2- 172.21.15.47:0/447802412 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300670c0 0x7f80301441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.515+0000 7f8035f57700 1 --2- 172.21.15.47:0/447802412 >> v2:172.21.15.18:3300/0 conn(0x7f803005fce0 0x7f8030066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8034f55700 1 -- 172.21.15.47:0/447802412 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110501903 0 0) 0x7f803005f5a0 con 0x7f80300600c0 2020-07-18T03:30:07.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8034f55700 1 -- 172.21.15.47:0/447802412 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8018003240 con 0x7f80300600c0 2020-07-18T03:30:07.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8034f55700 1 -- 172.21.15.47:0/447802412 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3653493445 0 0) 0x7f8018003240 con 0x7f80300600c0 2020-07-18T03:30:07.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8035756700 1 -- 172.21.15.47:0/447802412 >> v2:172.21.15.18:3300/0 conn(0x7f803005fce0 msgr2=0x7f8030066bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:07.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8035756700 1 --2- 172.21.15.47:0/447802412 >> v2:172.21.15.18:3300/0 conn(0x7f803005fce0 0x7f8030066bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8035756700 1 -- 172.21.15.47:0/447802412 >> v1:172.21.15.47:6789/0 conn(0x7f80300600c0 legacy=0x7f803014a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8035756700 1 -- 172.21.15.47:0/447802412 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803005c5e0 con 0x7f80300670c0 2020-07-18T03:30:07.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8035f57700 1 --2- 172.21.15.47:0/447802412 >> v2:172.21.15.18:3300/0 conn(0x7f803005fce0 0x7f8030066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:07.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8035756700 1 --2- 172.21.15.47:0/447802412 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300670c0 0x7f80301441b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8028001870 tx=0x7f802c0091d0).ready entity=mon.2 client_cookie=e27bcc5ad15f1c1c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8034f55700 1 -- 172.21.15.47:0/447802412 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80280016d0 con 0x7f80300670c0 2020-07-18T03:30:07.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8034f55700 1 -- 172.21.15.47:0/447802412 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c00b800 con 0x7f80300670c0 2020-07-18T03:30:07.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f8034f55700 1 -- 172.21.15.47:0/447802412 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802c00b970 con 0x7f80300670c0 2020-07-18T03:30:07.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 -- 172.21.15.47:0/447802412 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300670c0 msgr2=0x7f80301441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 --2- 172.21.15.47:0/447802412 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300670c0 0x7f80301441b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8028001870 tx=0x7f802c0091d0).stop 2020-07-18T03:30:07.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 -- 172.21.15.47:0/447802412 shutdown_connections 2020-07-18T03:30:07.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 --2- 172.21.15.47:0/447802412 >> v2:172.21.15.18:3300/0 conn(0x7f803005fce0 0x7f8030066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 --2- 172.21.15.47:0/447802412 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f80300670c0 0x7f80301441b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 -- 172.21.15.47:0/447802412 >> 172.21.15.47:0/447802412 conn(0x7f80300baf50 msgr2=0x7f803005b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 -- 172.21.15.47:0/447802412 shutdown_connections 2020-07-18T03:30:07.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.516+0000 7f80381bb700 1 -- 172.21.15.47:0/447802412 wait complete. 2020-07-18T03:30:07.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 Processor -- start 2020-07-18T03:30:07.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 -- start start 2020-07-18T03:30:07.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f803005fce0 0x7f8030143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8030145170 0x7f8030145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300b3d70 con 0x7f80300600c0 2020-07-18T03:30:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80300b3b60 con 0x7f8030145170 2020-07-18T03:30:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.518+0000 7f80381bb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b54c0 con 0x7f803005fce0 2020-07-18T03:30:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035756700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f80300600c0 0x7f8030143530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43046/0 (socket says 172.21.15.47:43046) 2020-07-18T03:30:07.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035756700 1 -- 172.21.15.47:0/93131421 learned_addr learned my addr 172.21.15.47:0/93131421 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:07.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035f57700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f803005fce0 0x7f8030143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8036758700 1 --2- 172.21.15.47:0/93131421 >> v2:172.21.15.18:3300/0 conn(0x7f8030145170 0x7f8030145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528087582 0 0) 0x7f80300b3d70 con 0x7f80300600c0 2020-07-18T03:30:07.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8008003240 con 0x7f80300600c0 2020-07-18T03:30:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 270543491 0 0) 0x7f8008003240 con 0x7f80300600c0 2020-07-18T03:30:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035f57700 1 -- 172.21.15.47:0/93131421 >> v2:172.21.15.18:3300/0 conn(0x7f8030145170 msgr2=0x7f8030145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035f57700 1 --2- 172.21.15.47:0/93131421 >> v2:172.21.15.18:3300/0 conn(0x7f8030145170 0x7f8030145620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035f57700 1 -- 172.21.15.47:0/93131421 >> v1:172.21.15.47:6789/0 conn(0x7f80300600c0 legacy=0x7f8030143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035f57700 1 -- 172.21.15.47:0/93131421 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300b4010 con 0x7f803005fce0 2020-07-18T03:30:07.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8036758700 1 --2- 172.21.15.47:0/93131421 >> v2:172.21.15.18:3300/0 conn(0x7f8030145170 0x7f8030145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:07.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f8035f57700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f803005fce0 0x7f8030143040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f802c009a90 tx=0x7f8020008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802c0098f0 con 0x7f803005fce0 2020-07-18T03:30:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802000c870 con 0x7f803005fce0 2020-07-18T03:30:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80301462d0 con 0x7f803005fce0 2020-07-18T03:30:07.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.519+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802000c9e0 con 0x7f803005fce0 2020-07-18T03:30:07.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.520+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8030145cd0 con 0x7f803005fce0 2020-07-18T03:30:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.521+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80300b41a0 con 0x7f803005fce0 2020-07-18T03:30:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.521+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f802000cbe0 con 0x7f803005fce0 2020-07-18T03:30:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.521+0000 7f801dffb700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f80080502c0 0x7f8008052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.521+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8020067d90 con 0x7f803005fce0 2020-07-18T03:30:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.521+0000 7f8035756700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f80080502c0 0x7f8008052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.522+0000 7f8035756700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f80080502c0 0x7f8008052780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f802c00b480 tx=0x7f802c016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.525+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f80200404a0 con 0x7f803005fce0 2020-07-18T03:30:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.682+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f803005b120 con 0x7f803005fce0 2020-07-18T03:30:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.683+0000 7f801dffb700 1 -- 172.21.15.47:0/93131421 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+10 (secure 0 0 0) 0x7f80200160a0 con 0x7f803005fce0 2020-07-18T03:30:07.685 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:30:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f80080502c0 msgr2=0x7f8008052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f80080502c0 0x7f8008052780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f802c00b480 tx=0x7f802c016000).stop 2020-07-18T03:30:07.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f803005fce0 msgr2=0x7f8030143040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f803005fce0 0x7f8030143040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f802c009a90 tx=0x7f8020008040).stop 2020-07-18T03:30:07.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 shutdown_connections 2020-07-18T03:30:07.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 --2- 172.21.15.47:0/93131421 >> v2:172.21.15.18:3300/0 conn(0x7f8030145170 0x7f8030145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f803005fce0 0x7f8030143040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 --2- 172.21.15.47:0/93131421 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f80080502c0 0x7f8008052780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 >> 172.21.15.47:0/93131421 conn(0x7f80300baf50 msgr2=0x7f803012f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 shutdown_connections 2020-07-18T03:30:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.684+0000 7f80381bb700 1 -- 172.21.15.47:0/93131421 wait complete. 2020-07-18T03:30:07.696 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:30:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.812+0000 7fb5a2498700 1 Processor -- start 2020-07-18T03:30:07.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.812+0000 7fb5a2498700 1 -- start start 2020-07-18T03:30:07.816 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb59c05bbf0 0x7fb59c0ba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c14a890 con 0x7fb59c09f7d0 2020-07-18T03:30:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c0a0190 con 0x7fb59c05bbf0 2020-07-18T03:30:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c05c5a0 con 0x7fb59c05c0a0 2020-07-18T03:30:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb59c05bbf0 0x7fb59c0ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb59c09f7d0 0x7fb59c1472c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43052/0 (socket says 172.21.15.47:43052) 2020-07-18T03:30:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59bfff700 1 -- 172.21.15.47:0/3505881496 learned_addr learned my addr 172.21.15.47:0/3505881496 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a0a35700 1 --2- 172.21.15.47:0/3505881496 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59b7fe700 1 -- 172.21.15.47:0/3505881496 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 msgr2=0x7fb59c14acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:07.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59b7fe700 1 --2- 172.21.15.47:0/3505881496 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c14acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59b7fe700 1 -- 172.21.15.47:0/3505881496 >> v1:172.21.15.47:6789/0 conn(0x7fb59c09f7d0 legacy=0x7fb59c1472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59b7fe700 1 -- 172.21.15.47:0/3505881496 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0635d0 con 0x7fb59c05bbf0 2020-07-18T03:30:07.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59b7fe700 1 --2- 172.21.15.47:0/3505881496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c05bbf0 0x7fb59c0ba530 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fb58c0010d0 tx=0x7fb590006dd0).ready entity=mon.1 client_cookie=7c4aa998225fd752 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.819 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59affd700 1 -- 172.21.15.47:0/3505881496 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5900096b0 con 0x7fb59c05bbf0 2020-07-18T03:30:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59affd700 1 -- 172.21.15.47:0/3505881496 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590009810 con 0x7fb59c05bbf0 2020-07-18T03:30:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb59affd700 1 -- 172.21.15.47:0/3505881496 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb590009b70 con 0x7fb59c05bbf0 2020-07-18T03:30:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 -- 172.21.15.47:0/3505881496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c05bbf0 msgr2=0x7fb59c0ba530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.813+0000 7fb5a2498700 1 --2- 172.21.15.47:0/3505881496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c05bbf0 0x7fb59c0ba530 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fb58c0010d0 tx=0x7fb590006dd0).stop 2020-07-18T03:30:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.814+0000 7fb5a2498700 1 -- 172.21.15.47:0/3505881496 shutdown_connections 2020-07-18T03:30:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.814+0000 7fb5a2498700 1 --2- 172.21.15.47:0/3505881496 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.814+0000 7fb5a2498700 1 --2- 172.21.15.47:0/3505881496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c05bbf0 0x7fb59c0ba530 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.814+0000 7fb5a2498700 1 -- 172.21.15.47:0/3505881496 >> 172.21.15.47:0/3505881496 conn(0x7fb59c060030 msgr2=0x7fb59c0b9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.814+0000 7fb5a2498700 1 -- 172.21.15.47:0/3505881496 shutdown_connections 2020-07-18T03:30:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.814+0000 7fb5a2498700 1 -- 172.21.15.47:0/3505881496 wait complete. 2020-07-18T03:30:07.822 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.815+0000 7fb5a2498700 1 Processor -- start 2020-07-18T03:30:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.815+0000 7fb5a2498700 1 -- start start 2020-07-18T03:30:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a2498700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a2498700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb59c09f7d0 0x7fb59c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a2498700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c0b9b20 con 0x7fb59c0b38b0 2020-07-18T03:30:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a2498700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c0647d0 con 0x7fb59c09f7d0 2020-07-18T03:30:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a2498700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c066090 con 0x7fb59c05c0a0 2020-07-18T03:30:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a0a35700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb59c0b38b0 0x7fb59c1406d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43062/0 (socket says 172.21.15.47:43062) 2020-07-18T03:30:07.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb5a0a35700 1 -- 172.21.15.47:0/682561609 learned_addr learned my addr 172.21.15.47:0/682561609 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:07.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb59bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb59b7fe700 1 --2- 172.21.15.47:0/682561609 >> v2:172.21.15.18:3300/0 conn(0x7fb59c09f7d0 0x7fb59c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741612935 0 0) 0x7fb59c0b9b20 con 0x7fb59c0b38b0 2020-07-18T03:30:07.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb574003240 con 0x7fb59c0b38b0 2020-07-18T03:30:07.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 234362403 0 0) 0x7fb574003240 con 0x7fb59c0b38b0 2020-07-18T03:30:07.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb59c0b9b20 con 0x7fb59c0b38b0 2020-07-18T03:30:07.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fb594001810 con 0x7fb59c0b38b0 2020-07-18T03:30:07.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 msgr2=0x7fb59c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb59c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 >> v2:172.21.15.18:3300/0 conn(0x7fb59c09f7d0 msgr2=0x7fb59c149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 --2- 172.21.15.47:0/682561609 >> v2:172.21.15.18:3300/0 conn(0x7fb59c09f7d0 0x7fb59c149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 >> v1:172.21.15.47:6789/0 conn(0x7fb59c0b38b0 legacy=0x7fb59c1406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.816+0000 7fb57bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb574003800 0x7fb574003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb5740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5740049d0 0x7fb574004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0647d0 con 0x7fb5740049d0 2020-07-18T03:30:07.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c066090 con 0x7fb574003800 2020-07-18T03:30:07.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb574004240 con 0x7fb59c05c0a0 2020-07-18T03:30:07.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb574003800 0x7fb574003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59b7fe700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb5740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb5a0a35700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5740049d0 0x7fb574004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 -- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 msgr2=0x7fb5740044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb59c05c0a0 0x7fb5740044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 -- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5740049d0 msgr2=0x7fb574004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5740049d0 0x7fb574004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:07.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c066330 con 0x7fb574003800 2020-07-18T03:30:07.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c0b3d60 con 0x7fb574003800 2020-07-18T03:30:07.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c1482d0 con 0x7fb574003800 2020-07-18T03:30:07.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb59bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb574003800 0x7fb574003c20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fb58c00d000 tx=0x7fb58c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.832 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb58c0072c0 con 0x7fb574003800 2020-07-18T03:30:07.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c007420 con 0x7fb574003800 2020-07-18T03:30:07.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.817+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb58c00da10 con 0x7fb574003800 2020-07-18T03:30:07.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.818+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c0664c0 con 0x7fb574003800 2020-07-18T03:30:07.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.818+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb58c00db70 con 0x7fb574003800 2020-07-18T03:30:07.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.819+0000 7fb57bfff700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb574058160 0x7fb57405a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:07.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.819+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb58c067590 con 0x7fb574003800 2020-07-18T03:30:07.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.819+0000 7fb59b7fe700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb574058160 0x7fb57405a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:07.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.819+0000 7fb59b7fe700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb574058160 0x7fb57405a620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb590016000 tx=0x7fb590015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:07.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.822+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb58c03fca0 con 0x7fb574003800 2020-07-18T03:30:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.981+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb59c044870 con 0x7fb574003800 2020-07-18T03:30:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.982+0000 7fb57bfff700 1 -- 172.21.15.47:0/682561609 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb58c0432c0 con 0x7fb574003800 2020-07-18T03:30:07.984 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:30:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb574058160 msgr2=0x7fb57405a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb574058160 0x7fb57405a620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb590016000 tx=0x7fb590015000).stop 2020-07-18T03:30:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb574003800 msgr2=0x7fb574003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 --2- 172.21.15.47:0/682561609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb574003800 0x7fb574003c20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fb58c00d000 tx=0x7fb58c009040).stop 2020-07-18T03:30:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a0a35700 1 -- 172.21.15.47:0/682561609 reap_dead start 2020-07-18T03:30:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 shutdown_connections 2020-07-18T03:30:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 >> 172.21.15.47:0/682561609 conn(0x7fb59c060030 msgr2=0x7fb59c0ba510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 shutdown_connections 2020-07-18T03:30:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:07.983+0000 7fb5a2498700 1 -- 172.21.15.47:0/682561609 wait complete. 2020-07-18T03:30:07.996 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:30:07.996 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:08.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.109+0000 7f06afee6700 1 Processor -- start 2020-07-18T03:30:08.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.109+0000 7f06afee6700 1 -- start start 2020-07-18T03:30:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06afee6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f06a8147a00 0x7f06a8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06afee6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a8147330 con 0x7f06a80ad100 2020-07-18T03:30:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06afee6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a80b57e0 con 0x7f06a81433e0 2020-07-18T03:30:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06afee6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80ae530 con 0x7f06a8147a00 2020-07-18T03:30:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06ae483700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f06a81433e0 0x7f06a8143820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43076/0 (socket says 172.21.15.47:43076) 2020-07-18T03:30:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06ae483700 1 -- 172.21.15.47:0/932767649 learned_addr learned my addr 172.21.15.47:0/932767649 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:08.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417474178 0 0) 0x7f06a80b57e0 con 0x7f06a81433e0 2020-07-18T03:30:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0690003240 con 0x7f06a81433e0 2020-07-18T03:30:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06ad481700 1 --2- 172.21.15.47:0/932767649 >> v2:172.21.15.18:3300/0 conn(0x7f06a8147a00 0x7f06a8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3493741793 0 0) 0x7f0690003240 con 0x7f06a81433e0 2020-07-18T03:30:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 >> v2:172.21.15.18:3300/0 conn(0x7f06a8147a00 msgr2=0x7f06a8147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 --2- 172.21.15.47:0/932767649 >> v2:172.21.15.18:3300/0 conn(0x7f06a8147a00 0x7f06a8147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 >> v1:172.21.15.18:6789/0 conn(0x7f06a80ad100 legacy=0x7f06a80b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80b5a80 con 0x7f06a81433e0 2020-07-18T03:30:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.110+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f06a0001a50 con 0x7f06a81433e0 2020-07-18T03:30:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 >> v1:172.21.15.47:6789/0 conn(0x7f06a81433e0 legacy=0x7f06a8143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0690003730 0x7f0690003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06900044e0 0x7f0690004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0690004e50 0x7f069000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80ae530 con 0x7f06900044e0 2020-07-18T03:30:08.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690004350 con 0x7f0690003730 2020-07-18T03:30:08.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069000bc50 con 0x7f0690004e50 2020-07-18T03:30:08.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0690003730 0x7f0690003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06ad481700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06900044e0 0x7f0690004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06ae483700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0690004e50 0x7f069000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 -- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0690004e50 msgr2=0x7f069000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:08.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0690004e50 0x7f069000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 -- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06900044e0 msgr2=0x7f0690004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06900044e0 0x7f0690004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 -- 172.21.15.47:0/932767649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690003240 con 0x7f0690003730 2020-07-18T03:30:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06ae483700 1 -- 172.21.15.47:0/932767649 reap_dead start 2020-07-18T03:30:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06adc82700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0690003730 0x7f0690003b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f06a805ca30 tx=0x7f0698007ae0).ready entity=mon.1 client_cookie=acaa99f9b61decec server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f069800c600 con 0x7f0690003730 2020-07-18T03:30:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069800c760 con 0x7f0690003730 2020-07-18T03:30:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06acc80700 1 -- 172.21.15.47:0/932767649 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f069800cac0 con 0x7f0690003730 2020-07-18T03:30:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 -- 172.21.15.47:0/932767649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0690003730 msgr2=0x7f0690003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0690003730 0x7f0690003b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f06a805ca30 tx=0x7f0698007ae0).stop 2020-07-18T03:30:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 -- 172.21.15.47:0/932767649 shutdown_connections 2020-07-18T03:30:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 --2- 172.21.15.47:0/932767649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0690003730 0x7f0690003b50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 -- 172.21.15.47:0/932767649 >> 172.21.15.47:0/932767649 conn(0x7f06a80b9340 msgr2=0x7f06a80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 -- 172.21.15.47:0/932767649 shutdown_connections 2020-07-18T03:30:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.111+0000 7f06afee6700 1 -- 172.21.15.47:0/932767649 wait complete. 2020-07-18T03:30:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.113+0000 7f06afee6700 1 Processor -- start 2020-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.113+0000 7f06afee6700 1 -- start start 2020-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.113+0000 7f06afee6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06a80b8920 0x7f06a81528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.113+0000 7f06afee6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f06a8146770 0x7f06a8144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.113+0000 7f06afee6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a8155210 con 0x7f06a80b8540 2020-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06afee6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a81554b0 con 0x7f06a8146770 2020-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06afee6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8146bc0 con 0x7f06a80b8920 2020-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06adc82700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f06a80b8540 0x7f06a8155950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43084/0 (socket says 172.21.15.47:43084) 2020-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06adc82700 1 -- 172.21.15.47:0/810190074 learned_addr learned my addr 172.21.15.47:0/810190074 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ae483700 1 --2- 172.21.15.47:0/810190074 >> v2:172.21.15.18:3300/0 conn(0x7f06a8146770 0x7f06a8144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ad481700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06a80b8920 0x7f06a81528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158382898 0 0) 0x7f06a8155210 con 0x7f06a80b8540 2020-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0680003240 con 0x7f06a80b8540 2020-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2830237899 0 0) 0x7f0680003240 con 0x7f06a80b8540 2020-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ae483700 1 -- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06a80b8920 msgr2=0x7f06a81528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ae483700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06a80b8920 0x7f06a81528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ae483700 1 -- 172.21.15.47:0/810190074 >> v1:172.21.15.47:6789/0 conn(0x7f06a80b8540 legacy=0x7f06a8155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ae483700 1 -- 172.21.15.47:0/810190074 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8156060 con 0x7f06a8146770 2020-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ad481700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06a80b8920 0x7f06a81528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06ae483700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06a8146770 0x7f06a8144f10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f069800c220 tx=0x7f06a80b5d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f069800c600 con 0x7f06a8146770 2020-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a8145400 con 0x7f06a8146770 2020-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0007e90 con 0x7f06a8146770 2020-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.114+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a000fad0 con 0x7f06a8146770 2020-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.115+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a813abc0 con 0x7f06a8146770 2020-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.116+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a8146d00 con 0x7f06a8146770 2020-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.116+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f06a001b020 con 0x7f06a8146770 2020-07-18T03:30:08.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.116+0000 7f0695ffb700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06800502e0 0x7f06800527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.116+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f06a00674c0 con 0x7f06a8146770 2020-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.116+0000 7f06adc82700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06800502e0 0x7f06800527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.116+0000 7f06adc82700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06800502e0 0x7f06800527a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f0698001040 tx=0x7f0698016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:08.143 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.120+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f06a003fb70 con 0x7f06a8146770 2020-07-18T03:30:08.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.327+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a8057450 con 0x7f06a8146770 2020-07-18T03:30:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.327+0000 7f0695ffb700 1 -- 172.21.15.47:0/810190074 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f06a0043190 con 0x7f06a8146770 2020-07-18T03:30:08.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:08.331 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:08.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06800502e0 msgr2=0x7f06800527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06800502e0 0x7f06800527a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f0698001040 tx=0x7f0698016000).stop 2020-07-18T03:30:08.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06a8146770 msgr2=0x7f06a8144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:08.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06a8146770 0x7f06a8144f10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f069800c220 tx=0x7f06a80b5d20).stop 2020-07-18T03:30:08.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 shutdown_connections 2020-07-18T03:30:08.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06a8146770 0x7f06a8144f10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f06800502e0 0x7f06800527a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 --2- 172.21.15.47:0/810190074 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f06a80b8920 0x7f06a81528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 >> 172.21.15.47:0/810190074 conn(0x7f06a80b9340 msgr2=0x7f06a80ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 shutdown_connections 2020-07-18T03:30:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:08.329+0000 7f06afee6700 1 -- 172.21.15.47:0/810190074 wait complete. 2020-07-18T03:30:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:08.544 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:29:57.859680+0000', 'modified': '2020-07-18T03:29:58.861271+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4973': {'gid': 4973, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/3198086169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 3198086169}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 3198086169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:30:08.545 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:30:08.546 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-07-18T03:30:08.693 INFO:teuthology.orchestra.run.smithi018.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":4973,"want_state":"up:active","state":"up:active","rank_uptime":10.822607269000001,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":53,"uptime":16.233153218000002} 2020-07-18T03:30:08.694 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 4973, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 53, "rank_uptime": 10.822607269, "state": "up:active", "uptime": 16.233153218, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:30:08.695 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:30:08.695 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:30:08.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:08.696 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:30:08.716 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:30:08.716 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:30:08.717 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:30:08.717 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:08.717 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:30:08.717 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:08.717 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:30:08.718 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:08.718 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:30:08.718 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:08.719 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:30:08.719 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42582sec preferred_lft 42582sec 2020-07-18T03:30:08.719 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:30:08.719 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:08.719 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:30:08.719 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:08.720 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:30:08.720 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:08.720 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:30:08.720 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 46:05:59:12:3d:c3 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:30:08.720 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:30:08.721 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:08.721 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7489:bfff:fe86:5a54/64 scope link 2020-07-18T03:30:08.721 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:08.721 INFO:teuthology.orchestra.run.smithi047.stdout:17: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:30:08.721 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 46:05:59:12:3d:c3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:30:08.722 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::4405:59ff:fe12:3dc3/64 scope link 2020-07-18T03:30:08.722 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:08.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:08.723 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:30:08.789 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:30:08.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:08.791 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:30:08.849 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:30:08.850 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:30:08.850 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:30:08.851 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:30:08.851 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:30:08.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:08.852 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:30:08.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:08.928 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:30:09.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.024 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:30:09.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.114 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:30:09.184 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-07-18T03:30:09.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.185 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:30:09.258 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:30:09.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.259 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:30:09.328 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:30:09.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.330 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:30:09.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.414 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1' 2020-07-18T03:30:09.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.488 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:30:09.567 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:30:09.568 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:30:09.568 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:30:09.568 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.568 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:30:09.569 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.569 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if17: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:30:09.569 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 2a:43:6a:70:e7:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:30:09.569 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:30:09.569 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.570 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::2843:6aff:fe70:e722/64 scope link 2020-07-18T03:30:09.570 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.581 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:30:09.660 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:30:09.661 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:30:09.661 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:30:09.661 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.662 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:30:09.662 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.662 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if17: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:30:09.662 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 2a:43:6a:70:e7:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:30:09.663 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:30:09.663 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.663 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::2843:6aff:fe70:e722/64 scope link 2020-07-18T03:30:09.663 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:30:09.673 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-07-18T03:30:09.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.674 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1' 2020-07-18T03:30:09.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.748 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:30:09.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.839 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:30:09.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.899 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:30:09.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:09.968 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:30:10.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:10.018 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 up' 2020-07-18T03:30:10.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:10.059 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.1 master ceph-brx' 2020-07-18T03:30:10.158 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:30:10.159 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:30:10.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:10.160 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:30:10.176 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:30:10.248 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:30:10.249 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:30:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:30:10.327 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:30:10.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:10.329 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:30:10.382 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:30:10.383 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:30:10.383 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:30:10.426 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:30:10.462 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:30:10.463 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:30:10.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:30:10.522+0000 7f13fcaa01c0 -1 init, newargv = 0x559e0f2dc550 newargc=15 2020-07-18T03:30:10.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[43013]: starting ceph client 2020-07-18T03:30:10.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[43013]: starting fuse 2020-07-18T03:30:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:30:10.541 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:30:10.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:10.543 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:30:10.596 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:30:10.596 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:30:10.597 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:30:10.598 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:30:10.598 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:30:10.599 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:30:10.600 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:30:10.601 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:30:10.601 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:10.602 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:30:10.603 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:30:10.603 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:30:10.605 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:30:10.606 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:10.607 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:30:10.608 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:30:10.609 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:30:10.610 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:30:10.611 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:10.611 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:30:10.612 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:30:10.612 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:30:10.613 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:30:10.614 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:30:10.615 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:30:10.615 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:30:10.616 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:30:10.617 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:10.618 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:30:10.619 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:30:10.728 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.43013.asok 2020-07-18T03:30:10.909 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43013.asok 2020-07-18T03:30:10.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:10.911 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43013.asok status 2020-07-18T03:30:11.062 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:30:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:30:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:30:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:30:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:30:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:30:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:30:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "43013", 2020-07-18T03:30:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:30:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:30:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:30:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6169, 2020-07-18T03:30:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:30:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:30:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:30:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6169 2020-07-18T03:30:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:30:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:30:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:30:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1846037581 2020-07-18T03:30:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:30:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:30:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:30:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:30:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1846037581 2020-07-18T03:30:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6169 192.168.0.2:0/1846037581", 2020-07-18T03:30:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/1846037581", 2020-07-18T03:30:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:30:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 23, 2020-07-18T03:30:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 54, 2020-07-18T03:30:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:30:11.069 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:30:11.069 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:30:11.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:11.073 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:30:11.074 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:30:11.089 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:30:11.091 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:30:11.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:11.092 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:30:11.165 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2020-07-18T03:30:11.340 INFO:teuthology.orchestra.run.smithi047.stdout:{"mds_reconnect_timeout":"45.000000"} 2020-07-18T03:30:11.341 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2020-07-18T03:30:11.342 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2020-07-18T03:30:11.472 INFO:teuthology.orchestra.run.smithi047.stdout:{"ms_max_backoff":"15.000000"} 2020-07-18T03:30:11.473 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2020-07-18T03:30:11.474 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:11.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.588+0000 7f1a0ebed700 1 Processor -- start 2020-07-18T03:30:11.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.588+0000 7f1a0ebed700 1 -- start start 2020-07-18T03:30:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0ebed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a08147150 0x7f1a081445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0ebed700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1a080641d0 0x7f1a0814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0ebed700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a08060ff0 con 0x7f1a08063d20 2020-07-18T03:30:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0ebed700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a08147530 con 0x7f1a080641d0 2020-07-18T03:30:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0ebed700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a08147740 con 0x7f1a08147150 2020-07-18T03:30:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0c989700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a08147150 0x7f1a081445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0c989700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a08147150 0x7f1a081445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33790/0 (socket says 172.21.15.47:33790) 2020-07-18T03:30:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.589+0000 7f1a0c989700 1 -- 172.21.15.47:0/2704598400 learned_addr learned my addr 172.21.15.47:0/2704598400 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0d18a700 1 --2- 172.21.15.47:0/2704598400 >> v2:172.21.15.18:3300/0 conn(0x7f1a080641d0 0x7f1a0814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0c989700 1 -- 172.21.15.47:0/2704598400 >> v2:172.21.15.18:3300/0 conn(0x7f1a080641d0 msgr2=0x7f1a0814ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0c989700 1 --2- 172.21.15.47:0/2704598400 >> v2:172.21.15.18:3300/0 conn(0x7f1a080641d0 0x7f1a0814ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0c989700 1 -- 172.21.15.47:0/2704598400 >> v1:172.21.15.18:6789/0 conn(0x7f1a08063d20 legacy=0x7f1a0814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0c989700 1 -- 172.21.15.47:0/2704598400 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a081479e0 con 0x7f1a08147150 2020-07-18T03:30:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0d18a700 1 --2- 172.21.15.47:0/2704598400 >> v2:172.21.15.18:3300/0 conn(0x7f1a080641d0 0x7f1a0814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:30:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0c989700 1 --2- 172.21.15.47:0/2704598400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a08147150 0x7f1a081445c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f1a08063a80 tx=0x7f19f8006e90).ready entity=mon.1 client_cookie=40867ff67a5d27e4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a077fe700 1 -- 172.21.15.47:0/2704598400 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19f8009930 con 0x7f1a08147150 2020-07-18T03:30:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a077fe700 1 -- 172.21.15.47:0/2704598400 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f8009a90 con 0x7f1a08147150 2020-07-18T03:30:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a077fe700 1 -- 172.21.15.47:0/2704598400 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19f8009e30 con 0x7f1a08147150 2020-07-18T03:30:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 -- 172.21.15.47:0/2704598400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a08147150 msgr2=0x7f1a081445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 --2- 172.21.15.47:0/2704598400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a08147150 0x7f1a081445c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f1a08063a80 tx=0x7f19f8006e90).stop 2020-07-18T03:30:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 -- 172.21.15.47:0/2704598400 shutdown_connections 2020-07-18T03:30:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 --2- 172.21.15.47:0/2704598400 >> v2:172.21.15.18:3300/0 conn(0x7f1a080641d0 0x7f1a0814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 --2- 172.21.15.47:0/2704598400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a08147150 0x7f1a081445c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 -- 172.21.15.47:0/2704598400 >> 172.21.15.47:0/2704598400 conn(0x7f1a0805f5a0 msgr2=0x7f1a0805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 -- 172.21.15.47:0/2704598400 shutdown_connections 2020-07-18T03:30:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.590+0000 7f1a0ebed700 1 -- 172.21.15.47:0/2704598400 wait complete. 2020-07-18T03:30:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.592+0000 7f1a0ebed700 1 Processor -- start 2020-07-18T03:30:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.592+0000 7f1a0ebed700 1 -- start start 2020-07-18T03:30:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.592+0000 7f1a0ebed700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 0x7f1a080b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.592+0000 7f1a0ebed700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a08158a10 con 0x7f1a08146660 2020-07-18T03:30:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a0ebed700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a08158cb0 con 0x7f1a08063d20 2020-07-18T03:30:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a0ebed700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080b3560 con 0x7f1a080641d0 2020-07-18T03:30:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a0c989700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1a08063d20 0x7f1a0812f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43116/0 (socket says 172.21.15.47:43116) 2020-07-18T03:30:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a0c989700 1 -- 172.21.15.47:0/1946778596 learned_addr learned my addr 172.21.15.47:0/1946778596 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323890732 0 0) 0x7f1a08158cb0 con 0x7f1a08063d20 2020-07-18T03:30:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e0003240 con 0x7f1a08063d20 2020-07-18T03:30:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a07fff700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 0x7f1a080b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 55352364 0 0) 0x7f19e0003240 con 0x7f1a08063d20 2020-07-18T03:30:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a08158cb0 con 0x7f1a08063d20 2020-07-18T03:30:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f19f8009930 con 0x7f1a08063d20 2020-07-18T03:30:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 msgr2=0x7f1a080b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 0x7f1a080b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 >> v1:172.21.15.47:6789/0 conn(0x7f1a08063d20 legacy=0x7f1a0812f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 >> v1:172.21.15.18:6789/0 conn(0x7f1a08146660 legacy=0x7f1a080b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 0x7f19e0003800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19e0003cf0 0x7f19e00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19e00049d0 0x7f19e0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080b3560 con 0x7f19e00049d0 2020-07-18T03:30:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00041a0 con 0x7f19e0003cf0 2020-07-18T03:30:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0004360 con 0x7f1a080641d0 2020-07-18T03:30:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a07fff700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19e0003cf0 0x7f19e00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a0c989700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 0x7f19e0003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.593+0000 7f1a0d18a700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19e00049d0 0x7f19e0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a07fff700 1 -- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 msgr2=0x7f19e0003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a07fff700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1a080641d0 0x7f19e0003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a07fff700 1 -- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19e00049d0 msgr2=0x7f19e0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a07fff700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19e00049d0 0x7f19e0004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a07fff700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080b2520 con 0x7f19e0003cf0 2020-07-18T03:30:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a080b2770 con 0x7f19e0003cf0 2020-07-18T03:30:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a080b30b0 con 0x7f19e0003cf0 2020-07-18T03:30:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a07fff700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19e0003cf0 0x7f19e00044e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f19fc00d000 tx=0x7f19fc007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19fc011590 con 0x7f19e0003cf0 2020-07-18T03:30:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0116f0 con 0x7f19e0003cf0 2020-07-18T03:30:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.594+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19fc011a50 con 0x7f19e0003cf0 2020-07-18T03:30:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.595+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a080b3390 con 0x7f19e0003cf0 2020-07-18T03:30:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.595+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f19fc011bb0 con 0x7f19e0003cf0 2020-07-18T03:30:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.595+0000 7f1a04ff9700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f19e0058300 0x7f19e005a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.595+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f19fc067880 con 0x7f19e0003cf0 2020-07-18T03:30:11.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.595+0000 7f1a0c989700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f19e0058300 0x7f19e005a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:11.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.596+0000 7f1a0c989700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f19e0058300 0x7f19e005a7c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f19f801c000 tx=0x7f19f8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:11.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.599+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f19fc040010 con 0x7f19e0003cf0 2020-07-18T03:30:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.808+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a08057450 con 0x7f19e0003cf0 2020-07-18T03:30:11.810 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.808+0000 7f1a04ff9700 1 -- 172.21.15.47:0/1946778596 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f19fc043630 con 0x7f19e0003cf0 2020-07-18T03:30:11.810 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:11.811 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.809+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f19e0058300 msgr2=0x7f19e005a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.809+0000 7f1a0ebed700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f19e0058300 0x7f19e005a7c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f19f801c000 tx=0x7f19f8015000).stop 2020-07-18T03:30:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19e0003cf0 msgr2=0x7f19e00044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19e0003cf0 0x7f19e00044e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f19fc00d000 tx=0x7f19fc007b50).stop 2020-07-18T03:30:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0d18a700 1 -- 172.21.15.47:0/1946778596 reap_dead start 2020-07-18T03:30:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 shutdown_connections 2020-07-18T03:30:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 --2- 172.21.15.47:0/1946778596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19e0003cf0 0x7f19e00044e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 >> 172.21.15.47:0/1946778596 conn(0x7f1a0805f5a0 msgr2=0x7f1a080b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 shutdown_connections 2020-07-18T03:30:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:11.810+0000 7f1a0ebed700 1 -- 172.21.15.47:0/1946778596 wait complete. 2020-07-18T03:30:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:11.821 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_defer_session_stale false 2020-07-18T03:30:11.964 INFO:teuthology.orchestra.run.smithi018.stdout:{"success":"mds_defer_session_stale = 'false' (not observed, change may require restart) "} 2020-07-18T03:30:11.965 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_defer_session_stale = 'false' (not observed, change may require restart) " } 2020-07-18T03:30:11.965 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.080+0000 7f3b49a71700 1 Processor -- start 2020-07-18T03:30:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.080+0000 7f3b49a71700 1 -- start start 2020-07-18T03:30:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.081+0000 7f3b49a71700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 0x7f3b440aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.081+0000 7f3b49a71700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b440b85a0 con 0x7f3b440b7f00 2020-07-18T03:30:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.081+0000 7f3b49a71700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b440b8840 con 0x7f3b441456c0 2020-07-18T03:30:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.081+0000 7f3b49a71700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44145330 con 0x7f3b44145b70 2020-07-18T03:30:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.081+0000 7f3b42ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3b441456c0 0x7f3b440ae520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43130/0 (socket says 172.21.15.47:43130) 2020-07-18T03:30:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.081+0000 7f3b42ffd700 1 -- 172.21.15.47:0/2126957838 learned_addr learned my addr 172.21.15.47:0/2126957838 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b43fff700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 0x7f3b440aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108942921 0 0) 0x7f3b440b8840 con 0x7f3b441456c0 2020-07-18T03:30:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b30003240 con 0x7f3b441456c0 2020-07-18T03:30:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1173168190 0 0) 0x7f3b30003240 con 0x7f3b441456c0 2020-07-18T03:30:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 msgr2=0x7f3b440aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 0x7f3b440aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 >> v1:172.21.15.18:6789/0 conn(0x7f3b440b7f00 legacy=0x7f3b44063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b440b8ae0 con 0x7f3b441456c0 2020-07-18T03:30:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f3b38001710 con 0x7f3b441456c0 2020-07-18T03:30:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 >> v1:172.21.15.47:6789/0 conn(0x7f3b441456c0 legacy=0x7f3b440ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b30003730 0x7f3b30003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b30004070 0x7f3b30004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 0x7f3b30004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44145330 con 0x7f3b30004070 2020-07-18T03:30:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30005110 con 0x7f3b30003730 2020-07-18T03:30:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300052d0 con 0x7f3b44145b70 2020-07-18T03:30:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b437fe700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b30003730 0x7f3b30003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b42ffd700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b30004070 0x7f3b30004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b43fff700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 0x7f3b30004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b437fe700 1 -- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 msgr2=0x7f3b30004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b437fe700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44145b70 0x7f3b30004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b437fe700 1 -- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b30004070 msgr2=0x7f3b30004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b437fe700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b30004070 0x7f3b30004540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.082+0000 7f3b437fe700 1 -- 172.21.15.47:0/2126957838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30003240 con 0x7f3b30003730 2020-07-18T03:30:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b437fe700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b30003730 0x7f3b30003b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f3b34000fe0 tx=0x7f3b34007b50).ready entity=mon.1 client_cookie=b651fef8a92fe72f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3400c870 con 0x7f3b30003730 2020-07-18T03:30:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3400c9d0 con 0x7f3b30003730 2020-07-18T03:30:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b427fc700 1 -- 172.21.15.47:0/2126957838 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3400cb40 con 0x7f3b30003730 2020-07-18T03:30:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b49a71700 1 -- 172.21.15.47:0/2126957838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b30003730 msgr2=0x7f3b30003b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b49a71700 1 --2- 172.21.15.47:0/2126957838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b30003730 0x7f3b30003b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f3b34000fe0 tx=0x7f3b34007b50).stop 2020-07-18T03:30:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b43fff700 1 -- 172.21.15.47:0/2126957838 reap_dead start 2020-07-18T03:30:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b49a71700 1 -- 172.21.15.47:0/2126957838 shutdown_connections 2020-07-18T03:30:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b49a71700 1 -- 172.21.15.47:0/2126957838 >> 172.21.15.47:0/2126957838 conn(0x7f3b4405f5a0 msgr2=0x7f3b4405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b49a71700 1 -- 172.21.15.47:0/2126957838 shutdown_connections 2020-07-18T03:30:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.083+0000 7f3b49a71700 1 -- 172.21.15.47:0/2126957838 wait complete. 2020-07-18T03:30:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.084+0000 7f3b49a71700 1 Processor -- start 2020-07-18T03:30:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.084+0000 7f3b49a71700 1 -- start start 2020-07-18T03:30:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b49a71700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3b440b6c90 0x7f3b4413fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b49a71700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b440b7070 0x7f3b44141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b49a71700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44142290 0x7f3b4412b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b49a71700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b44142720 con 0x7f3b440b6c90 2020-07-18T03:30:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b49a71700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b44155690 con 0x7f3b440b7070 2020-07-18T03:30:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b49a71700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44141660 con 0x7f3b44142290 2020-07-18T03:30:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b440b7070 0x7f3b44141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b440b7070 0x7f3b44141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33824/0 (socket says 172.21.15.47:33824) 2020-07-18T03:30:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 -- 172.21.15.47:0/4125719961 learned_addr learned my addr 172.21.15.47:0/4125719961 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b437fe700 1 --2- 172.21.15.47:0/4125719961 >> v2:172.21.15.18:3300/0 conn(0x7f3b440b6c90 0x7f3b4413fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b43fff700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44142290 0x7f3b4412b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 -- 172.21.15.47:0/4125719961 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44142290 msgr2=0x7f3b4412b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44142290 0x7f3b4412b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 -- 172.21.15.47:0/4125719961 >> v2:172.21.15.18:3300/0 conn(0x7f3b440b6c90 msgr2=0x7f3b4413fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 --2- 172.21.15.47:0/4125719961 >> v2:172.21.15.18:3300/0 conn(0x7f3b440b6c90 0x7f3b4413fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.085+0000 7f3b42ffd700 1 -- 172.21.15.47:0/4125719961 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44141900 con 0x7f3b440b7070 2020-07-18T03:30:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.086+0000 7f3b42ffd700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b440b7070 0x7f3b44141da0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f3b3800d000 tx=0x7f3b440b8730).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.086+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3800fa90 con 0x7f3b440b7070 2020-07-18T03:30:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.086+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3800fbf0 con 0x7f3b440b7070 2020-07-18T03:30:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.086+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b440b74b0 con 0x7f3b440b7070 2020-07-18T03:30:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.086+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b38012350 con 0x7f3b440b7070 2020-07-18T03:30:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.086+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b44155410 con 0x7f3b440b7070 2020-07-18T03:30:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.087+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b440648b0 con 0x7f3b440b7070 2020-07-18T03:30:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.087+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3b3801b030 con 0x7f3b440b7070 2020-07-18T03:30:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.087+0000 7f3b237fe700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3b1c050090 0x7f3b1c052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.087+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3b380675b0 con 0x7f3b440b7070 2020-07-18T03:30:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.087+0000 7f3b437fe700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3b1c050090 0x7f3b1c052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.088+0000 7f3b437fe700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3b1c050090 0x7f3b1c052550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f3b34019000 tx=0x7f3b34016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.091+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3b3803fd40 con 0x7f3b440b7070 2020-07-18T03:30:12.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.299+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b44141a90 con 0x7f3b440b7070 2020-07-18T03:30:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.299+0000 7f3b237fe700 1 -- 172.21.15.47:0/4125719961 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f3b38019020 con 0x7f3b440b7070 2020-07-18T03:30:12.301 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:12.301 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3b1c050090 msgr2=0x7f3b1c052550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3b1c050090 0x7f3b1c052550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f3b34019000 tx=0x7f3b34016000).stop 2020-07-18T03:30:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b440b7070 msgr2=0x7f3b44141da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b440b7070 0x7f3b44141da0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f3b3800d000 tx=0x7f3b440b8730).stop 2020-07-18T03:30:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 shutdown_connections 2020-07-18T03:30:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3b1c050090 0x7f3b1c052550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3b44142290 0x7f3b4412b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 --2- 172.21.15.47:0/4125719961 >> v2:172.21.15.18:3300/0 conn(0x7f3b440b6c90 0x7f3b4413fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 --2- 172.21.15.47:0/4125719961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b440b7070 0x7f3b44141da0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 >> 172.21.15.47:0/4125719961 conn(0x7f3b4405f5a0 msgr2=0x7f3b4405d320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 shutdown_connections 2020-07-18T03:30:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.300+0000 7f3b49a71700 1 -- 172.21.15.47:0/4125719961 wait complete. 2020-07-18T03:30:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:12.322 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:30:12.323 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:30:12.323 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:30:12.324 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:30:12.324 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:30:12.324 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:12.325 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:30:12.326 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:30:12.326 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:30:12.327 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:30:12.327 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:12.328 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:30:12.328 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:30:12.329 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:30:12.329 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:30:12.334 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:12.334 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:30:12.335 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:30:12.335 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:30:12.336 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:30:12.336 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:30:12.337 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:30:12.337 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:30:12.337 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:30:12.338 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:30:12.338 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:30:12.339 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:30:12.441 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.42196.asok 2020-07-18T03:30:12.622 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42196.asok 2020-07-18T03:30:12.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:30:12.624 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42196.asok mds_sessions 2020-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6109, 2020-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6109 2020-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:30:12.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:30:12.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:30:12.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1554713892 2020-07-18T03:30:12.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:30:12.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:12.779 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6109 v1:192.168.0.1:0/1554713892", 2020-07-18T03:30:12.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "v1:192.168.0.1:0/1554713892", 2020-07-18T03:30:12.780 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions": [ 2020-07-18T03:30:12.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:30:12.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": 0, 2020-07-18T03:30:12.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrs": { 2020-07-18T03:30:12.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:30:12.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:30:12.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:30:12.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6832", 2020-07-18T03:30:12.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3198086169 2020-07-18T03:30:12.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:12.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:30:12.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:30:12.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6833", 2020-07-18T03:30:12.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3198086169 2020-07-18T03:30:12.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:30:12.783 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:30:12.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:30:12.783 INFO:teuthology.orchestra.run.smithi047.stdout: "seq": 0, 2020-07-18T03:30:12.783 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_gen": 0, 2020-07-18T03:30:12.783 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_ttl": "2020-07-18T03:31:06.819575+0000", 2020-07-18T03:30:12.784 INFO:teuthology.orchestra.run.smithi047.stdout: "last_cap_renew_request": "2020-07-18T03:30:06.819575+0000", 2020-07-18T03:30:12.784 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_renew_seq": 2, 2020-07-18T03:30:12.784 INFO:teuthology.orchestra.run.smithi047.stdout: "num_caps": 1, 2020-07-18T03:30:12.784 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "open" 2020-07-18T03:30:12.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:30:12.785 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:30:12.785 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 23 2020-07-18T03:30:12.785 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:30:12.787 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.900+0000 7f09fe5ac700 1 Processor -- start 2020-07-18T03:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.900+0000 7f09fe5ac700 1 -- start start 2020-07-18T03:30:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fe5ac700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fe5ac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09f8147a00 0x7f09f8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fe5ac700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f8147330 con 0x7f09f805f5a0 2020-07-18T03:30:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fe5ac700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f80ae530 con 0x7f09f8147a00 2020-07-18T03:30:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fe5ac700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f80b57e0 con 0x7f09f80ad100 2020-07-18T03:30:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09f8147a00 0x7f09f8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fcb49700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f09f805f5a0 0x7f09f805fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43154/0 (socket says 172.21.15.47:43154) 2020-07-18T03:30:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09fcb49700 1 -- 172.21.15.47:0/3973033329 learned_addr learned my addr 172.21.15.47:0/3973033329 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f7fff700 1 --2- 172.21.15.47:0/3973033329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f77fe700 1 -- 172.21.15.47:0/3973033329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 msgr2=0x7f09f80b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f77fe700 1 --2- 172.21.15.47:0/3973033329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f77fe700 1 -- 172.21.15.47:0/3973033329 >> v1:172.21.15.47:6789/0 conn(0x7f09f805f5a0 legacy=0x7f09f805fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f6ffd700 1 -- 172.21.15.47:0/3973033329 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090681683 0 0) 0x7f09f8147330 con 0x7f09f805f5a0 2020-07-18T03:30:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f77fe700 1 -- 172.21.15.47:0/3973033329 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f80b5a80 con 0x7f09f8147a00 2020-07-18T03:30:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.901+0000 7f09f77fe700 1 --2- 172.21.15.47:0/3973033329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f8147a00 0x7f09f8147e40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f09ec002f30 tx=0x7f09ec006dd0).ready entity=mon.1 client_cookie=6a8e61cda9362372 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09f6ffd700 1 -- 172.21.15.47:0/3973033329 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09ec0097f0 con 0x7f09f8147a00 2020-07-18T03:30:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09f6ffd700 1 -- 172.21.15.47:0/3973033329 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec009950 con 0x7f09f8147a00 2020-07-18T03:30:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09f6ffd700 1 -- 172.21.15.47:0/3973033329 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09ec009cb0 con 0x7f09f8147a00 2020-07-18T03:30:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3973033329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f8147a00 msgr2=0x7f09f8147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3973033329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f8147a00 0x7f09f8147e40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f09ec002f30 tx=0x7f09ec006dd0).stop 2020-07-18T03:30:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3973033329 shutdown_connections 2020-07-18T03:30:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3973033329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3973033329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f8147a00 0x7f09f8147e40 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3973033329 >> 172.21.15.47:0/3973033329 conn(0x7f09f80b9340 msgr2=0x7f09f80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3973033329 shutdown_connections 2020-07-18T03:30:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.902+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3973033329 wait complete. 2020-07-18T03:30:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.903+0000 7f09fe5ac700 1 Processor -- start 2020-07-18T03:30:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.903+0000 7f09fe5ac700 1 -- start start 2020-07-18T03:30:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fe5ac700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f09f805f5a0 0x7f09f8130f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fe5ac700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fe5ac700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f814be80 con 0x7f09f814a3a0 2020-07-18T03:30:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fe5ac700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f814c120 con 0x7f09f805f5a0 2020-07-18T03:30:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fe5ac700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f814ccd0 con 0x7f09f80ad100 2020-07-18T03:30:12.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fcb49700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f09f814a3a0 0x7f09f814c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43160/0 (socket says 172.21.15.47:43160) 2020-07-18T03:30:12.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09fcb49700 1 -- 172.21.15.47:0/3352147460 learned_addr learned my addr 172.21.15.47:0/3352147460 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:12.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09f77fe700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.919 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09f7fff700 1 --2- 172.21.15.47:0/3352147460 >> v2:172.21.15.18:3300/0 conn(0x7f09f805f5a0 0x7f09f8130f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770833230 0 0) 0x7f09f814be80 con 0x7f09f814a3a0 2020-07-18T03:30:12.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.904+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d0003240 con 0x7f09f814a3a0 2020-07-18T03:30:12.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09f7fff700 1 -- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 msgr2=0x7f09f80b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09f7fff700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:12.920 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09f7fff700 1 -- 172.21.15.47:0/3352147460 >> v1:172.21.15.47:6789/0 conn(0x7f09f814a3a0 legacy=0x7f09f814c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:12.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1860227262 0 0) 0x7f09d0003240 con 0x7f09f814a3a0 2020-07-18T03:30:12.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09f7fff700 1 -- 172.21.15.47:0/3352147460 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f814bb70 con 0x7f09f805f5a0 2020-07-18T03:30:12.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09f77fe700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:30:12.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09f7fff700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09f805f5a0 0x7f09f8130f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f09f0001e10 tx=0x7f09e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:12.921 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09f0001c70 con 0x7f09f805f5a0 2020-07-18T03:30:12.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e800f5f0 con 0x7f09f805f5a0 2020-07-18T03:30:12.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f80b30a0 con 0x7f09f805f5a0 2020-07-18T03:30:12.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09e800f950 con 0x7f09f805f5a0 2020-07-18T03:30:12.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.905+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f8141be0 con 0x7f09f805f5a0 2020-07-18T03:30:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.906+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f8057450 con 0x7f09f805f5a0 2020-07-18T03:30:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.906+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f09e800fb00 con 0x7f09f805f5a0 2020-07-18T03:30:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.907+0000 7f09d7fff700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f09d0050310 0x7f09d00527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.907+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f09e80673a0 con 0x7f09f805f5a0 2020-07-18T03:30:12.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.907+0000 7f09f77fe700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f09d0050310 0x7f09d00527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.907+0000 7f09f77fe700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f09d0050310 0x7f09d00527d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f09ec01c000 tx=0x7f09ec015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:12.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:12.911+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f09e803fa50 con 0x7f09f805f5a0 2020-07-18T03:30:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.122+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09f806a100 con 0x7f09f805f5a0 2020-07-18T03:30:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.123+0000 7f09d7fff700 1 -- 172.21.15.47:0/3352147460 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f09e8043070 con 0x7f09f805f5a0 2020-07-18T03:30:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f09d0050310 msgr2=0x7f09d00527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f09d0050310 0x7f09d00527d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f09ec01c000 tx=0x7f09ec015000).stop 2020-07-18T03:30:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09f805f5a0 msgr2=0x7f09f8130f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09f805f5a0 0x7f09f8130f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f09f0001e10 tx=0x7f09e8008040).stop 2020-07-18T03:30:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 shutdown_connections 2020-07-18T03:30:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f09d0050310 0x7f09d00527d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09f805f5a0 0x7f09f8130f60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 --2- 172.21.15.47:0/3352147460 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f09f80ad100 0x7f09f80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 >> 172.21.15.47:0/3352147460 conn(0x7f09f80b9340 msgr2=0x7f09f80ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 shutdown_connections 2020-07-18T03:30:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.125+0000 7f09fe5ac700 1 -- 172.21.15.47:0/3352147460 wait complete. 2020-07-18T03:30:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:13.130 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:13.130 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:13.137 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session config 6109 timeout 120 2020-07-18T03:30:13.286 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2020-07-18T03:30:13.286 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:30:13.287 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:30:13.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:30:13.288 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:30:13.328 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:30:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.491+0000 7fc03c1bd700 1 Processor -- start 2020-07-18T03:30:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.491+0000 7fc03c1bd700 1 -- start start 2020-07-18T03:30:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc03c1bd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc034147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc03c1bd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc034147330 con 0x7fc0341433e0 2020-07-18T03:30:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc03c1bd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0340b57e0 con 0x7fc0340ad100 2020-07-18T03:30:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc03c1bd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340ae530 con 0x7fc034147a00 2020-07-18T03:30:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc039f59700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc0340ad100 0x7fc0340b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43164/0 (socket says 172.21.15.47:43164) 2020-07-18T03:30:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc039f59700 1 -- 172.21.15.47:0/1847552253 learned_addr learned my addr 172.21.15.47:0/1847552253 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:30:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692589486 0 0) 0x7fc0340b57e0 con 0x7fc0340ad100 2020-07-18T03:30:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc01c003240 con 0x7fc0340ad100 2020-07-18T03:30:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc039758700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc034147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3836344051 0 0) 0x7fc01c003240 con 0x7fc0340ad100 2020-07-18T03:30:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 msgr2=0x7fc034147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:13.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc034147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 >> v1:172.21.15.18:6789/0 conn(0x7fc0341433e0 legacy=0x7fc034143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340b5a80 con 0x7fc0340ad100 2020-07-18T03:30:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fc0300016d0 con 0x7fc0340ad100 2020-07-18T03:30:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.492+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 >> v1:172.21.15.47:6789/0 conn(0x7fc0340ad100 legacy=0x7fc0340b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc01c003730 0x7fc01c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc01c004070 0x7fc01c004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc01c004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340ae530 con 0x7fc01c004070 2020-07-18T03:30:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0050e0 con 0x7fc01c003730 2020-07-18T03:30:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0052a0 con 0x7fc034147a00 2020-07-18T03:30:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc01c003730 0x7fc01c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039758700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc01c004070 0x7fc01c004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03a75a700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc01c004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 -- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 msgr2=0x7fc01c004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc01c004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 -- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc01c004070 msgr2=0x7fc01c004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:30:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc01c004070 0x7fc01c004540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 -- 172.21.15.47:0/1847552253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c003240 con 0x7fc01c003730 2020-07-18T03:30:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03a75a700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc034147a00 0x7fc01c004ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:30:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc039f59700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc01c003730 0x7fc01c003b10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fc0300013e0 tx=0x7fc0340b03e0).ready entity=mon.1 client_cookie=296030d52ecbca4f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc03000ce00 con 0x7fc01c003730 2020-07-18T03:30:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300072e0 con 0x7fc01c003730 2020-07-18T03:30:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc038f57700 1 -- 172.21.15.47:0/1847552253 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0300057e0 con 0x7fc01c003730 2020-07-18T03:30:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03c1bd700 1 -- 172.21.15.47:0/1847552253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc01c003730 msgr2=0x7fc01c003b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/1847552253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc01c003730 0x7fc01c003b10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fc0300013e0 tx=0x7fc0340b03e0).stop 2020-07-18T03:30:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03a75a700 1 -- 172.21.15.47:0/1847552253 reap_dead start 2020-07-18T03:30:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03c1bd700 1 -- 172.21.15.47:0/1847552253 shutdown_connections 2020-07-18T03:30:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03c1bd700 1 -- 172.21.15.47:0/1847552253 >> 172.21.15.47:0/1847552253 conn(0x7fc0340b9340 msgr2=0x7fc0340b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03c1bd700 1 -- 172.21.15.47:0/1847552253 shutdown_connections 2020-07-18T03:30:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.493+0000 7fc03c1bd700 1 -- 172.21.15.47:0/1847552253 wait complete. 2020-07-18T03:30:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 Processor -- start 2020-07-18T03:30:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 -- start start 2020-07-18T03:30:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0340b2920 0x7fc0341569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc0340b2d00 0x7fc034141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc03413eeb0 0x7fc03413aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc034156ee0 con 0x7fc0340b2d00 2020-07-18T03:30:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc034141170 con 0x7fc0340b2920 2020-07-18T03:30:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.495+0000 7fc03c1bd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc034141410 con 0x7fc03413eeb0 2020-07-18T03:30:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0340b2920 0x7fc0341569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0340b2920 0x7fc0341569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33856/0 (socket says 172.21.15.47:33856) 2020-07-18T03:30:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 -- 172.21.15.47:0/2385827037 learned_addr learned my addr 172.21.15.47:0/2385827037 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:30:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039758700 1 --2- 172.21.15.47:0/2385827037 >> v2:172.21.15.18:3300/0 conn(0x7fc0340b2d00 0x7fc034141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc03a75a700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc03413eeb0 0x7fc03413aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 -- 172.21.15.47:0/2385827037 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc03413eeb0 msgr2=0x7fc03413aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc03413eeb0 0x7fc03413aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 -- 172.21.15.47:0/2385827037 >> v2:172.21.15.18:3300/0 conn(0x7fc0340b2d00 msgr2=0x7fc034141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 --2- 172.21.15.47:0/2385827037 >> v2:172.21.15.18:3300/0 conn(0x7fc0340b2d00 0x7fc034141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 -- 172.21.15.47:0/2385827037 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0341416b0 con 0x7fc0340b2920 2020-07-18T03:30:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc039f59700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0340b2920 0x7fc0341569f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7fc034140e90 tx=0x7fc030015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc030013850 con 0x7fc0340b2920 2020-07-18T03:30:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300139b0 con 0x7fc0340b2920 2020-07-18T03:30:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc034142040 con 0x7fc0340b2920 2020-07-18T03:30:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc030013d10 con 0x7fc0340b2920 2020-07-18T03:30:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.496+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03413b2d0 con 0x7fc0340b2920 2020-07-18T03:30:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.497+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc034140fb0 con 0x7fc0340b2920 2020-07-18T03:30:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.497+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc030026030 con 0x7fc0340b2920 2020-07-18T03:30:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.498+0000 7fc021ffb700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc00c050000 0x7fc00c0524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:30:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.498+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc030069030 con 0x7fc0340b2920 2020-07-18T03:30:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.498+0000 7fc039758700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc00c050000 0x7fc00c0524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:30:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.498+0000 7fc039758700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc00c050000 0x7fc00c0524c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc024008000 tx=0x7fc024008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:30:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.501+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc0300417c0 con 0x7fc0340b2920 2020-07-18T03:30:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.709+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc034057450 con 0x7fc0340b2920 2020-07-18T03:30:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.709+0000 7fc021ffb700 1 -- 172.21.15.47:0/2385827037 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fc030044de0 con 0x7fc0340b2920 2020-07-18T03:30:13.711 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:30:13.711 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:30:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.710+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc00c050000 msgr2=0x7fc00c0524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.710+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc00c050000 0x7fc00c0524c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc024008000 tx=0x7fc024008040).stop 2020-07-18T03:30:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.710+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0340b2920 msgr2=0x7fc0341569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:30:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.710+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0340b2920 0x7fc0341569f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7fc034140e90 tx=0x7fc030015000).stop 2020-07-18T03:30:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 shutdown_connections 2020-07-18T03:30:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc00c050000 0x7fc00c0524c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc03413eeb0 0x7fc03413aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/2385827037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0340b2920 0x7fc0341569f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 --2- 172.21.15.47:0/2385827037 >> v2:172.21.15.18:3300/0 conn(0x7fc0340b2d00 0x7fc034141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:30:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 >> 172.21.15.47:0/2385827037 conn(0x7fc0340b9340 msgr2=0x7fc0340b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:30:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 shutdown_connections 2020-07-18T03:30:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:30:13.711+0000 7fc03c1bd700 1 -- 172.21.15.47:0/2385827037 wait complete. 2020-07-18T03:30:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:30:13.725 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2020-07-18T03:30:13.875 INFO:teuthology.orchestra.run.smithi018.stdout:[{"id":6169,"entity":{"name":{"type":"client","num":6169},"addr":{"type":"any","addr":"192.168.0.2:0","nonce":1846037581}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":1,"uptime":3.3379608639999998,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99232192191744761,"halflife":300},"inst":"client.6169 192.168.0.2:0/1846037581","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"1","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"43013","root":"/"}},{"id":6109,"entity":{"name":{"type":"client","num":6109},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1554713892}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":1,"uptime":8.0449056710000004,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.98159081033161111,"halflife":300},"inst":"client.6109 192.168.0.1:0/1554713892","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"0","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"42196","root":"/","timeout":"120"}}] 2020-07-18T03:30:13.876 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "1", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "43013", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.2:0", "nonce": 1846037581, "type": "any" }, "name": { "num": 6169, "type": "client" } }, "id": 6169, "inst": "client.6169 192.168.0.2:0/1846037581", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9923219219174476 }, "state": "open", "uptime": 3.337960864, "used_inos": [] }, { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "0", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "42196", "root": "/", "timeout": "120" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1554713892, "type": "any" }, "name": { "num": 6109, "type": "client" } }, "id": 6109, "inst": "client.6109 192.168.0.1:0/1554713892", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9815908103316111 }, "state": "open", "uptime": 8.044905671, "used_inos": [] } ] 2020-07-18T03:30:19.894 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:30:19.898 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:30:49.994 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:30:50.003 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:31:20.076 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:31:20.082 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:31:43.879 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:31:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.008+0000 7fef9a885700 1 Processor -- start 2020-07-18T03:31:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.008+0000 7fef9a885700 1 -- start start 2020-07-18T03:31:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef9a885700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef940b2e10 0x7fef940b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:31:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef9a885700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fef940b1560 0x7fef940b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:31:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef9a885700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef940afbd0 con 0x7fef940b2130 2020-07-18T03:31:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef9a885700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef94144e30 con 0x7fef940b1560 2020-07-18T03:31:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef9a885700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef941491c0 con 0x7fef940b2e10 2020-07-18T03:31:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef93fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef940b2e10 0x7fef940b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:31:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef98e22700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fef940b2130 0x7fef941454f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43264/0 (socket says 172.21.15.47:43264) 2020-07-18T03:31:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef98e22700 1 -- 172.21.15.47:0/2742756197 learned_addr learned my addr 172.21.15.47:0/2742756197 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:31:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef937fe700 1 --2- 172.21.15.47:0/2742756197 >> v2:172.21.15.18:3300/0 conn(0x7fef940b1560 0x7fef940b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:31:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef93fff700 1 -- 172.21.15.47:0/2742756197 >> v2:172.21.15.18:3300/0 conn(0x7fef940b1560 msgr2=0x7fef940b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:31:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef93fff700 1 --2- 172.21.15.47:0/2742756197 >> v2:172.21.15.18:3300/0 conn(0x7fef940b1560 0x7fef940b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef93fff700 1 -- 172.21.15.47:0/2742756197 >> v1:172.21.15.47:6789/0 conn(0x7fef940b2130 legacy=0x7fef941454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:31:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef92ffd700 1 -- 172.21.15.47:0/2742756197 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507447101 0 0) 0x7fef8c001450 con 0x7fef940b2130 2020-07-18T03:31:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef93fff700 1 -- 172.21.15.47:0/2742756197 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94145050 con 0x7fef940b2e10 2020-07-18T03:31:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.009+0000 7fef93fff700 1 --2- 172.21.15.47:0/2742756197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef940b2e10 0x7fef940b31f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fef940b9e90 tx=0x7fef84006dd0).ready entity=mon.1 client_cookie=80ab3d6ad1ef8fcf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:31:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef92ffd700 1 -- 172.21.15.47:0/2742756197 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef84009770 con 0x7fef940b2e10 2020-07-18T03:31:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef92ffd700 1 -- 172.21.15.47:0/2742756197 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef840098d0 con 0x7fef940b2e10 2020-07-18T03:31:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef92ffd700 1 -- 172.21.15.47:0/2742756197 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef84009c30 con 0x7fef940b2e10 2020-07-18T03:31:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 -- 172.21.15.47:0/2742756197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef940b2e10 msgr2=0x7fef940b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:31:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 --2- 172.21.15.47:0/2742756197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef940b2e10 0x7fef940b31f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fef940b9e90 tx=0x7fef84006dd0).stop 2020-07-18T03:31:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 -- 172.21.15.47:0/2742756197 shutdown_connections 2020-07-18T03:31:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 --2- 172.21.15.47:0/2742756197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef940b2e10 0x7fef940b31f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 --2- 172.21.15.47:0/2742756197 >> v2:172.21.15.18:3300/0 conn(0x7fef940b1560 0x7fef940b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 -- 172.21.15.47:0/2742756197 >> 172.21.15.47:0/2742756197 conn(0x7fef940b9340 msgr2=0x7fef940b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:31:44.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 -- 172.21.15.47:0/2742756197 shutdown_connections 2020-07-18T03:31:44.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.010+0000 7fef9a885700 1 -- 172.21.15.47:0/2742756197 wait complete. 2020-07-18T03:31:44.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.011+0000 7fef9a885700 1 Processor -- start 2020-07-18T03:31:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.011+0000 7fef9a885700 1 -- start start 2020-07-18T03:31:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef9a885700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fef940b2130 0x7fef94142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:31:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef9a885700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 0x7fef94132f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:31:44.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef9a885700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef94146390 con 0x7fef940b1560 2020-07-18T03:31:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef9a885700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef941461a0 con 0x7fef940b2130 2020-07-18T03:31:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef9a885700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef940ae100 con 0x7fef94132ab0 2020-07-18T03:31:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef98e22700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 0x7fef94132f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:31:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef937fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fef940b2130 0x7fef94142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:31:44.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef98e22700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 0x7fef94132f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54778/0 (socket says 172.21.15.47:54778) 2020-07-18T03:31:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.012+0000 7fef98e22700 1 -- 172.21.15.47:0/2916571667 learned_addr learned my addr 172.21.15.47:0/2916571667 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:31:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef98e22700 1 -- 172.21.15.47:0/2916571667 >> v2:172.21.15.18:3300/0 conn(0x7fef940b2130 msgr2=0x7fef94142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:31:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef98e22700 1 --2- 172.21.15.47:0/2916571667 >> v2:172.21.15.18:3300/0 conn(0x7fef940b2130 0x7fef94142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef98e22700 1 -- 172.21.15.47:0/2916571667 >> v1:172.21.15.18:6789/0 conn(0x7fef940b1560 legacy=0x7fef94065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:31:44.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef98e22700 1 -- 172.21.15.47:0/2916571667 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940ade60 con 0x7fef94132ab0 2020-07-18T03:31:44.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef98e22700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 0x7fef94132f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fef8c000f00 tx=0x7fef8c007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:31:44.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef8c005b80 con 0x7fef94132ab0 2020-07-18T03:31:44.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c005ce0 con 0x7fef94132ab0 2020-07-18T03:31:44.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef94141b60 con 0x7fef94132ab0 2020-07-18T03:31:44.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef8c007300 con 0x7fef94132ab0 2020-07-18T03:31:44.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.013+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef941467f0 con 0x7fef94132ab0 2020-07-18T03:31:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.014+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef94057450 con 0x7fef94132ab0 2020-07-18T03:31:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.014+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fef8c010030 con 0x7fef94132ab0 2020-07-18T03:31:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.015+0000 7fef73fff700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef6c04ffb0 0x7fef6c052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:31:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.015+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fef8c067b70 con 0x7fef94132ab0 2020-07-18T03:31:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.015+0000 7fef93fff700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef6c04ffb0 0x7fef6c052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:31:44.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.015+0000 7fef93fff700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef6c04ffb0 0x7fef6c052470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fef940b8680 tx=0x7fef84015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:31:44.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.019+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fef8c040300 con 0x7fef94132ab0 2020-07-18T03:31:44.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.226+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef940afbd0 con 0x7fef94132ab0 2020-07-18T03:31:44.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.228+0000 7fef73fff700 1 -- 172.21.15.47:0/2916571667 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fef940afbd0 con 0x7fef94132ab0 2020-07-18T03:31:44.231 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:31:44.231 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:31:44.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef6c04ffb0 msgr2=0x7fef6c052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:31:44.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef6c04ffb0 0x7fef6c052470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fef940b8680 tx=0x7fef84015000).stop 2020-07-18T03:31:44.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 msgr2=0x7fef94132f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:31:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 0x7fef94132f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fef8c000f00 tx=0x7fef8c007b50).stop 2020-07-18T03:31:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 shutdown_connections 2020-07-18T03:31:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fef6c04ffb0 0x7fef6c052470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 --2- 172.21.15.47:0/2916571667 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fef94132ab0 0x7fef94132f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 --2- 172.21.15.47:0/2916571667 >> v2:172.21.15.18:3300/0 conn(0x7fef940b2130 0x7fef94142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:31:44.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 >> 172.21.15.47:0/2916571667 conn(0x7fef940b9340 msgr2=0x7fef940b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:31:44.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 shutdown_connections 2020-07-18T03:31:44.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:31:44.230+0000 7fef9a885700 1 -- 172.21.15.47:0/2916571667 wait complete. 2020-07-18T03:31:44.235 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:31:44.243 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2020-07-18T03:31:44.391 INFO:teuthology.orchestra.run.smithi018.stdout:[{"id":6169,"entity":{"name":{"type":"client","num":6169},"addr":{"type":"any","addr":"192.168.0.2:0","nonce":1846037581}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":93.856905401999995,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.805050011886843,"halflife":300},"inst":"client.6169 192.168.0.2:0/1846037581","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"1","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"43013","root":"/"}},{"id":6109,"entity":{"name":{"type":"client","num":6109},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1554713892}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":98.562850221000005,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.79634592552083006,"halflife":300},"inst":"client.6109 192.168.0.1:0/1554713892","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"0","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"42196","root":"/","timeout":"120"}}] 2020-07-18T03:31:44.393 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "1", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "43013", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.2:0", "nonce": 1846037581, "type": "any" }, "name": { "num": 6169, "type": "client" } }, "id": 6169, "inst": "client.6169 192.168.0.2:0/1846037581", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.805050011886843 }, "state": "open", "uptime": 93.856905402, "used_inos": [] }, { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "0", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "42196", "root": "/", "timeout": "120" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1554713892, "type": "any" }, "name": { "num": 6109, "type": "client" } }, "id": 6109, "inst": "client.6109 192.168.0.1:0/1554713892", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.7963459255208301 }, "state": "open", "uptime": 98.562850221, "used_inos": [] } ] 2020-07-18T03:31:50.169 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:31:50.172 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:32:20.265 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:32:20.270 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:32:44.397 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:32:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.532+0000 7f94ad718700 1 Processor -- start 2020-07-18T03:32:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.532+0000 7f94ad718700 1 -- start start 2020-07-18T03:32:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94ad718700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94a80b2290 0x7f94a81437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94ad718700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f94a80b6530 0x7f94a80b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94ad718700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a8143ca0 con 0x7f94a80b4b30 2020-07-18T03:32:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94ad718700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80b3130 con 0x7f94a80b6530 2020-07-18T03:32:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94ad718700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80b33d0 con 0x7f94a80b2290 2020-07-18T03:32:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94a80b2290 0x7f94a81437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94a80b2290 0x7f94a81437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34042/0 (socket says 172.21.15.47:34042) 2020-07-18T03:32:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 -- 172.21.15.47:0/4291507452 learned_addr learned my addr 172.21.15.47:0/4291507452 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a77fe700 1 --2- 172.21.15.47:0/4291507452 >> v2:172.21.15.18:3300/0 conn(0x7f94a80b6530 0x7f94a80b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 -- 172.21.15.47:0/4291507452 >> v2:172.21.15.18:3300/0 conn(0x7f94a80b6530 msgr2=0x7f94a80b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 --2- 172.21.15.47:0/4291507452 >> v2:172.21.15.18:3300/0 conn(0x7f94a80b6530 0x7f94a80b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 -- 172.21.15.47:0/4291507452 >> v1:172.21.15.18:6789/0 conn(0x7f94a80b4b30 legacy=0x7f94a80b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 -- 172.21.15.47:0/4291507452 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80b2670 con 0x7f94a80b2290 2020-07-18T03:32:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.533+0000 7f94a7fff700 1 --2- 172.21.15.47:0/4291507452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a80b2290 0x7f94a81437b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f9498002f30 tx=0x7f9498006e90).ready entity=mon.1 client_cookie=11da17ecc3498579 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94a6ffd700 1 -- 172.21.15.47:0/4291507452 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9498009850 con 0x7f94a80b2290 2020-07-18T03:32:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94a6ffd700 1 -- 172.21.15.47:0/4291507452 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94980099b0 con 0x7f94a80b2290 2020-07-18T03:32:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94a6ffd700 1 -- 172.21.15.47:0/4291507452 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9498009d10 con 0x7f94a80b2290 2020-07-18T03:32:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 -- 172.21.15.47:0/4291507452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a80b2290 msgr2=0x7f94a81437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 --2- 172.21.15.47:0/4291507452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a80b2290 0x7f94a81437b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f9498002f30 tx=0x7f9498006e90).stop 2020-07-18T03:32:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 -- 172.21.15.47:0/4291507452 shutdown_connections 2020-07-18T03:32:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 --2- 172.21.15.47:0/4291507452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a80b2290 0x7f94a81437b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 --2- 172.21.15.47:0/4291507452 >> v2:172.21.15.18:3300/0 conn(0x7f94a80b6530 0x7f94a80b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 -- 172.21.15.47:0/4291507452 >> 172.21.15.47:0/4291507452 conn(0x7f94a80b9e50 msgr2=0x7f94a80b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 -- 172.21.15.47:0/4291507452 shutdown_connections 2020-07-18T03:32:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.534+0000 7f94ad718700 1 -- 172.21.15.47:0/4291507452 wait complete. 2020-07-18T03:32:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.535+0000 7f94ad718700 1 Processor -- start 2020-07-18T03:32:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.535+0000 7f94ad718700 1 -- start start 2020-07-18T03:32:44.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94ad718700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94a80b4b30 0x7f94a8140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:44.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94ad718700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94a813b9a0 0x7f94a81381b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:44.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94ad718700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a8140f20 con 0x7f94a80b6530 2020-07-18T03:32:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94ad718700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a8141900 con 0x7f94a813b9a0 2020-07-18T03:32:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94ad718700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81411c0 con 0x7f94a80b4b30 2020-07-18T03:32:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94acf17700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94a813b9a0 0x7f94a81381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94acf17700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94a813b9a0 0x7f94a81381b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34048/0 (socket says 172.21.15.47:34048) 2020-07-18T03:32:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94acf17700 1 -- 172.21.15.47:0/1571174042 learned_addr learned my addr 172.21.15.47:0/1571174042 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:44.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.536+0000 7f94a7fff700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94a80b4b30 0x7f94a8140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:44.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94acf17700 1 -- 172.21.15.47:0/1571174042 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94a80b4b30 msgr2=0x7f94a8140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:44.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94acf17700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94a80b4b30 0x7f94a8140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94acf17700 1 -- 172.21.15.47:0/1571174042 >> v1:172.21.15.18:6789/0 conn(0x7f94a80b6530 legacy=0x7f94a80660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94acf17700 1 -- 172.21.15.47:0/1571174042 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8141460 con 0x7f94a813b9a0 2020-07-18T03:32:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94a7fff700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94a80b4b30 0x7f94a8140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:32:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94acf17700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a813b9a0 0x7f94a81381b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f94a81588a0 tx=0x7f94a0007e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94a000a770 con 0x7f94a813b9a0 2020-07-18T03:32:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a000a8d0 con 0x7f94a813b9a0 2020-07-18T03:32:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a8140c20 con 0x7f94a813b9a0 2020-07-18T03:32:44.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94a000ac30 con 0x7f94a813b9a0 2020-07-18T03:32:44.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.537+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a81388f0 con 0x7f94a813b9a0 2020-07-18T03:32:44.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.538+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a81415f0 con 0x7f94a813b9a0 2020-07-18T03:32:44.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.538+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f94a0019030 con 0x7f94a813b9a0 2020-07-18T03:32:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.538+0000 7f9487fff700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9480050010 0x7f94800524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.539+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f94a0067670 con 0x7f94a813b9a0 2020-07-18T03:32:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.539+0000 7f94a7fff700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9480050010 0x7f94800524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.539+0000 7f94a7fff700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9480050010 0x7f94800524d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f949801a000 tx=0x7f9498015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.542+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f94a003fd80 con 0x7f94a813b9a0 2020-07-18T03:32:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.750+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94a8057450 con 0x7f94a813b9a0 2020-07-18T03:32:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.751+0000 7f9487fff700 1 -- 172.21.15.47:0/1571174042 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f94a00433a0 con 0x7f94a813b9a0 2020-07-18T03:32:44.753 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:44.753 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:32:44.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9480050010 msgr2=0x7f94800524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:44.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9480050010 0x7f94800524d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f949801a000 tx=0x7f9498015000).stop 2020-07-18T03:32:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a813b9a0 msgr2=0x7f94a81381b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a813b9a0 0x7f94a81381b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f94a81588a0 tx=0x7f94a0007e90).stop 2020-07-18T03:32:44.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 shutdown_connections 2020-07-18T03:32:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9480050010 0x7f94800524d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a813b9a0 0x7f94a81381b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 --2- 172.21.15.47:0/1571174042 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94a80b4b30 0x7f94a8140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 >> 172.21.15.47:0/1571174042 conn(0x7f94a80b9e50 msgr2=0x7f94a80ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 shutdown_connections 2020-07-18T03:32:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:44.752+0000 7f94ad718700 1 -- 172.21.15.47:0/1571174042 wait complete. 2020-07-18T03:32:44.759 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:32:44.765 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2020-07-18T03:32:44.915 INFO:teuthology.orchestra.run.smithi018.stdout:[{"id":6169,"entity":{"name":{"type":"client","num":6169},"addr":{"type":"any","addr":"192.168.0.2:0","nonce":1846037581}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":154.38019085100001,"requests_in_flight":0,"completed_requests":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":2.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.69998990942495398,"halflife":300},"inst":"client.6169 192.168.0.2:0/1846037581","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"used_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000007fff"},"metric_spec":{"metric_flags":{"feature_bits":"0x"}},"ceph_sha1":"0baf19aeafe8fa5b354a28b0fb65984f752d8b9d","ceph_version":"ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)","entity_id":"1","hostname":"smithi047","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"43013","root":"/"}}] 2020-07-18T03:32:44.916 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "client_metadata": { "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", "client_features": { "feature_bits": "0x0000000000007fff" }, "entity_id": "1", "hostname": "smithi047", "metric_spec": { "metric_flags": { "feature_bits": "0x" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "43013", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "entity": { "addr": { "addr": "192.168.0.2:0", "nonce": 1846037581, "type": "any" }, "name": { "num": 6169, "type": "client" } }, "id": 6169, "inst": "client.6169 192.168.0.2:0/1846037581", "num_caps": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 2.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.699989909424954 }, "state": "open", "uptime": 154.380190851, "used_inos": [] } ] 2020-07-18T03:32:44.916 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:32:44.917 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:32:44.918 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:32:44.918 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:32:44.964 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:32:44.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:44.965 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:32:45.019 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:32:45.020 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:32:45.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.022 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:32:45.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.093 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:32:45.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.199 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:32:45.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.242 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:32:45.296 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:32:45.296 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:45.296 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:32:45.297 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.297 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:32:45.297 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.297 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:45.298 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.298 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:32:45.298 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.298 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:32:45.299 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42426sec preferred_lft 42426sec 2020-07-18T03:32:45.299 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:32:45.299 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.299 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:45.300 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.300 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:45.300 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.300 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:32:45.304 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.304 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:32:45.304 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.304 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7489:bfff:fe86:5a54/64 scope link 2020-07-18T03:32:45.305 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.305 INFO:teuthology.orchestra.run.smithi047.stdout:18: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:32:45.305 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:32:45.306 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::34ba:4dff:fef9:6475/64 scope link 2020-07-18T03:32:45.306 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.307 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:32:45.367 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.368 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:45.368 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.368 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.368 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.369 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.369 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.369 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.369 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.369 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.370 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.370 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.370 INFO:teuthology.orchestra.run.smithi047.stdout:18: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.370 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:32:45.372 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:32:45.373 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:32:45.433 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:32:45.495 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:32:45.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.496 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:32:45.512 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-07-18T03:32:45.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:32:45.513 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:32:45.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.514 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:32:45.566 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:32:45.567 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:45.567 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:32:45.567 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.567 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:32:45.568 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.568 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:45.568 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.568 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:32:45.568 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.568 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:32:45.569 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42425sec preferred_lft 42425sec 2020-07-18T03:32:45.569 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:32:45.569 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.569 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:45.569 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.570 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:45.570 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.570 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:32:45.570 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.570 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:32:45.571 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.571 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7489:bfff:fe86:5a54/64 scope link 2020-07-18T03:32:45.571 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.571 INFO:teuthology.orchestra.run.smithi047.stdout:18: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:32:45.571 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:32:45.572 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::34ba:4dff:fef9:6475/64 scope link 2020-07-18T03:32:45.572 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:45.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.573 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:32:45.638 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.638 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:45.639 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.639 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.639 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.639 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.639 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.640 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.640 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.640 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.640 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.640 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:45.641 INFO:teuthology.orchestra.run.smithi047.stdout:18: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:45.641 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 36:ba:4d:f9:64:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:32:45.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.643 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:32:45.697 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:32:45.698 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:32:45.699 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:32:45.779 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[43013]: fuse finished with error 0 and tester_r 0 2020-07-18T03:32:45.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.781 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:32:45.836 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:32:45.837 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:32:45.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.838 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:32:45.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:45.915 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.1' 2020-07-18T03:32:46.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.047 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:32:46.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.098 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:32:46.162 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:32:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:32:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:32:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.164 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.165 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:32:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:32:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42425sec preferred_lft 42425sec 2020-07-18T03:32:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:32:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.167 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.168 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.168 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state DOWN group default qlen 1000 2020-07-18T03:32:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:32:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7489:bfff:fe86:5a54/64 scope link 2020-07-18T03:32:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.172 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:32:46.232 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:46.233 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.234 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:32:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.234 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.235 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.235 INFO:teuthology.orchestra.run.smithi047.stdout:16: ceph-brx: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:32:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.236 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:32:46.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.237 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:32:46.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.310 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:32:46.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.405 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:32:46.423 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:32:46.423 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:32:46.423 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:32:46.424 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:32:46.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.425 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:32:46.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.506 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:32:46.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.588 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:32:46.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.667 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:32:46.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.721 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:32:46.740 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-07-18T03:32:46.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:32:46.741 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-07-18T03:32:46.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.742 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:32:46.798 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:32:46.798 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:46.798 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:32:46.799 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.799 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:32:46.799 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.799 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.799 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.800 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:32:46.800 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.800 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:32:46.801 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42424sec preferred_lft 42424sec 2020-07-18T03:32:46.801 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:32:46.801 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.801 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.802 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.802 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.802 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.803 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:32:46.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.867 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:32:46.924 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2020-07-18T03:32:46.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:32:46.925 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-07-18T03:32:46.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.926 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:32:46.981 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:32:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:32:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:32:46.982 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:32:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.983 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.983 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.983 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:32:46.984 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.984 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:32:46.984 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42423sec preferred_lft 42423sec 2020-07-18T03:32:46.984 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:32:46.984 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:32:46.985 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.985 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.985 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:32:46.985 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:32:46.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:32:46.986 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:32:47.050 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout' 2020-07-18T03:32:47.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.168+0000 7f8fb9196700 1 Processor -- start 2020-07-18T03:32:47.169 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.168+0000 7f8fb9196700 1 -- start start 2020-07-18T03:32:47.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.168+0000 7f8fb9196700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8fb4147e40 0x7f8fb4143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.168+0000 7f8fb9196700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb4147330 con 0x7f8fb40ad100 2020-07-18T03:32:47.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.168+0000 7f8fb9196700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb40b57e0 con 0x7f8fb4147a00 2020-07-18T03:32:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.168+0000 7f8fb9196700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40ae530 con 0x7f8fb4147e40 2020-07-18T03:32:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8fb4147a00 0x7f8fb41433e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43378/0 (socket says 172.21.15.47:43378) 2020-07-18T03:32:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb259c700 1 -- 172.21.15.47:0/2092220636 learned_addr learned my addr 172.21.15.47:0/2092220636 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb359e700 1 --2- 172.21.15.47:0/2092220636 >> v2:172.21.15.18:3300/0 conn(0x7f8fb4147e40 0x7f8fb4143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906490762 0 0) 0x7f8fb40b57e0 con 0x7f8fb4147a00 2020-07-18T03:32:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa0003240 con 0x7f8fb4147a00 2020-07-18T03:32:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978465288 0 0) 0x7f8fb4147330 con 0x7f8fb40ad100 2020-07-18T03:32:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb40b57e0 con 0x7f8fb40ad100 2020-07-18T03:32:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2422584772 0 0) 0x7f8fa0003240 con 0x7f8fb4147a00 2020-07-18T03:32:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 >> v2:172.21.15.18:3300/0 conn(0x7f8fb4147e40 msgr2=0x7f8fb4143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:47.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 --2- 172.21.15.47:0/2092220636 >> v2:172.21.15.18:3300/0 conn(0x7f8fb4147e40 0x7f8fb4143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:47.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 >> v1:172.21.15.18:6789/0 conn(0x7f8fb40ad100 legacy=0x7f8fb40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:47.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40b5a80 con 0x7f8fb4147a00 2020-07-18T03:32:47.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8fa80016d0 con 0x7f8fb4147a00 2020-07-18T03:32:47.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 >> v1:172.21.15.47:6789/0 conn(0x7f8fb4147a00 legacy=0x7f8fb41433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fa00038c0 0x7f8fa0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fa00044e0 0x7f8fa0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fa0004e50 0x7f8fa000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40ae530 con 0x7f8fa00038c0 2020-07-18T03:32:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa000bc20 con 0x7f8fa00044e0 2020-07-18T03:32:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa000bdb0 con 0x7f8fa0004e50 2020-07-18T03:32:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fa00044e0 0x7f8fa0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.169+0000 7f8fb2d9d700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fa00038c0 0x7f8fa0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb359e700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fa0004e50 0x7f8fa000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb259c700 1 -- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fa0004e50 msgr2=0x7f8fa000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fa0004e50 0x7f8fa000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb259c700 1 -- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fa00038c0 msgr2=0x7f8fa0003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fa00038c0 0x7f8fa0003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb359e700 1 -- 172.21.15.47:0/2092220636 reap_dead start 2020-07-18T03:32:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb259c700 1 -- 172.21.15.47:0/2092220636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0003240 con 0x7f8fa00044e0 2020-07-18T03:32:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fa00044e0 0x7f8fa0004960 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f8fa80014a0 tx=0x7f8fb40b9e60).ready entity=mon.1 client_cookie=1f51f3a87d6009fc server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa800cdc0 con 0x7f8fa00044e0 2020-07-18T03:32:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa80070d0 con 0x7f8fa00044e0 2020-07-18T03:32:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb1d9b700 1 -- 172.21.15.47:0/2092220636 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa8007430 con 0x7f8fa00044e0 2020-07-18T03:32:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 -- 172.21.15.47:0/2092220636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fa00044e0 msgr2=0x7f8fa0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:47.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fa00044e0 0x7f8fa0004960 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f8fa80014a0 tx=0x7f8fb40b9e60).stop 2020-07-18T03:32:47.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 -- 172.21.15.47:0/2092220636 shutdown_connections 2020-07-18T03:32:47.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2092220636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fa00044e0 0x7f8fa0004960 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:47.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 -- 172.21.15.47:0/2092220636 >> 172.21.15.47:0/2092220636 conn(0x7f8fb40b9340 msgr2=0x7f8fb40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:47.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 -- 172.21.15.47:0/2092220636 shutdown_connections 2020-07-18T03:32:47.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.170+0000 7f8fb9196700 1 -- 172.21.15.47:0/2092220636 wait complete. 2020-07-18T03:32:47.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 Processor -- start 2020-07-18T03:32:47.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 -- start start 2020-07-18T03:32:47.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 0x7f8fb4061ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8fb40b2ec0 0x7f8fb4061fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb4152ce0 con 0x7f8fb40624b0 2020-07-18T03:32:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb41413b0 con 0x7f8fb40b2ec0 2020-07-18T03:32:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.172+0000 7f8fb9196700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb41408b0 con 0x7f8fb4151890 2020-07-18T03:32:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 0x7f8fb4061ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 0x7f8fb4061ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54898/0 (socket says 172.21.15.47:54898) 2020-07-18T03:32:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 -- 172.21.15.47:0/2554223125 learned_addr learned my addr 172.21.15.47:0/2554223125 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2554223125 >> v2:172.21.15.18:3300/0 conn(0x7f8fb40b2ec0 0x7f8fb4061fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 -- 172.21.15.47:0/2554223125 >> v2:172.21.15.18:3300/0 conn(0x7f8fb40b2ec0 msgr2=0x7f8fb4061fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 --2- 172.21.15.47:0/2554223125 >> v2:172.21.15.18:3300/0 conn(0x7f8fb40b2ec0 0x7f8fb4061fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 -- 172.21.15.47:0/2554223125 >> v1:172.21.15.18:6789/0 conn(0x7f8fb40624b0 legacy=0x7f8fb41556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 -- 172.21.15.47:0/2554223125 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4152f80 con 0x7f8fb4151890 2020-07-18T03:32:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619316394 0 0) 0x7f8fb4152ce0 con 0x7f8fb40624b0 2020-07-18T03:32:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb2d9d700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 0x7f8fb4061ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f8fa400d000 tx=0x7f8fa4007ae0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa400fa10 con 0x7f8fb4151890 2020-07-18T03:32:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa400fb70 con 0x7f8fb4151890 2020-07-18T03:32:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb40b0530 con 0x7f8fb4151890 2020-07-18T03:32:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa4014450 con 0x7f8fb4151890 2020-07-18T03:32:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.173+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb4151e30 con 0x7f8fb4151890 2020-07-18T03:32:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.174+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb41446e0 con 0x7f8fb4151890 2020-07-18T03:32:47.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.175+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8fa401b070 con 0x7f8fb4151890 2020-07-18T03:32:47.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.175+0000 7f8f9affd700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8f9004fe40 0x7f8f90052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:47.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.175+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f8fa4067d00 con 0x7f8fb4151890 2020-07-18T03:32:47.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.175+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8f9004fe40 0x7f8f90052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:47.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.175+0000 7f8fb259c700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8f9004fe40 0x7f8f90052300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f8fa801b000 tx=0x7f8fa8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:47.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.179+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8fa406b030 con 0x7f8fb4151890 2020-07-18T03:32:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:47.394+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]} v 0) v1 -- 0x7f8fb4143e20 con 0x7f8fb4151890 2020-07-18T03:32:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.326+0000 7f8f9affd700 1 -- 172.21.15.47:0/2554223125 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]}]=0 v288) v1 ==== 159+0+0 (secure 0 0 0) 0x7f8fa4019020 con 0x7f8fb4151890 2020-07-18T03:32:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8f9004fe40 msgr2=0x7f8f90052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8f9004fe40 0x7f8f90052300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f8fa801b000 tx=0x7f8fa8015000).stop 2020-07-18T03:32:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 msgr2=0x7f8fb4061ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 0x7f8fb4061ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f8fa400d000 tx=0x7f8fa4007ae0).stop 2020-07-18T03:32:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 shutdown_connections 2020-07-18T03:32:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8f9004fe40 0x7f8f90052300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2554223125 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8fb4151890 0x7f8fb4061ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 --2- 172.21.15.47:0/2554223125 >> v2:172.21.15.18:3300/0 conn(0x7f8fb40b2ec0 0x7f8fb4061fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 >> 172.21.15.47:0/2554223125 conn(0x7f8fb40b9340 msgr2=0x7f8fb40ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 shutdown_connections 2020-07-18T03:32:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.329+0000 7f8fb9196700 1 -- 172.21.15.47:0/2554223125 wait complete. 2020-07-18T03:32:48.345 INFO:tasks.cephfs_test_runner:test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2020-07-18T03:32:48.346 INFO:tasks.cephfs_test_runner:Starting test: test_create_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) 2020-07-18T03:32:48.347 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync' 2020-07-18T03:32:48.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.472+0000 7fc52f0c0700 1 Processor -- start 2020-07-18T03:32:48.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.472+0000 7fc52f0c0700 1 -- start start 2020-07-18T03:32:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.472+0000 7fc52f0c0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5280ad9e0 0x7fc5281428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.472+0000 7fc52f0c0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280b3ac0 con 0x7fc5280b41c0 2020-07-18T03:32:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.472+0000 7fc52f0c0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280b3d20 con 0x7fc5280b5370 2020-07-18T03:32:48.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.472+0000 7fc52f0c0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280b5a10 con 0x7fc5280ad9e0 2020-07-18T03:32:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52ce5c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc5280b5370 0x7fc52805fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43396/0 (socket says 172.21.15.47:43396) 2020-07-18T03:32:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52d65d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5280ad9e0 0x7fc5281428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52ce5c700 1 -- 172.21.15.47:0/2150199397 learned_addr learned my addr 172.21.15.47:0/2150199397 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52d65d700 1 -- 172.21.15.47:0/2150199397 >> v1:172.21.15.47:6789/0 conn(0x7fc5280b5370 legacy=0x7fc52805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52d65d700 1 -- 172.21.15.47:0/2150199397 >> v1:172.21.15.18:6789/0 conn(0x7fc5280b41c0 legacy=0x7fc5280ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52d65d700 1 -- 172.21.15.47:0/2150199397 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280b5750 con 0x7fc5280ad9e0 2020-07-18T03:32:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52d65d700 1 --2- 172.21.15.47:0/2150199397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280ad9e0 0x7fc5281428f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fc5280afb30 tx=0x7fc5200091d0).ready entity=mon.1 client_cookie=98b7b07da46be69e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc5277fe700 1 -- 172.21.15.47:0/2150199397 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc520009e80 con 0x7fc5280ad9e0 2020-07-18T03:32:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc5277fe700 1 -- 172.21.15.47:0/2150199397 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52000bbc0 con 0x7fc5280ad9e0 2020-07-18T03:32:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc5277fe700 1 -- 172.21.15.47:0/2150199397 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52000bd30 con 0x7fc5280ad9e0 2020-07-18T03:32:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 -- 172.21.15.47:0/2150199397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280ad9e0 msgr2=0x7fc5281428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/2150199397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280ad9e0 0x7fc5281428f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fc5280afb30 tx=0x7fc5200091d0).stop 2020-07-18T03:32:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 -- 172.21.15.47:0/2150199397 shutdown_connections 2020-07-18T03:32:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/2150199397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280ad9e0 0x7fc5281428f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:48.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 -- 172.21.15.47:0/2150199397 >> 172.21.15.47:0/2150199397 conn(0x7fc5280b9340 msgr2=0x7fc5280b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 -- 172.21.15.47:0/2150199397 shutdown_connections 2020-07-18T03:32:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.473+0000 7fc52f0c0700 1 -- 172.21.15.47:0/2150199397 wait complete. 2020-07-18T03:32:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.475+0000 7fc52f0c0700 1 Processor -- start 2020-07-18T03:32:48.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.475+0000 7fc52f0c0700 1 -- start start 2020-07-18T03:32:48.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52f0c0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5280b41c0 0x7fc528063220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:48.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52f0c0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5280b5370 0x7fc5280640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:48.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52f0c0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc528063710 con 0x7fc528061ea0 2020-07-18T03:32:48.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52f0c0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc528062dc0 con 0x7fc5280b41c0 2020-07-18T03:32:48.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52f0c0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280645d0 con 0x7fc5280b5370 2020-07-18T03:32:48.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5280b41c0 0x7fc528063220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:48.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5280b41c0 0x7fc528063220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34082/0 (socket says 172.21.15.47:34082) 2020-07-18T03:32:48.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 -- 172.21.15.47:0/1283546566 learned_addr learned my addr 172.21.15.47:0/1283546566 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc527fff700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5280b5370 0x7fc5280640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 -- 172.21.15.47:0/1283546566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5280b5370 msgr2=0x7fc5280640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5280b5370 0x7fc5280640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:48.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 -- 172.21.15.47:0/1283546566 >> v1:172.21.15.18:6789/0 conn(0x7fc528061ea0 legacy=0x7fc528065be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:48.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 -- 172.21.15.47:0/1283546566 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc528066520 con 0x7fc5280b41c0 2020-07-18T03:32:48.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc52ce5c700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280b41c0 0x7fc528063220 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fc528060880 tx=0x7fc510007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:48.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5100072e0 con 0x7fc5280b41c0 2020-07-18T03:32:48.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.476+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510007440 con 0x7fc5280b41c0 2020-07-18T03:32:48.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.477+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5100057e0 con 0x7fc5280b41c0 2020-07-18T03:32:48.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.477+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5280667c0 con 0x7fc5280b41c0 2020-07-18T03:32:48.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.477+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc528135de0 con 0x7fc5280b41c0 2020-07-18T03:32:48.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.477+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528057450 con 0x7fc5280b41c0 2020-07-18T03:32:48.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.478+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc51001a060 con 0x7fc5280b41c0 2020-07-18T03:32:48.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.478+0000 7fc524ff9700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc504050040 0x7fc504052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:48.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.478+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fc510067060 con 0x7fc5280b41c0 2020-07-18T03:32:48.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.478+0000 7fc527fff700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc504050040 0x7fc504052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:48.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.478+0000 7fc527fff700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc504050040 0x7fc504052500 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fc51800d000 tx=0x7fc518007ae0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:48.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.481+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc51003f7d0 con 0x7fc5280b41c0 2020-07-18T03:32:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:48.694+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]} v 0) v1 -- 0x7fc5280b01a0 con 0x7fc5280b41c0 2020-07-18T03:32:49.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.328+0000 7fc524ff9700 1 -- 172.21.15.47:0/1283546566 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]}]=0 v289) v1 ==== 152+0+0 (secure 0 0 0) 0x7fc510018020 con 0x7fc5280b41c0 2020-07-18T03:32:49.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc504050040 msgr2=0x7fc504052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc504050040 0x7fc504052500 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fc51800d000 tx=0x7fc518007ae0).stop 2020-07-18T03:32:49.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280b41c0 msgr2=0x7fc528063220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280b41c0 0x7fc528063220 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fc528060880 tx=0x7fc510007be0).stop 2020-07-18T03:32:49.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 shutdown_connections 2020-07-18T03:32:49.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc504050040 0x7fc504052500 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5280b41c0 0x7fc528063220 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 --2- 172.21.15.47:0/1283546566 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5280b5370 0x7fc5280640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 >> 172.21.15.47:0/1283546566 conn(0x7fc5280b9340 msgr2=0x7fc52814bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:49.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 shutdown_connections 2020-07-18T03:32:49.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.330+0000 7fc52f0c0700 1 -- 172.21.15.47:0/1283546566 wait complete. 2020-07-18T03:32:49.344 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:32:49.345 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:32:49.401 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:32:49.455 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:32:49.456 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-07-18T03:32:49.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 Processor -- start 2020-07-18T03:32:49.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 -- start start 2020-07-18T03:32:49.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:49.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 0x7f2f7c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:49.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c147330 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c0ae530 con 0x7f2f7c0ad100 2020-07-18T03:32:49.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.576+0000 7f2f82f3a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b57e0 con 0x7f2f7c147a00 2020-07-18T03:32:49.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 0x7f2f7c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:49.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f80cd6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:49.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 0x7f2f7c147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:54920/0 (socket says 172.21.15.47:54920) 2020-07-18T03:32:49.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 -- 172.21.15.47:0/1679524444 learned_addr learned my addr 172.21.15.47:0/1679524444 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:49.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 -- 172.21.15.47:0/1679524444 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 msgr2=0x7f2f7c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:49.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 --2- 172.21.15.47:0/1679524444 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 -- 172.21.15.47:0/1679524444 >> v1:172.21.15.18:6789/0 conn(0x7f2f7c05f5a0 legacy=0x7f2f7c05fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 -- 172.21.15.47:0/1679524444 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0b5a80 con 0x7f2f7c147a00 2020-07-18T03:32:49.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f80cd6700 1 --2- 172.21.15.47:0/1679524444 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:32:49.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.577+0000 7f2f7bfff700 1 --2- 172.21.15.47:0/1679524444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 0x7f2f7c147e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2f70002f30 tx=0x7f2f70006e90).ready entity=mon.2 client_cookie=ca8550cfc2e6eab4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:49.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f7b7fe700 1 -- 172.21.15.47:0/1679524444 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f70009850 con 0x7f2f7c147a00 2020-07-18T03:32:49.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f7b7fe700 1 -- 172.21.15.47:0/1679524444 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700099b0 con 0x7f2f7c147a00 2020-07-18T03:32:49.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f7b7fe700 1 -- 172.21.15.47:0/1679524444 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f70009d10 con 0x7f2f7c147a00 2020-07-18T03:32:49.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 -- 172.21.15.47:0/1679524444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 msgr2=0x7f2f7c147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/1679524444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 0x7f2f7c147e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2f70002f30 tx=0x7f2f70006e90).stop 2020-07-18T03:32:49.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 -- 172.21.15.47:0/1679524444 shutdown_connections 2020-07-18T03:32:49.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/1679524444 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/1679524444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2f7c147a00 0x7f2f7c147e40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 -- 172.21.15.47:0/1679524444 >> 172.21.15.47:0/1679524444 conn(0x7f2f7c0b9340 msgr2=0x7f2f7c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:49.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 -- 172.21.15.47:0/1679524444 shutdown_connections 2020-07-18T03:32:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.578+0000 7f2f82f3a700 1 -- 172.21.15.47:0/1679524444 wait complete. 2020-07-18T03:32:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.579+0000 7f2f82f3a700 1 Processor -- start 2020-07-18T03:32:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.579+0000 7f2f82f3a700 1 -- start start 2020-07-18T03:32:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f82f3a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2f7c05f5a0 0x7f2f7c145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f82f3a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c13cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f82f3a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c140260 con 0x7f2f7c13fd10 2020-07-18T03:32:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f82f3a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b18b0 con 0x7f2f7c0ad100 2020-07-18T03:32:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f82f3a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c13e0f0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f80cd6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2f7c05f5a0 0x7f2f7c145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f80cd6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2f7c05f5a0 0x7f2f7c145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34094/0 (socket says 172.21.15.47:34094) 2020-07-18T03:32:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f80cd6700 1 -- 172.21.15.47:0/2334316492 learned_addr learned my addr 172.21.15.47:0/2334316492 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.580+0000 7f2f7bfff700 1 --2- 172.21.15.47:0/2334316492 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c13cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f80cd6700 1 -- 172.21.15.47:0/2334316492 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 msgr2=0x7f2f7c13cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f80cd6700 1 --2- 172.21.15.47:0/2334316492 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c13cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f80cd6700 1 -- 172.21.15.47:0/2334316492 >> v1:172.21.15.18:6789/0 conn(0x7f2f7c13fd10 legacy=0x7f2f7c0b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f80cd6700 1 -- 172.21.15.47:0/2334316492 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c1461f0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f80cd6700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f7c05f5a0 0x7f2f7c145d00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f2f6c00c000 tx=0x7f2f6c006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f6c00cbf0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00cd50 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c0b2d60 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f6c0125e0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.581+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c140820 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.582+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c1403f0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.582+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2f6c012740 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.582+0000 7f2f78ff9700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f5404ffb0 0x7f2f54052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.582+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f2f6c0683c0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.582+0000 7f2f7bfff700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f5404ffb0 0x7f2f54052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.583+0000 7f2f7bfff700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f5404ffb0 0x7f2f54052470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f2f7001c000 tx=0x7f2f70015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:49.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.586+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2f6c040ad0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.743 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.742+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2f7c057450 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.743+0000 7f2f78ff9700 1 -- 172.21.15.47:0/2334316492 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+34117 (secure 0 0 0) 0x7f2f6c0440f0 con 0x7f2f7c05f5a0 2020-07-18T03:32:49.745 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:49.745 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:32:49.745 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 55, 2020-07-18T03:32:49.746 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", 2020-07-18T03:32:49.746 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2020-07-18T03:25:24.387359+0000", 2020-07-18T03:32:49.746 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2020-07-18T03:32:08.258768+0000", 2020-07-18T03:32:49.746 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:32:49.746 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:32:49.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-07-18T03:32:49.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2020-07-18T03:32:49.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2020-07-18T03:32:49.747 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2020-07-18T03:32:49.747 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2020-07-18T03:32:49.748 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2020-07-18T03:32:49.748 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2020-07-18T03:32:49.748 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:32:49.748 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2020-07-18T03:32:49.748 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2020-07-18T03:32:49.749 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-07-18T03:32:49.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2020-07-18T03:32:49.749 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2020-07-18T03:32:49.749 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 10, 2020-07-18T03:32:49.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2020-07-18T03:32:49.750 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "luminous", 2020-07-18T03:32:49.750 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2020-07-18T03:32:49.750 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "pacific", 2020-07-18T03:32:49.750 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2020-07-18T03:32:49.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.751 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2020-07-18T03:32:49.751 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2020-07-18T03:32:49.751 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:27.725131+0000", 2020-07-18T03:32:49.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:32:49.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:32:49.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:32:49.752 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:32:49.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:32:49.752 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:32:49.752 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:32:49.753 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:32:49.753 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2020-07-18T03:32:49.753 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2020-07-18T03:32:49.753 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2020-07-18T03:32:49.753 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2020-07-18T03:32:49.754 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2020-07-18T03:32:49.754 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:32:49.754 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:32:49.754 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:32:49.754 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:32:49.754 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:32:49.755 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:32:49.755 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:32:49.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.755 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "15", 2020-07-18T03:32:49.755 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:32:49.756 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:32:49.756 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:32:49.756 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:32:49.756 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:32:49.756 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:32:49.757 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:32:49.757 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:32:49.757 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:32:49.757 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:32:49.757 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:32:49.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:32:49.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:32:49.758 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:32:49.758 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:32:49.758 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:32:49.758 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:32:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:32:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:32:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:32:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:32:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:32:49.760 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:32:49.760 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:32:49.760 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:32:49.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:32:49.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.761 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:32:49.761 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:32:49.761 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:32:49.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:32:49.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:32:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:32:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:32:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:32:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:32:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:32:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:32:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:32:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2020-07-18T03:32:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:32:49.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2020-07-18T03:32:49.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.764 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2020-07-18T03:32:49.764 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2020-07-18T03:32:49.765 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:37.896092+0000", 2020-07-18T03:32:49.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:32:49.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:32:49.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:32:49.765 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:32:49.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:32:49.766 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:32:49.766 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:32:49.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:32:49.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:32:49.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:32:49.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:32:49.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:32:49.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:32:49.767 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:32:49.767 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:32:49.768 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:32:49.768 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:32:49.768 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:32:49.768 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:32:49.768 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:32:49.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "18", 2020-07-18T03:32:49.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:32:49.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:32:49.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:32:49.770 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:32:49.770 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:32:49.770 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:32:49.770 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:32:49.770 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:32:49.770 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:32:49.771 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:32:49.771 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:32:49.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:32:49.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:32:49.771 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:32:49.772 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:32:49.772 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:32:49.772 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:32:49.772 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:32:49.772 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:32:49.773 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:32:49.773 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:32:49.773 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:32:49.773 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:32:49.773 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:32:49.773 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:32:49.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:32:49.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.774 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:32:49.774 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:32:49.774 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:32:49.775 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:32:49.775 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:32:49.775 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:32:49.775 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:32:49.775 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:32:49.776 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:32:49.776 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:32:49.776 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:32:49.776 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:32:49.776 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:32:49.776 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2020-07-18T03:32:49.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.777 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 9, 2020-07-18T03:32:49.777 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2020-07-18T03:32:49.778 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:29:55.419365+0000", 2020-07-18T03:32:49.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:32:49.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:32:49.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:32:49.778 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:32:49.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:32:49.779 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:32:49.779 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:32:49.779 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:32:49.779 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:32:49.780 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:32:49.780 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:32:49.780 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:32:49.780 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:32:49.780 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:32:49.781 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:32:49.781 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:32:49.781 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:32:49.781 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:32:49.781 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:32:49.781 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:32:49.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.782 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "52", 2020-07-18T03:32:49.782 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:32:49.782 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:32:49.782 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:32:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:32:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:32:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:32:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:32:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:32:49.784 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:32:49.784 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:32:49.784 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:32:49.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:32:49.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:32:49.784 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:32:49.785 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:32:49.785 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:32:49.785 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:32:49.785 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:32:49.785 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:32:49.786 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:32:49.786 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:32:49.786 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:32:49.786 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:32:49.786 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:32:49.787 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:32:49.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:32:49.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.787 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:32:49.787 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:32:49.788 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:32:49.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:32:49.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:32:49.788 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:32:49.788 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:32:49.788 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:32:49.789 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:32:49.789 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:32:49.789 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:32:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:32:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2020-07-18T03:32:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2020-07-18T03:32:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2020-07-18T03:32:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.790 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:32:49.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:32:49.791 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2020-07-18T03:32:49.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.792 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 10, 2020-07-18T03:32:49.792 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2020-07-18T03:32:49.792 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:29:56.654688+0000", 2020-07-18T03:32:49.792 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:32:49.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:32:49.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:32:49.793 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:32:49.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:32:49.793 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:32:49.794 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:32:49.794 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:32:49.794 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:32:49.794 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:32:49.794 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:32:49.795 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:32:49.795 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:32:49.795 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:32:49.795 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:32:49.795 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:32:49.796 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:32:49.796 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:32:49.796 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:32:49.796 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:32:49.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.797 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "54", 2020-07-18T03:32:49.797 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:32:49.797 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:32:49.797 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:32:49.797 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:32:49.797 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:32:49.798 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:32:49.798 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:32:49.798 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:32:49.798 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:32:49.798 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:32:49.799 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:32:49.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:32:49.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:32:49.799 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:32:49.799 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:32:49.800 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:32:49.800 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:32:49.800 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:32:49.800 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:32:49.800 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:32:49.801 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:32:49.801 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:32:49.801 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:32:49.801 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:32:49.801 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:32:49.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:32:49.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.802 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:32:49.802 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:32:49.802 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:32:49.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:32:49.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:32:49.803 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:32:49.803 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:32:49.803 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:32:49.804 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:32:49.804 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:32:49.804 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:32:49.804 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:32:49.804 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:32:49.805 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:32:49.805 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2020-07-18T03:32:49.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.806 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:32:49.806 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2020-07-18T03:32:49.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.806 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:32:49.806 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", 2020-07-18T03:32:49.807 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.807 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.807 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.807 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.808 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.808 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.808 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.808 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 52, 2020-07-18T03:32:49.808 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.809 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.809 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2020-07-18T03:32:49.810 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2020-07-18T03:32:49.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.812 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2020-07-18T03:32:49.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2020-07-18T03:32:49.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.814 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2020-07-18T03:32:49.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2020-07-18T03:32:49.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.816 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.817 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2020-07-18T03:32:49.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2020-07-18T03:32:49.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:32:49.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6821/23865", 2020-07-18T03:32:49.820 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6825/23865", 2020-07-18T03:32:49.820 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6832/23865", 2020-07-18T03:32:49.820 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6829/23865", 2020-07-18T03:32:49.820 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.820 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.821 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.821 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.821 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:32:49.821 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", 2020-07-18T03:32:49.822 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.822 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.822 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.822 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.822 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.823 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.823 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.823 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 41, 2020-07-18T03:32:49.823 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.823 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2020-07-18T03:32:49.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2020-07-18T03:32:49.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.826 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.826 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.826 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2020-07-18T03:32:49.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2020-07-18T03:32:49.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.829 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2020-07-18T03:32:49.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2020-07-18T03:32:49.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.831 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.831 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2020-07-18T03:32:49.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2020-07-18T03:32:49.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:32:49.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.835 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.835 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/23853", 2020-07-18T03:32:49.835 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/23853", 2020-07-18T03:32:49.836 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/23853", 2020-07-18T03:32:49.836 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/23853", 2020-07-18T03:32:49.836 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.836 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.837 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.837 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.838 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:32:49.838 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "dc0b1a97-efca-483d-839c-37c286699283", 2020-07-18T03:32:49.838 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.838 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.838 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.839 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.839 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.839 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.839 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.839 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 51, 2020-07-18T03:32:49.840 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.840 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.840 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2020-07-18T03:32:49.841 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2020-07-18T03:32:49.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.842 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.842 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.843 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2020-07-18T03:32:49.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2020-07-18T03:32:49.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.845 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.845 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.846 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2020-07-18T03:32:49.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2020-07-18T03:32:49.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.847 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.847 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2020-07-18T03:32:49.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2020-07-18T03:32:49.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:32:49.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.850 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.850 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/23872", 2020-07-18T03:32:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6824/23872", 2020-07-18T03:32:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/23872", 2020-07-18T03:32:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6828/23872", 2020-07-18T03:32:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.852 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.852 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.852 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:32:49.852 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", 2020-07-18T03:32:49.853 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.853 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.853 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.853 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.853 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.854 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.854 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.854 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 52, 2020-07-18T03:32:49.854 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.854 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2020-07-18T03:32:49.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2020-07-18T03:32:49.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.857 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.857 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.857 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2020-07-18T03:32:49.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2020-07-18T03:32:49.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.859 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2020-07-18T03:32:49.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2020-07-18T03:32:49.862 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.862 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.862 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.863 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2020-07-18T03:32:49.863 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2020-07-18T03:32:49.864 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:32:49.864 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.865 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.865 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/23854", 2020-07-18T03:32:49.865 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/23854", 2020-07-18T03:32:49.865 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/23854", 2020-07-18T03:32:49.866 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/23854", 2020-07-18T03:32:49.866 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.866 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.866 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.867 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.867 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.868 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:32:49.868 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", 2020-07-18T03:32:49.868 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.868 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.868 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.869 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.869 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.869 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.869 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.869 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 51, 2020-07-18T03:32:49.870 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.870 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.870 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.870 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6808", 2020-07-18T03:32:49.871 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.872 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6809", 2020-07-18T03:32:49.872 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.872 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.872 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.873 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.873 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.874 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6810", 2020-07-18T03:32:49.874 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.874 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.874 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6811", 2020-07-18T03:32:49.875 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.875 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.875 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.875 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.876 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.876 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.876 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6814", 2020-07-18T03:32:49.876 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.877 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.877 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6815", 2020-07-18T03:32:49.877 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.877 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.878 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.878 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.878 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.878 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6812", 2020-07-18T03:32:49.879 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.880 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6813", 2020-07-18T03:32:49.880 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:32:49.880 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.880 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.881 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6809/24092", 2020-07-18T03:32:49.881 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6811/24092", 2020-07-18T03:32:49.881 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6815/24092", 2020-07-18T03:32:49.881 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6813/24092", 2020-07-18T03:32:49.881 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.882 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.882 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.882 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.882 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:32:49.883 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", 2020-07-18T03:32:49.883 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.883 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.883 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.883 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.884 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.884 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.884 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.884 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 51, 2020-07-18T03:32:49.884 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.885 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.885 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.885 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6800", 2020-07-18T03:32:49.886 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.886 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.886 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6801", 2020-07-18T03:32:49.887 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.887 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.887 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.887 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.887 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.888 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6802", 2020-07-18T03:32:49.888 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.889 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.889 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6803", 2020-07-18T03:32:49.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.889 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.890 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.890 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6806", 2020-07-18T03:32:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6807", 2020-07-18T03:32:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6804", 2020-07-18T03:32:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6805", 2020-07-18T03:32:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:32:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6801/24091", 2020-07-18T03:32:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6803/24091", 2020-07-18T03:32:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6807/24091", 2020-07-18T03:32:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6805/24091", 2020-07-18T03:32:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:32:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", 2020-07-18T03:32:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 49, 2020-07-18T03:32:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6816", 2020-07-18T03:32:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6817", 2020-07-18T03:32:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.911 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6819", 2020-07-18T03:32:49.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6821", 2020-07-18T03:32:49.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.913 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.913 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6828", 2020-07-18T03:32:49.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6830", 2020-07-18T03:32:49.915 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.915 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.916 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6823", 2020-07-18T03:32:49.917 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6826", 2020-07-18T03:32:49.918 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:32:49.918 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.918 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.918 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6817/24093", 2020-07-18T03:32:49.919 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6821/24093", 2020-07-18T03:32:49.919 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6830/24093", 2020-07-18T03:32:49.919 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6826/24093", 2020-07-18T03:32:49.919 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.919 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.920 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.920 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.920 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:32:49.920 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", 2020-07-18T03:32:49.921 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:32:49.921 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:32:49.921 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:32:49.921 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:32:49.921 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:32:49.922 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:32:49.922 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:32:49.922 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 49, 2020-07-18T03:32:49.922 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:32:49.922 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:32:49.923 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:32:49.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.923 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6818", 2020-07-18T03:32:49.923 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6820", 2020-07-18T03:32:49.924 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.925 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.925 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.925 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:32:49.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6822", 2020-07-18T03:32:49.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.926 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6824", 2020-07-18T03:32:49.927 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.927 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.927 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.928 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:32:49.928 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.928 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.929 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6829", 2020-07-18T03:32:49.929 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6831", 2020-07-18T03:32:49.930 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.930 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.930 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.930 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:32:49.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:32:49.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:32:49.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6825", 2020-07-18T03:32:49.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:32:49.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6827", 2020-07-18T03:32:49.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:32:49.932 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.933 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6820/24094", 2020-07-18T03:32:49.933 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6824/24094", 2020-07-18T03:32:49.933 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6831/24094", 2020-07-18T03:32:49.934 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6827/24094", 2020-07-18T03:32:49.934 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:32:49.934 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:32:49.934 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:32:49.934 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:49.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.935 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:32:49.935 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2020-07-18T03:32:49.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.935 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:32:49.936 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.936 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.936 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.936 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.936 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.937 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.933644+0000", 2020-07-18T03:32:49.937 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.937 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:32:49.938 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.938 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.938 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.938 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.938 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.938 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.929073+0000", 2020-07-18T03:32:49.939 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:32:49.939 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.940 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.940 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.940 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.940 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.940 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.974285+0000", 2020-07-18T03:32:49.940 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:32:49.941 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.941 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.942 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.942 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.942 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.942 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.909674+0000", 2020-07-18T03:32:49.942 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:32:49.943 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.943 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.943 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.944 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.944 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.944 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.935590+0000", 2020-07-18T03:32:49.944 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:32:49.945 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.945 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.945 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.945 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.946 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.946 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.916457+0000", 2020-07-18T03:32:49.946 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.947 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:32:49.947 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.947 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.947 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.947 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.947 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.948 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.990672+0000", 2020-07-18T03:32:49.948 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:49.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:32:49.949 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:32:49.949 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:32:49.949 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:32:49.949 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:32:49.949 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:32:49.950 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.948777+0000", 2020-07-18T03:32:49.950 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:32:49.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.950 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:32:49.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2020-07-18T03:32:49.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2020-07-18T03:32:49.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2020-07-18T03:32:49.951 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2020-07-18T03:32:49.951 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2020-07-18T03:32:49.951 INFO:teuthology.orchestra.run.smithi047.stdout: "192.168.0.1:0/1554713892": "2020-07-18T04:32:07.456701+0000" 2020-07-18T03:32:49.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.952 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2020-07-18T03:32:49.952 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2020-07-18T03:32:49.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2020-07-18T03:32:49.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:32:49.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2020-07-18T03:32:49.953 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2020-07-18T03:32:49.953 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:32:49.953 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2020-07-18T03:32:49.953 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:32:49.953 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:32:49.954 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2020-07-18T03:32:49.954 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2020-07-18T03:32:49.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.954 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2020-07-18T03:32:49.954 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:32:49.955 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:32:49.955 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2020-07-18T03:32:49.955 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:32:49.955 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:32:49.955 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2020-07-18T03:32:49.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:49.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:49.956 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2020-07-18T03:32:49.956 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2020-07-18T03:32:49.956 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2020-07-18T03:32:49.956 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2020-07-18T03:32:49.957 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2020-07-18T03:32:49.957 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:32:49.958 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f5404ffb0 msgr2=0x7f2f54052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f5404ffb0 0x7f2f54052470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f2f7001c000 tx=0x7f2f70015000).stop 2020-07-18T03:32:49.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f7c05f5a0 msgr2=0x7f2f7c145d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:49.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f7c05f5a0 0x7f2f7c145d00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f2f6c00c000 tx=0x7f2f6c006dd0).stop 2020-07-18T03:32:49.959 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 shutdown_connections 2020-07-18T03:32:49.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f5404ffb0 0x7f2f54052470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/2334316492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f7c05f5a0 0x7f2f7c145d00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 --2- 172.21.15.47:0/2334316492 >> v2:172.21.15.18:3300/0 conn(0x7f2f7c0ad100 0x7f2f7c13cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:49.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 >> 172.21.15.47:0/2334316492 conn(0x7f2f7c0b9340 msgr2=0x7f2f7c0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:49.960 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 shutdown_connections 2020-07-18T03:32:49.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:49.745+0000 7f2f82f3a700 1 -- 172.21.15.47:0/2334316492 wait complete. 2020-07-18T03:32:49.962 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:32:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 Processor -- start 2020-07-18T03:32:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 -- start start 2020-07-18T03:32:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f255c0b41c0 0x7f255c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c0b3ac0 con 0x7f255c0b5370 2020-07-18T03:32:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0b5a10 con 0x7f255c0b41c0 2020-07-18T03:32:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.077+0000 7f2561940700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0b0040 con 0x7f255c0ad7c0 2020-07-18T03:32:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f255c0b5370 0x7f255c05fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43422/0 (socket says 172.21.15.47:43422) 2020-07-18T03:32:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255affd700 1 -- 172.21.15.47:0/3549549029 learned_addr learned my addr 172.21.15.47:0/3549549029 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255a7fc700 1 --2- 172.21.15.47:0/3549549029 >> v2:172.21.15.18:3300/0 conn(0x7f255c0b41c0 0x7f255c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255b7fe700 1 --2- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f2559ffb700 1 -- 172.21.15.47:0/3549549029 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271167297 0 0) 0x7f255c0b3ac0 con 0x7f255c0b5370 2020-07-18T03:32:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f2559ffb700 1 -- 172.21.15.47:0/3549549029 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f253c003240 con 0x7f255c0b5370 2020-07-18T03:32:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f2559ffb700 1 -- 172.21.15.47:0/3549549029 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 117239587 0 0) 0x7f253c003240 con 0x7f255c0b5370 2020-07-18T03:32:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255a7fc700 1 -- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 msgr2=0x7f255c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255a7fc700 1 --2- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255a7fc700 1 -- 172.21.15.47:0/3549549029 >> v1:172.21.15.47:6789/0 conn(0x7f255c0b5370 legacy=0x7f255c05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255a7fc700 1 -- 172.21.15.47:0/3549549029 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0b3d20 con 0x7f255c0b41c0 2020-07-18T03:32:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f255a7fc700 1 --2- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f255c0b41c0 0x7f255c0ad2d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f2554001bf0 tx=0x7f2544006dd0).ready entity=mon.0 client_cookie=3bf991748b9369fe server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f2559ffb700 1 -- 172.21.15.47:0/3549549029 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2554001a50 con 0x7f255c0b41c0 2020-07-18T03:32:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.078+0000 7f2559ffb700 1 -- 172.21.15.47:0/3549549029 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254400d570 con 0x7f255c0b41c0 2020-07-18T03:32:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2559ffb700 1 -- 172.21.15.47:0/3549549029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f254400d6e0 con 0x7f255c0b41c0 2020-07-18T03:32:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 -- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f255c0b41c0 msgr2=0x7f255c0ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 --2- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f255c0b41c0 0x7f255c0ad2d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f2554001bf0 tx=0x7f2544006dd0).stop 2020-07-18T03:32:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 -- 172.21.15.47:0/3549549029 shutdown_connections 2020-07-18T03:32:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 --2- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c1428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 --2- 172.21.15.47:0/3549549029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f255c0b41c0 0x7f255c0ad2d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 -- 172.21.15.47:0/3549549029 >> 172.21.15.47:0/3549549029 conn(0x7f255c0b9340 msgr2=0x7f255c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 -- 172.21.15.47:0/3549549029 shutdown_connections 2020-07-18T03:32:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.079+0000 7f2561940700 1 -- 172.21.15.47:0/3549549029 wait complete. 2020-07-18T03:32:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.080+0000 7f2561940700 1 Processor -- start 2020-07-18T03:32:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.080+0000 7f2561940700 1 -- start start 2020-07-18T03:32:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f2561940700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c12f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f2561940700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f255c065d60 0x7f255c0661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f2561940700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c159740 con 0x7f255c0b5370 2020-07-18T03:32:50.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f2561940700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c1305a0 con 0x7f255c065d60 2020-07-18T03:32:50.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f2561940700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c130100 con 0x7f255c0ad7c0 2020-07-18T03:32:50.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f255c065d60 0x7f255c0661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f255c065d60 0x7f255c0661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34112/0 (socket says 172.21.15.47:34112) 2020-07-18T03:32:50.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 -- 172.21.15.47:0/2562810630 learned_addr learned my addr 172.21.15.47:0/2562810630 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:50.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255affd700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c12f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 -- 172.21.15.47:0/2562810630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 msgr2=0x7f255c12f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c12f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 -- 172.21.15.47:0/2562810630 >> v1:172.21.15.18:6789/0 conn(0x7f255c0b5370 legacy=0x7f255c14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 -- 172.21.15.47:0/2562810630 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c1593c0 con 0x7f255c065d60 2020-07-18T03:32:50.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.081+0000 7f255b7fe700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f255c065d60 0x7f255c0661f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f254c00c000 tx=0x7f254c006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:50.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.082+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f254c00ca10 con 0x7f255c065d60 2020-07-18T03:32:50.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.082+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c00cb70 con 0x7f255c065d60 2020-07-18T03:32:50.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.082+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c1422d0 con 0x7f255c065d60 2020-07-18T03:32:50.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.082+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f254c0123f0 con 0x7f255c065d60 2020-07-18T03:32:50.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.082+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c14aa50 con 0x7f255c065d60 2020-07-18T03:32:50.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.083+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c1595b0 con 0x7f255c065d60 2020-07-18T03:32:50.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.083+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f254c012550 con 0x7f255c065d60 2020-07-18T03:32:50.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.083+0000 7f2552ffd700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2538050010 0x7f25380524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.083+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f254c068200 con 0x7f255c065d60 2020-07-18T03:32:50.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.083+0000 7f255affd700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2538050010 0x7f25380524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.083+0000 7f255affd700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2538050010 0x7f25380524d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f255400d000 tx=0x7f25540014e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:50.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.086+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f254c040910 con 0x7f255c065d60 2020-07-18T03:32:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.291+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f255c1598d0 con 0x7f255c065d60 2020-07-18T03:32:50.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.291+0000 7f2552ffd700 1 -- 172.21.15.47:0/2562810630 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f254c043f30 con 0x7f255c065d60 2020-07-18T03:32:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:50.293 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:29:58.861271+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/3198086169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3198086169},{"type":"v1","addr":"172.21.15.18:6833","nonce":3198086169}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:32:50.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2538050010 msgr2=0x7f25380524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2538050010 0x7f25380524d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f255400d000 tx=0x7f25540014e0).stop 2020-07-18T03:32:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f255c065d60 msgr2=0x7f255c0661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f255c065d60 0x7f255c0661f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f254c00c000 tx=0x7f254c006dd0).stop 2020-07-18T03:32:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 shutdown_connections 2020-07-18T03:32:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2538050010 0x7f25380524d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f255c065d60 0x7f255c0661f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 --2- 172.21.15.47:0/2562810630 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f255c0ad7c0 0x7f255c12f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 >> 172.21.15.47:0/2562810630 conn(0x7f255c0b9340 msgr2=0x7f255c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 shutdown_connections 2020-07-18T03:32:50.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.293+0000 7f2561940700 1 -- 172.21.15.47:0/2562810630 wait complete. 2020-07-18T03:32:50.297 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2020-07-18T03:32:50.305 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-07-18T03:32:50.335 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:32:50.339 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:32:50.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.422+0000 7f0c380c1700 1 Processor -- start 2020-07-18T03:32:50.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.422+0000 7f0c380c1700 1 -- start start 2020-07-18T03:32:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c380c1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c300b3310 0x7f0c30140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c380c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0c301411e0 0x7f0c30066b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c380c1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c300b6350 con 0x7f0c300b36f0 2020-07-18T03:32:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c380c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c300b65f0 con 0x7f0c301411e0 2020-07-18T03:32:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c380c1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c300b6890 con 0x7f0c300b3310 2020-07-18T03:32:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3665e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0c301411e0 0x7f0c30066b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3565c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c300b36f0 0x7f0c30066430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43436/0 (socket says 172.21.15.47:43436) 2020-07-18T03:32:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3565c700 1 -- 172.21.15.47:0/3429674613 learned_addr learned my addr 172.21.15.47:0/3429674613 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c35e5d700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c300b3310 0x7f0c30140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3665e700 1 -- 172.21.15.47:0/3429674613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c300b3310 msgr2=0x7f0c30140cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3665e700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c300b3310 0x7f0c30140cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3665e700 1 -- 172.21.15.47:0/3429674613 >> v1:172.21.15.47:6789/0 conn(0x7f0c300b36f0 legacy=0x7f0c30066430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c34e5b700 1 -- 172.21.15.47:0/3429674613 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790335149 0 0) 0x7f0c300b6350 con 0x7f0c300b36f0 2020-07-18T03:32:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3665e700 1 -- 172.21.15.47:0/3429674613 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c30141670 con 0x7f0c301411e0 2020-07-18T03:32:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c35e5d700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c300b3310 0x7f0c30140cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:32:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c3665e700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c301411e0 0x7f0c30066b40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f0c280032b0 tx=0x7f0c280091d0).ready entity=mon.1 client_cookie=37e405a9d6d2ee5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.423+0000 7f0c34e5b700 1 -- 172.21.15.47:0/3429674613 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2800ba50 con 0x7f0c301411e0 2020-07-18T03:32:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c34e5b700 1 -- 172.21.15.47:0/3429674613 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2800bbb0 con 0x7f0c301411e0 2020-07-18T03:32:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c34e5b700 1 -- 172.21.15.47:0/3429674613 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2800bd20 con 0x7f0c301411e0 2020-07-18T03:32:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 -- 172.21.15.47:0/3429674613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c301411e0 msgr2=0x7f0c30066b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c301411e0 0x7f0c30066b40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f0c280032b0 tx=0x7f0c280091d0).stop 2020-07-18T03:32:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 -- 172.21.15.47:0/3429674613 shutdown_connections 2020-07-18T03:32:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c301411e0 0x7f0c30066b40 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 --2- 172.21.15.47:0/3429674613 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c300b3310 0x7f0c30140cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 -- 172.21.15.47:0/3429674613 >> 172.21.15.47:0/3429674613 conn(0x7f0c3006b070 msgr2=0x7f0c3005f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 -- 172.21.15.47:0/3429674613 shutdown_connections 2020-07-18T03:32:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.424+0000 7f0c380c1700 1 -- 172.21.15.47:0/3429674613 wait complete. 2020-07-18T03:32:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.425+0000 7f0c380c1700 1 Processor -- start 2020-07-18T03:32:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.425+0000 7f0c380c1700 1 -- start start 2020-07-18T03:32:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c380c1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0c300b3310 0x7f0c301419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c380c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0c30137710 0x7f0c301493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c380c1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c30147670 con 0x7f0c300b36f0 2020-07-18T03:32:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c380c1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c30147910 con 0x7f0c300b3310 2020-07-18T03:32:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c380c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c30141ec0 con 0x7f0c30137710 2020-07-18T03:32:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3565c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c300b36f0 0x7f0c30146b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43444/0 (socket says 172.21.15.47:43444) 2020-07-18T03:32:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3565c700 1 -- 172.21.15.47:0/1435232802 learned_addr learned my addr 172.21.15.47:0/1435232802 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3665e700 1 --2- 172.21.15.47:0/1435232802 >> v2:172.21.15.47:3300/0 conn(0x7f0c30137710 0x7f0c301493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422279122 0 0) 0x7f0c30147670 con 0x7f0c300b36f0 2020-07-18T03:32:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c35e5d700 1 --2- 172.21.15.47:0/1435232802 >> v2:172.21.15.18:3300/0 conn(0x7f0c300b3310 0x7f0c301419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003240 con 0x7f0c300b36f0 2020-07-18T03:32:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3665e700 1 -- 172.21.15.47:0/1435232802 >> v2:172.21.15.18:3300/0 conn(0x7f0c300b3310 msgr2=0x7f0c301419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3665e700 1 --2- 172.21.15.47:0/1435232802 >> v2:172.21.15.18:3300/0 conn(0x7f0c300b3310 0x7f0c301419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3665e700 1 -- 172.21.15.47:0/1435232802 >> v1:172.21.15.47:6789/0 conn(0x7f0c300b36f0 legacy=0x7f0c30146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3665e700 1 -- 172.21.15.47:0/1435232802 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c30147bb0 con 0x7f0c30137710 2020-07-18T03:32:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c3665e700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30137710 0x7f0c301493a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f0c280032b0 tx=0x7f0c28016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.426+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c28010940 con 0x7f0c30137710 2020-07-18T03:32:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.427+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c28010aa0 con 0x7f0c30137710 2020-07-18T03:32:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.427+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c30147280 con 0x7f0c30137710 2020-07-18T03:32:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.427+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c28010e00 con 0x7f0c30137710 2020-07-18T03:32:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.427+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c30140900 con 0x7f0c30137710 2020-07-18T03:32:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.428+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c30069f60 con 0x7f0c30137710 2020-07-18T03:32:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.428+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0c2801e070 con 0x7f0c30137710 2020-07-18T03:32:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.428+0000 7f0c1dffb700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c080503e0 0x7f0c080528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.428+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f0c28068e70 con 0x7f0c30137710 2020-07-18T03:32:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.428+0000 7f0c35e5d700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c080503e0 0x7f0c080528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.429+0000 7f0c35e5d700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c080503e0 0x7f0c080528a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0c2c00b000 tx=0x7f0c2c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.432+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0c28041620 con 0x7f0c30137710 2020-07-18T03:32:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:50.638+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0c300b6190 con 0x7f0c30137710 2020-07-18T03:32:51.331 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:32:51.331+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:32:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.335+0000 7f0c1dffb700 1 -- 172.21.15.47:0/1435232802 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0c28015090 con 0x7f0c30137710 2020-07-18T03:32:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c080503e0 msgr2=0x7f0c080528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c080503e0 0x7f0c080528a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0c2c00b000 tx=0x7f0c2c008040).stop 2020-07-18T03:32:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30137710 msgr2=0x7f0c301493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30137710 0x7f0c301493a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f0c280032b0 tx=0x7f0c28016000).stop 2020-07-18T03:32:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 shutdown_connections 2020-07-18T03:32:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30137710 0x7f0c301493a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 --2- 172.21.15.47:0/1435232802 >> v2:172.21.15.18:3300/0 conn(0x7f0c300b3310 0x7f0c301419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 --2- 172.21.15.47:0/1435232802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c080503e0 0x7f0c080528a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 >> 172.21.15.47:0/1435232802 conn(0x7f0c3006b070 msgr2=0x7f0c300b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 shutdown_connections 2020-07-18T03:32:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.338+0000 7f0c380c1700 1 -- 172.21.15.47:0/1435232802 wait complete. 2020-07-18T03:32:51.342 INFO:tasks.ceph.mds.b.smithi018.stderr: -4400> 2020-07-18T03:30:03.292+0000 7ff632655700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:32:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-07-18T03:32:51.353 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:32:51.366 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:32:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.516+0000 7f1e239df700 1 Processor -- start 2020-07-18T03:32:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.516+0000 7f1e239df700 1 -- start start 2020-07-18T03:32:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.516+0000 7f1e239df700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e1c0b31f0 0x7f1e1c0b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0afbd0 con 0x7f1e1c0b2e10 2020-07-18T03:32:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c1491c0 con 0x7f1e1c0b2620 2020-07-18T03:32:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c144e30 con 0x7f1e1c0b31f0 2020-07-18T03:32:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e21f7c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1e1c0b2620 0x7f1e1c1454f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43450/0 (socket says 172.21.15.47:43450) 2020-07-18T03:32:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e21f7c700 1 -- 172.21.15.47:0/1190602016 learned_addr learned my addr 172.21.15.47:0/1190602016 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e20f7a700 1 --2- 172.21.15.47:0/1190602016 >> v2:172.21.15.47:3300/0 conn(0x7f1e1c0b31f0 0x7f1e1c0b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e0ffff700 1 -- 172.21.15.47:0/1190602016 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678694184 0 0) 0x7f1e1c1491c0 con 0x7f1e1c0b2620 2020-07-18T03:32:51.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e0ffff700 1 -- 172.21.15.47:0/1190602016 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e04003240 con 0x7f1e1c0b2620 2020-07-18T03:32:51.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e20f7a700 1 -- 172.21.15.47:0/1190602016 >> v1:172.21.15.47:6789/0 conn(0x7f1e1c0b2620 legacy=0x7f1e1c1454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e20f7a700 1 -- 172.21.15.47:0/1190602016 >> v1:172.21.15.18:6789/0 conn(0x7f1e1c0b2e10 legacy=0x7f1e1c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:51.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e20f7a700 1 -- 172.21.15.47:0/1190602016 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c145050 con 0x7f1e1c0b31f0 2020-07-18T03:32:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e20f7a700 1 --2- 172.21.15.47:0/1190602016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b31f0 0x7f1e1c0b2130 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f1e1c0b0bb0 tx=0x7f1e18006dd0).ready entity=mon.1 client_cookie=3f5186808565a88a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e0ffff700 1 -- 172.21.15.47:0/1190602016 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e18009770 con 0x7f1e1c0b31f0 2020-07-18T03:32:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e0ffff700 1 -- 172.21.15.47:0/1190602016 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e180098d0 con 0x7f1e1c0b31f0 2020-07-18T03:32:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e0ffff700 1 -- 172.21.15.47:0/1190602016 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e18009c30 con 0x7f1e1c0b31f0 2020-07-18T03:32:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 -- 172.21.15.47:0/1190602016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b31f0 msgr2=0x7f1e1c0b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 --2- 172.21.15.47:0/1190602016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b31f0 0x7f1e1c0b2130 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f1e1c0b0bb0 tx=0x7f1e18006dd0).stop 2020-07-18T03:32:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 -- 172.21.15.47:0/1190602016 shutdown_connections 2020-07-18T03:32:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 --2- 172.21.15.47:0/1190602016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b31f0 0x7f1e1c0b2130 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.517+0000 7f1e239df700 1 -- 172.21.15.47:0/1190602016 >> 172.21.15.47:0/1190602016 conn(0x7f1e1c0b9340 msgr2=0x7f1e1c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.518+0000 7f1e239df700 1 -- 172.21.15.47:0/1190602016 shutdown_connections 2020-07-18T03:32:51.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.518+0000 7f1e239df700 1 -- 172.21.15.47:0/1190602016 wait complete. 2020-07-18T03:32:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.519+0000 7f1e239df700 1 Processor -- start 2020-07-18T03:32:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.519+0000 7f1e239df700 1 -- start start 2020-07-18T03:32:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e239df700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e1c0b2620 0x7f1e1c147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e239df700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1e1c146420 0x7f1e1c0621a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e239df700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0630d0 con 0x7f1e1c0b2e10 2020-07-18T03:32:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e239df700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c062b90 con 0x7f1e1c146420 2020-07-18T03:32:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e239df700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c062e30 con 0x7f1e1c0b2620 2020-07-18T03:32:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e2177b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e1c0b2620 0x7f1e1c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e20f7a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1e1c0b2e10 0x7f1e1c1537a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43456/0 (socket says 172.21.15.47:43456) 2020-07-18T03:32:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e20f7a700 1 -- 172.21.15.47:0/1323921202 learned_addr learned my addr 172.21.15.47:0/1323921202 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e21f7c700 1 --2- 172.21.15.47:0/1323921202 >> v2:172.21.15.18:3300/0 conn(0x7f1e1c146420 0x7f1e1c0621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e2177b700 1 -- 172.21.15.47:0/1323921202 >> v2:172.21.15.18:3300/0 conn(0x7f1e1c146420 msgr2=0x7f1e1c0621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e2177b700 1 --2- 172.21.15.47:0/1323921202 >> v2:172.21.15.18:3300/0 conn(0x7f1e1c146420 0x7f1e1c0621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e2177b700 1 -- 172.21.15.47:0/1323921202 >> v1:172.21.15.47:6789/0 conn(0x7f1e1c0b2e10 legacy=0x7f1e1c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353169149 0 0) 0x7f1e1c0630d0 con 0x7f1e1c0b2e10 2020-07-18T03:32:51.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e2177b700 1 -- 172.21.15.47:0/1323921202 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c062690 con 0x7f1e1c0b2620 2020-07-18T03:32:51.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e21f7c700 1 --2- 172.21.15.47:0/1323921202 >> v2:172.21.15.18:3300/0 conn(0x7f1e1c146420 0x7f1e1c0621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:32:51.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.520+0000 7f1e2177b700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b2620 0x7f1e1c147150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f1e10001f50 tx=0x7f1e10007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:51.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.521+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e1000e4a0 con 0x7f1e1c0b2620 2020-07-18T03:32:51.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.521+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c062940 con 0x7f1e1c0b2620 2020-07-18T03:32:51.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.521+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1000e600 con 0x7f1e1c0b2620 2020-07-18T03:32:51.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.521+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e1000e770 con 0x7f1e1c0b2620 2020-07-18T03:32:51.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.521+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c13ca80 con 0x7f1e1c0b2620 2020-07-18T03:32:51.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.522+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c0b1f00 con 0x7f1e1c0b2620 2020-07-18T03:32:51.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.522+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1e10010030 con 0x7f1e1c0b2620 2020-07-18T03:32:51.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.522+0000 7f1e0d7fa700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1df404fe70 0x7f1df4052330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.522+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f1e10067aa0 con 0x7f1e1c0b2620 2020-07-18T03:32:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.522+0000 7f1e20f7a700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1df404fe70 0x7f1df4052330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.522+0000 7f1e20f7a700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1df404fe70 0x7f1df4052330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f1e18006f70 tx=0x7f1e18015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.526+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1e10040130 con 0x7f1e1c0b2620 2020-07-18T03:32:51.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.735+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e1c057450 con 0x7f1e1c0b2620 2020-07-18T03:32:51.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.735+0000 7f1e0d7fa700 1 -- 172.21.15.47:0/1323921202 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+1774 (secure 0 0 0) 0x7f1e10043750 con 0x7f1e1c0b2620 2020-07-18T03:32:51.737 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:51.737 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2019047305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2019047305},{"type":"v1","addr":"172.21.15.47:6835","nonce":2019047305}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:29:57.859680+0000","modified":"2020-07-18T03:32:51.331144+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-07-18T03:32:51.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1df404fe70 msgr2=0x7f1df4052330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1df404fe70 0x7f1df4052330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f1e18006f70 tx=0x7f1e18015000).stop 2020-07-18T03:32:51.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b2620 msgr2=0x7f1e1c147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b2620 0x7f1e1c147150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f1e10001f50 tx=0x7f1e10007b50).stop 2020-07-18T03:32:51.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 shutdown_connections 2020-07-18T03:32:51.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 --2- 172.21.15.47:0/1323921202 >> v2:172.21.15.18:3300/0 conn(0x7f1e1c146420 0x7f1e1c0621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e1c0b2620 0x7f1e1c147150 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 --2- 172.21.15.47:0/1323921202 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1df404fe70 0x7f1df4052330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 >> 172.21.15.47:0/1323921202 conn(0x7f1e1c0b9340 msgr2=0x7f1e1c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:51.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 shutdown_connections 2020-07-18T03:32:51.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.737+0000 7f1e239df700 1 -- 172.21.15.47:0/1323921202 wait complete. 2020-07-18T03:32:51.759 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2020-07-18T03:32:51.761 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-07-18T03:32:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.876+0000 7fe6d0539700 1 Processor -- start 2020-07-18T03:32:51.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.876+0000 7fe6d0539700 1 -- start start 2020-07-18T03:32:51.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6d0539700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6c80ad330 0x7fe6c8143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6d0539700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b59b0 0x7fe6c80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6d0539700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe6c80b6300 0x7fe6c80b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6d0539700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c8147de0 con 0x7fe6c80b6300 2020-07-18T03:32:51.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6d0539700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c80ae000 con 0x7fe6c80ad330 2020-07-18T03:32:51.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6d0539700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c8148010 con 0x7fe6c80b59b0 2020-07-18T03:32:51.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6ce2d5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6c80ad330 0x7fe6c8143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6ce2d5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6c80ad330 0x7fe6c8143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34146/0 (socket says 172.21.15.47:34146) 2020-07-18T03:32:51.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.877+0000 7fe6ce2d5700 1 -- 172.21.15.47:0/2008557643 learned_addr learned my addr 172.21.15.47:0/2008557643 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:51.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6cead6700 1 --2- 172.21.15.47:0/2008557643 >> v2:172.21.15.18:3300/0 conn(0x7fe6c80b6300 0x7fe6c80b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6cdad4700 1 --2- 172.21.15.47:0/2008557643 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b59b0 0x7fe6c80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6ce2d5700 1 -- 172.21.15.47:0/2008557643 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b59b0 msgr2=0x7fe6c80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:51.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6ce2d5700 1 --2- 172.21.15.47:0/2008557643 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b59b0 0x7fe6c80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6ce2d5700 1 -- 172.21.15.47:0/2008557643 >> v2:172.21.15.18:3300/0 conn(0x7fe6c80b6300 msgr2=0x7fe6c80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:51.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6ce2d5700 1 --2- 172.21.15.47:0/2008557643 >> v2:172.21.15.18:3300/0 conn(0x7fe6c80b6300 0x7fe6c80b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6ce2d5700 1 -- 172.21.15.47:0/2008557643 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c80ad710 con 0x7fe6c80ad330 2020-07-18T03:32:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6ce2d5700 1 --2- 172.21.15.47:0/2008557643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c80ad330 0x7fe6c8143640 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fe6bc002f30 tx=0x7fe6bc006e90).ready entity=mon.1 client_cookie=8e71c041e245b6bc server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6cd2d3700 1 -- 172.21.15.47:0/2008557643 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6bc009920 con 0x7fe6c80ad330 2020-07-18T03:32:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6cd2d3700 1 -- 172.21.15.47:0/2008557643 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc009a80 con 0x7fe6c80ad330 2020-07-18T03:32:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6cd2d3700 1 -- 172.21.15.47:0/2008557643 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6bc009de0 con 0x7fe6c80ad330 2020-07-18T03:32:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 -- 172.21.15.47:0/2008557643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c80ad330 msgr2=0x7fe6c8143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 --2- 172.21.15.47:0/2008557643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c80ad330 0x7fe6c8143640 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fe6bc002f30 tx=0x7fe6bc006e90).stop 2020-07-18T03:32:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 -- 172.21.15.47:0/2008557643 shutdown_connections 2020-07-18T03:32:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 --2- 172.21.15.47:0/2008557643 >> v2:172.21.15.18:3300/0 conn(0x7fe6c80b6300 0x7fe6c80b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 --2- 172.21.15.47:0/2008557643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6c80ad330 0x7fe6c8143640 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 --2- 172.21.15.47:0/2008557643 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b59b0 0x7fe6c80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 -- 172.21.15.47:0/2008557643 >> 172.21.15.47:0/2008557643 conn(0x7fe6c805f5a0 msgr2=0x7fe6c805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:51.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 -- 172.21.15.47:0/2008557643 shutdown_connections 2020-07-18T03:32:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.878+0000 7fe6d0539700 1 -- 172.21.15.47:0/2008557643 wait complete. 2020-07-18T03:32:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 Processor -- start 2020-07-18T03:32:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 -- start start 2020-07-18T03:32:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b6300 0x7fe6c8066760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe6c813fe90 0x7fe6c8155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c81404c0 con 0x7fe6c80b59b0 2020-07-18T03:32:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c80629e0 con 0x7fe6c813fe90 2020-07-18T03:32:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6d0539700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c8140760 con 0x7fe6c80b6300 2020-07-18T03:32:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6ce2d5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe6c80b59b0 0x7fe6c8061d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43470/0 (socket says 172.21.15.47:43470) 2020-07-18T03:32:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6ce2d5700 1 -- 172.21.15.47:0/1020271374 learned_addr learned my addr 172.21.15.47:0/1020271374 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.880+0000 7fe6cdad4700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b6300 0x7fe6c8066760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6cead6700 1 --2- 172.21.15.47:0/1020271374 >> v2:172.21.15.18:3300/0 conn(0x7fe6c813fe90 0x7fe6c8155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190103183 0 0) 0x7fe6c81404c0 con 0x7fe6c80b59b0 2020-07-18T03:32:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a0003240 con 0x7fe6c80b59b0 2020-07-18T03:32:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6cdad4700 1 -- 172.21.15.47:0/1020271374 >> v2:172.21.15.18:3300/0 conn(0x7fe6c813fe90 msgr2=0x7fe6c8155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6cdad4700 1 --2- 172.21.15.47:0/1020271374 >> v2:172.21.15.18:3300/0 conn(0x7fe6c813fe90 0x7fe6c8155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6cdad4700 1 -- 172.21.15.47:0/1020271374 >> v1:172.21.15.47:6789/0 conn(0x7fe6c80b59b0 legacy=0x7fe6c8061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6cdad4700 1 -- 172.21.15.47:0/1020271374 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8140a00 con 0x7fe6c80b6300 2020-07-18T03:32:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6cdad4700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b6300 0x7fe6c8066760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe6c8065420 tx=0x7fe6c4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6c400f7e0 con 0x7fe6c80b6300 2020-07-18T03:32:51.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c400f940 con 0x7fe6c80b6300 2020-07-18T03:32:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c81551d0 con 0x7fe6c80b6300 2020-07-18T03:32:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6c400fca0 con 0x7fe6c80b6300 2020-07-18T03:32:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.881+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c8155f80 con 0x7fe6c80b6300 2020-07-18T03:32:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.882+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c8140b90 con 0x7fe6c80b6300 2020-07-18T03:32:51.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.882+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe6c401a030 con 0x7fe6c80b6300 2020-07-18T03:32:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.883+0000 7fe6ba7fc700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6a00503f0 0x7fe6a00528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.883+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fe6c4067ab0 con 0x7fe6c80b6300 2020-07-18T03:32:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.883+0000 7fe6ce2d5700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6a00503f0 0x7fe6a00528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.883+0000 7fe6ce2d5700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6a00503f0 0x7fe6a00528b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fe6bc006f90 tx=0x7fe6bc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:51.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:51.887+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe6c4069e60 con 0x7fe6c80b6300 2020-07-18T03:32:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.094+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6c805b120 con 0x7fe6c80b6300 2020-07-18T03:32:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.339+0000 7fe6ba7fc700 1 -- 172.21.15.47:0/1020271374 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6c40400a0 con 0x7fe6c80b6300 2020-07-18T03:32:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.340+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6a00503f0 msgr2=0x7fe6a00528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.340+0000 7fe6d0539700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6a00503f0 0x7fe6a00528b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fe6bc006f90 tx=0x7fe6bc015000).stop 2020-07-18T03:32:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b6300 msgr2=0x7fe6c8066760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b6300 0x7fe6c8066760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe6c8065420 tx=0x7fe6c4008040).stop 2020-07-18T03:32:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 shutdown_connections 2020-07-18T03:32:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 --2- 172.21.15.47:0/1020271374 >> v2:172.21.15.18:3300/0 conn(0x7fe6c813fe90 0x7fe6c8155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6a00503f0 0x7fe6a00528b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 --2- 172.21.15.47:0/1020271374 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe6c80b6300 0x7fe6c8066760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 >> 172.21.15.47:0/1020271374 conn(0x7fe6c805f5a0 msgr2=0x7fe6c80b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 shutdown_connections 2020-07-18T03:32:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.341+0000 7fe6d0539700 1 -- 172.21.15.47:0/1020271374 wait complete. 2020-07-18T03:32:52.353 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-07-18T03:32:52.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.472+0000 7f2fc3782700 1 Processor -- start 2020-07-18T03:32:52.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.472+0000 7f2fc3782700 1 -- start start 2020-07-18T03:32:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc3782700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2fbc0ad330 0x7f2fbc143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc3782700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc147de0 con 0x7f2fbc0b3a10 2020-07-18T03:32:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc3782700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0ae000 con 0x7f2fbc0b59b0 2020-07-18T03:32:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc3782700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc148010 con 0x7f2fbc0ad330 2020-07-18T03:32:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc0d1d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2fbc0b59b0 0x7f2fbc0b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43476/0 (socket says 172.21.15.47:43476) 2020-07-18T03:32:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc0d1d700 1 -- 172.21.15.47:0/2284754570 learned_addr learned my addr 172.21.15.47:0/2284754570 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fc151e700 1 --2- 172.21.15.47:0/2284754570 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc0ad330 0x7f2fbc143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056684104 0 0) 0x7f2fbc0ae000 con 0x7f2fbc0b59b0 2020-07-18T03:32:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.473+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa4003240 con 0x7f2fbc0b59b0 2020-07-18T03:32:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2620262195 0 0) 0x7f2fa4003240 con 0x7f2fbc0b59b0 2020-07-18T03:32:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc0ad330 msgr2=0x7f2fbc143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 --2- 172.21.15.47:0/2284754570 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc0ad330 0x7f2fbc143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 >> v1:172.21.15.18:6789/0 conn(0x7f2fbc0b3a10 legacy=0x7f2fbc0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0ad710 con 0x7f2fbc0b59b0 2020-07-18T03:32:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f2fb80016d0 con 0x7f2fbc0b59b0 2020-07-18T03:32:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 >> v1:172.21.15.47:6789/0 conn(0x7f2fbc0b59b0 legacy=0x7f2fbc0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa4003730 0x7f2fa4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa40044e0 0x7f2fa4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fa4004e50 0x7f2fa400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc148010 con 0x7f2fa40044e0 2020-07-18T03:32:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa400bc20 con 0x7f2fa4003730 2020-07-18T03:32:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa400bd60 con 0x7f2fa4004e50 2020-07-18T03:32:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa4003730 0x7f2fa4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc0d1d700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa40044e0 0x7f2fa4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc1d1f700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fa4004e50 0x7f2fa400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 -- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fa4004e50 msgr2=0x7f2fa400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fa4004e50 0x7f2fa400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 -- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa40044e0 msgr2=0x7f2fa4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa40044e0 0x7f2fa4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 -- 172.21.15.47:0/2284754570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4003240 con 0x7f2fa4003730 2020-07-18T03:32:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc0d1d700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa40044e0 0x7f2fa4004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:32:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc1d1f700 1 -- 172.21.15.47:0/2284754570 reap_dead start 2020-07-18T03:32:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc151e700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa4003730 0x7f2fa4003b70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f2fac00c000 tx=0x7f2fac008040).ready entity=mon.1 client_cookie=2ba11c2e0dfe5f59 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fac00e970 con 0x7f2fa4003730 2020-07-18T03:32:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00ead0 con 0x7f2fa4003730 2020-07-18T03:32:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fabfff700 1 -- 172.21.15.47:0/2284754570 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fac00ee30 con 0x7f2fa4003730 2020-07-18T03:32:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc3782700 1 -- 172.21.15.47:0/2284754570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa4003730 msgr2=0x7f2fa4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.474+0000 7f2fc3782700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa4003730 0x7f2fa4003b70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f2fac00c000 tx=0x7f2fac008040).stop 2020-07-18T03:32:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.475+0000 7f2fc3782700 1 -- 172.21.15.47:0/2284754570 shutdown_connections 2020-07-18T03:32:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.475+0000 7f2fc3782700 1 --2- 172.21.15.47:0/2284754570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa4003730 0x7f2fa4003b70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.475+0000 7f2fc3782700 1 -- 172.21.15.47:0/2284754570 >> 172.21.15.47:0/2284754570 conn(0x7f2fbc05f5a0 msgr2=0x7f2fbc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.475+0000 7f2fc3782700 1 -- 172.21.15.47:0/2284754570 shutdown_connections 2020-07-18T03:32:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.475+0000 7f2fc3782700 1 -- 172.21.15.47:0/2284754570 wait complete. 2020-07-18T03:32:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.476+0000 7f2fc3782700 1 Processor -- start 2020-07-18T03:32:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.476+0000 7f2fc3782700 1 -- start start 2020-07-18T03:32:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc3782700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fbc13a5f0 0x7f2fbc151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc3782700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2fbc13a9d0 0x7f2fbc141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc3782700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc1470d0 0x7f2fbc0b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc3782700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc142ba0 con 0x7f2fbc13a9d0 2020-07-18T03:32:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc3782700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc142660 con 0x7f2fbc1470d0 2020-07-18T03:32:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc3782700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc142900 con 0x7f2fbc13a5f0 2020-07-18T03:32:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc1470d0 0x7f2fbc0b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc1470d0 0x7f2fbc0b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34170/0 (socket says 172.21.15.47:34170) 2020-07-18T03:32:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 -- 172.21.15.47:0/3676894324 learned_addr learned my addr 172.21.15.47:0/3676894324 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc0d1d700 1 --2- 172.21.15.47:0/3676894324 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc13a9d0 0x7f2fbc141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 -- 172.21.15.47:0/3676894324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fbc13a5f0 msgr2=0x7f2fbc151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc151e700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fbc13a5f0 0x7f2fbc151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fbc13a5f0 0x7f2fbc151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 -- 172.21.15.47:0/3676894324 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc13a9d0 msgr2=0x7f2fbc141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 --2- 172.21.15.47:0/3676894324 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc13a9d0 0x7f2fbc141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.477+0000 7f2fc1d1f700 1 -- 172.21.15.47:0/3676894324 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc1520e0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.478+0000 7f2fc1d1f700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc1470d0 0x7f2fbc0b28b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f2fb40011d0 tx=0x7f2fb4009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.478+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb40032c0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.478+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4010040 con 0x7f2fbc1470d0 2020-07-18T03:32:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.478+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc1422b0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.478+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb4001930 con 0x7f2fbc1470d0 2020-07-18T03:32:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.478+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc0b2fe0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.479+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc1552a0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.479+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2fb401c060 con 0x7f2fbc1470d0 2020-07-18T03:32:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.479+0000 7f2fa97fa700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9404fee0 0x7f2f940523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.479+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f2fb40671a0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.479+0000 7f2fc151e700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9404fee0 0x7f2f940523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.479+0000 7f2fc151e700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9404fee0 0x7f2f940523a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f2fac007000 tx=0x7f2fac01a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.482+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2fbc1552a0 con 0x7f2fbc1470d0 2020-07-18T03:32:52.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:52.639+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2fbc1552a0 con 0x7f2fbc1470d0 2020-07-18T03:32:53.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.387+0000 7f2fa97fa700 1 -- 172.21.15.47:0/3676894324 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2fbc1552a0 con 0x7f2fbc1470d0 2020-07-18T03:32:53.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9404fee0 msgr2=0x7f2f940523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9404fee0 0x7f2f940523a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f2fac007000 tx=0x7f2fac01a000).stop 2020-07-18T03:32:53.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc1470d0 msgr2=0x7f2fbc0b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc1470d0 0x7f2fbc0b28b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f2fb40011d0 tx=0x7f2fb4009040).stop 2020-07-18T03:32:53.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 shutdown_connections 2020-07-18T03:32:53.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9404fee0 0x7f2f940523a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:53.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc1470d0 0x7f2fbc0b28b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:53.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 --2- 172.21.15.47:0/3676894324 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f2fbc13a5f0 0x7f2fbc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:53.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 --2- 172.21.15.47:0/3676894324 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc13a9d0 0x7f2fbc141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:53.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 >> 172.21.15.47:0/3676894324 conn(0x7f2fbc05f5a0 msgr2=0x7f2fbc0b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:53.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 shutdown_connections 2020-07-18T03:32:53.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.389+0000 7f2fc3782700 1 -- 172.21.15.47:0/3676894324 wait complete. 2020-07-18T03:32:53.393 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2020-07-18T03:32:53.403 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-07-18T03:32:53.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.527+0000 7f8788c78700 1 Processor -- start 2020-07-18T03:32:53.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.527+0000 7f8788c78700 1 -- start start 2020-07-18T03:32:53.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f8788c78700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8784142a20 0x7f8784142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:53.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f8788c78700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87840af4b0 con 0x7f8784061c60 2020-07-18T03:32:53.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f8788c78700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87840b6d80 con 0x7f87840b3af0 2020-07-18T03:32:53.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f8788c78700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87840af710 con 0x7f8784142a20 2020-07-18T03:32:53.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f878259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8784142a20 0x7f8784142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:53.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f8782d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f87840b3af0 0x7f87840b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43496/0 (socket says 172.21.15.47:43496) 2020-07-18T03:32:53.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.528+0000 7f8782d9d700 1 -- 172.21.15.47:0/1996926073 learned_addr learned my addr 172.21.15.47:0/1996926073 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:53.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878259c700 1 -- 172.21.15.47:0/1996926073 >> v1:172.21.15.47:6789/0 conn(0x7f87840b3af0 legacy=0x7f87840b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878259c700 1 -- 172.21.15.47:0/1996926073 >> v1:172.21.15.18:6789/0 conn(0x7f8784061c60 legacy=0x7f87840620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878259c700 1 -- 172.21.15.47:0/1996926073 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840af9b0 con 0x7f8784142a20 2020-07-18T03:32:53.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878159a700 1 -- 172.21.15.47:0/1996926073 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057065260 0 0) 0x7f87840b6d80 con 0x7f87840b3af0 2020-07-18T03:32:53.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878259c700 1 --2- 172.21.15.47:0/1996926073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784142a20 0x7f8784142e00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f8778002f30 tx=0x7f8778006dd0).ready entity=mon.1 client_cookie=73e0a69c7433fdd8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:53.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878159a700 1 -- 172.21.15.47:0/1996926073 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87780097f0 con 0x7f8784142a20 2020-07-18T03:32:53.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878159a700 1 -- 172.21.15.47:0/1996926073 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778009950 con 0x7f8784142a20 2020-07-18T03:32:53.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f878159a700 1 -- 172.21.15.47:0/1996926073 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8778009cb0 con 0x7f8784142a20 2020-07-18T03:32:53.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 -- 172.21.15.47:0/1996926073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784142a20 msgr2=0x7f8784142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 --2- 172.21.15.47:0/1996926073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784142a20 0x7f8784142e00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f8778002f30 tx=0x7f8778006dd0).stop 2020-07-18T03:32:53.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 -- 172.21.15.47:0/1996926073 shutdown_connections 2020-07-18T03:32:53.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 --2- 172.21.15.47:0/1996926073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784142a20 0x7f8784142e00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:53.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 -- 172.21.15.47:0/1996926073 >> 172.21.15.47:0/1996926073 conn(0x7f878405f5a0 msgr2=0x7f878405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:53.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 -- 172.21.15.47:0/1996926073 shutdown_connections 2020-07-18T03:32:53.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.529+0000 7f8788c78700 1 -- 172.21.15.47:0/1996926073 wait complete. 2020-07-18T03:32:53.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 Processor -- start 2020-07-18T03:32:53.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 -- start start 2020-07-18T03:32:53.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8784061c60 0x7f8784145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:53.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87840b9f50 0x7f878413eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:53.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8784063c50 con 0x7f87840b3af0 2020-07-18T03:32:53.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8784063ef0 con 0x7f8784061c60 2020-07-18T03:32:53.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8788c78700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8784062fa0 con 0x7f87840b9f50 2020-07-18T03:32:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8782d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87840b9f50 0x7f878413eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8782d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87840b9f50 0x7f878413eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34182/0 (socket says 172.21.15.47:34182) 2020-07-18T03:32:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.531+0000 7f8782d9d700 1 -- 172.21.15.47:0/2743661184 learned_addr learned my addr 172.21.15.47:0/2743661184 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f878259c700 1 --2- 172.21.15.47:0/2743661184 >> v2:172.21.15.18:3300/0 conn(0x7f8784061c60 0x7f8784145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8782d9d700 1 -- 172.21.15.47:0/2743661184 >> v2:172.21.15.18:3300/0 conn(0x7f8784061c60 msgr2=0x7f8784145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8782d9d700 1 --2- 172.21.15.47:0/2743661184 >> v2:172.21.15.18:3300/0 conn(0x7f8784061c60 0x7f8784145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8782d9d700 1 -- 172.21.15.47:0/2743661184 >> v1:172.21.15.18:6789/0 conn(0x7f87840b3af0 legacy=0x7f87840b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8782d9d700 1 -- 172.21.15.47:0/2743661184 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840666c0 con 0x7f87840b9f50 2020-07-18T03:32:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8782d9d700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87840b9f50 0x7f878413eb40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f87840b3410 tx=0x7f877c007ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f877c005b80 con 0x7f87840b9f50 2020-07-18T03:32:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c005ce0 con 0x7f87840b9f50 2020-07-18T03:32:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8784066970 con 0x7f87840b9f50 2020-07-18T03:32:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f877c004a70 con 0x7f87840b9f50 2020-07-18T03:32:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.532+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87841456c0 con 0x7f87840b9f50 2020-07-18T03:32:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.533+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878405b120 con 0x7f87840b9f50 2020-07-18T03:32:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.533+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f877c01b060 con 0x7f87840b9f50 2020-07-18T03:32:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.533+0000 7f87727fc700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f875804ff70 0x7f8758052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.534+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f877c067640 con 0x7f87840b9f50 2020-07-18T03:32:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.534+0000 7f878259c700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f875804ff70 0x7f8758052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.534+0000 7f878259c700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f875804ff70 0x7f8758052430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f877801a000 tx=0x7f8778015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.537+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f877c03ff10 con 0x7f87840b9f50 2020-07-18T03:32:53.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:53.692+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87840690d0 con 0x7f87840b9f50 2020-07-18T03:32:54.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.391+0000 7f87727fc700 1 -- 172.21.15.47:0/2743661184 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 173+0+0 (secure 0 0 0) 0x7f877c019020 con 0x7f87840b9f50 2020-07-18T03:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f875804ff70 msgr2=0x7f8758052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f875804ff70 0x7f8758052430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f877801a000 tx=0x7f8778015000).stop 2020-07-18T03:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87840b9f50 msgr2=0x7f878413eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87840b9f50 0x7f878413eb40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f87840b3410 tx=0x7f877c007ce0).stop 2020-07-18T03:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 shutdown_connections 2020-07-18T03:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f875804ff70 0x7f8758052430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 --2- 172.21.15.47:0/2743661184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87840b9f50 0x7f878413eb40 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 --2- 172.21.15.47:0/2743661184 >> v2:172.21.15.18:3300/0 conn(0x7f8784061c60 0x7f8784145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 >> 172.21.15.47:0/2743661184 conn(0x7f878405f5a0 msgr2=0x7f87840aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 shutdown_connections 2020-07-18T03:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.393+0000 7f8788c78700 1 -- 172.21.15.47:0/2743661184 wait complete. 2020-07-18T03:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2020-07-18T03:32:54.411 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:32:54.411 INFO:tasks.ceph.mds.a:Stopping old one... 2020-07-18T03:32:54.412 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:32:54.412 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:32:54.413 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:32:54.413 INFO:tasks.ceph.mds.b:Stopping old one... 2020-07-18T03:32:54.413 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:32:54.414 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:32:54.421 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:32:54.422 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:32:54.423 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:32:54.423 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:32:54.464 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:32:54.465 DEBUG:teuthology.parallel:result is None 2020-07-18T03:32:54.466 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:32:54.466 DEBUG:teuthology.parallel:result is None 2020-07-18T03:32:54.467 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-07-18T03:32:54.551 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:32:54.552 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:32:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.598+0000 7f8eb3063700 1 Processor -- start 2020-07-18T03:32:54.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.598+0000 7f8eb3063700 1 -- start start 2020-07-18T03:32:54.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.598+0000 7f8eb3063700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8eac05ffb0 0x7f8eac0625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:54.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.598+0000 7f8eb3063700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8eac1471d0 0x7f8eac147620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:54.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.598+0000 7f8eb3063700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac0b5830 con 0x7f8eac0aea60 2020-07-18T03:32:54.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.598+0000 7f8eb3063700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac0b5a60 con 0x7f8eac05ffb0 2020-07-18T03:32:54.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac060390 con 0x7f8eac1471d0 2020-07-18T03:32:54.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8eac1471d0 0x7f8eac147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:54.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8eac1471d0 0x7f8eac147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34202/0 (socket says 172.21.15.47:34202) 2020-07-18T03:32:54.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 -- 172.21.15.47:0/4156334699 learned_addr learned my addr 172.21.15.47:0/4156334699 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:54.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb0dff700 1 --2- 172.21.15.47:0/4156334699 >> v2:172.21.15.18:3300/0 conn(0x7f8eac05ffb0 0x7f8eac0625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 -- 172.21.15.47:0/4156334699 >> v2:172.21.15.18:3300/0 conn(0x7f8eac05ffb0 msgr2=0x7f8eac0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 --2- 172.21.15.47:0/4156334699 >> v2:172.21.15.18:3300/0 conn(0x7f8eac05ffb0 0x7f8eac0625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 -- 172.21.15.47:0/4156334699 >> v1:172.21.15.18:6789/0 conn(0x7f8eac0aea60 legacy=0x7f8eac142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 -- 172.21.15.47:0/4156334699 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac146da0 con 0x7f8eac1471d0 2020-07-18T03:32:54.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb1600700 1 --2- 172.21.15.47:0/4156334699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac1471d0 0x7f8eac147620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f8eac05c960 tx=0x7f8ea400a410).ready entity=mon.1 client_cookie=f5e49e6d112b60de server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eab7fe700 1 -- 172.21.15.47:0/4156334699 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ea4010410 con 0x7f8eac1471d0 2020-07-18T03:32:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eab7fe700 1 -- 172.21.15.47:0/4156334699 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4010570 con 0x7f8eac1471d0 2020-07-18T03:32:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eab7fe700 1 -- 172.21.15.47:0/4156334699 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ea40108d0 con 0x7f8eac1471d0 2020-07-18T03:32:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 -- 172.21.15.47:0/4156334699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac1471d0 msgr2=0x7f8eac147620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:54.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 --2- 172.21.15.47:0/4156334699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac1471d0 0x7f8eac147620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f8eac05c960 tx=0x7f8ea400a410).stop 2020-07-18T03:32:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 -- 172.21.15.47:0/4156334699 shutdown_connections 2020-07-18T03:32:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 --2- 172.21.15.47:0/4156334699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac1471d0 0x7f8eac147620 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 --2- 172.21.15.47:0/4156334699 >> v2:172.21.15.18:3300/0 conn(0x7f8eac05ffb0 0x7f8eac0625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 -- 172.21.15.47:0/4156334699 >> 172.21.15.47:0/4156334699 conn(0x7f8eac0b93c0 msgr2=0x7f8eac0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:54.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 -- 172.21.15.47:0/4156334699 shutdown_connections 2020-07-18T03:32:54.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.599+0000 7f8eb3063700 1 -- 172.21.15.47:0/4156334699 wait complete. 2020-07-18T03:32:54.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.601+0000 7f8eb3063700 1 Processor -- start 2020-07-18T03:32:54.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.601+0000 7f8eb3063700 1 -- start start 2020-07-18T03:32:54.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb3063700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8eac05ffb0 0x7f8eac145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:54.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb3063700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8eac0aea60 0x7f8eac145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb3063700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac148510 con 0x7f8eac147f90 2020-07-18T03:32:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb3063700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac1487b0 con 0x7f8eac05ffb0 2020-07-18T03:32:54.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb3063700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac148a50 con 0x7f8eac0aea60 2020-07-18T03:32:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8eac05ffb0 0x7f8eac145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8eac05ffb0 0x7f8eac145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34204/0 (socket says 172.21.15.47:34204) 2020-07-18T03:32:54.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 -- 172.21.15.47:0/603518979 learned_addr learned my addr 172.21.15.47:0/603518979 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:54.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eabfff700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8eac0aea60 0x7f8eac145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:54.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 -- 172.21.15.47:0/603518979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8eac0aea60 msgr2=0x7f8eac145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:54.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8eac0aea60 0x7f8eac145c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:54.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 -- 172.21.15.47:0/603518979 >> v1:172.21.15.18:6789/0 conn(0x7f8eac147f90 legacy=0x7f8eac1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:54.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 -- 172.21.15.47:0/603518979 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac146130 con 0x7f8eac05ffb0 2020-07-18T03:32:54.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8eb0dff700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac05ffb0 0x7f8eac145750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f8eac142dd0 tx=0x7f8ea0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:54.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.602+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ea0009980 con 0x7f8eac05ffb0 2020-07-18T03:32:54.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.603+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea0009ae0 con 0x7f8eac05ffb0 2020-07-18T03:32:54.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.603+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac148ef0 con 0x7f8eac05ffb0 2020-07-18T03:32:54.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.603+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ea0009e40 con 0x7f8eac05ffb0 2020-07-18T03:32:54.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.603+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac156190 con 0x7f8eac05ffb0 2020-07-18T03:32:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.604+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8ea0013480 con 0x7f8eac05ffb0 2020-07-18T03:32:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.604+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac156400 con 0x7f8eac05ffb0 2020-07-18T03:32:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.604+0000 7f8ea8ff9700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e840500c0 0x7f8e84052580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.604+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f8ea0067ed0 con 0x7f8eac05ffb0 2020-07-18T03:32:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.604+0000 7f8eabfff700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e840500c0 0x7f8e84052580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.604+0000 7f8eabfff700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e840500c0 0x7f8e84052580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8e98008000 tx=0x7f8e98009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:54.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.608+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8eac156400 con 0x7f8eac05ffb0 2020-07-18T03:32:54.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2020-07-18T03:32:54.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:54.785+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f8eac148370 con 0x7f8eac05ffb0 2020-07-18T03:32:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.563+0000 7f8ea8ff9700 1 -- 172.21.15.47:0/603518979 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8eac148370 con 0x7f8eac05ffb0 2020-07-18T03:32:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e840500c0 msgr2=0x7f8e84052580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e840500c0 0x7f8e84052580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8e98008000 tx=0x7f8e98009040).stop 2020-07-18T03:32:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac05ffb0 msgr2=0x7f8eac145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac05ffb0 0x7f8eac145750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f8eac142dd0 tx=0x7f8ea0008040).stop 2020-07-18T03:32:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 shutdown_connections 2020-07-18T03:32:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8e840500c0 0x7f8e84052580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eac05ffb0 0x7f8eac145750 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 --2- 172.21.15.47:0/603518979 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8eac0aea60 0x7f8eac145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 >> 172.21.15.47:0/603518979 conn(0x7f8eac0b93c0 msgr2=0x7f8eac142480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 shutdown_connections 2020-07-18T03:32:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.565+0000 7f8eb3063700 1 -- 172.21.15.47:0/603518979 wait complete. 2020-07-18T03:32:55.569 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2020-07-18T03:32:55.580 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2020-07-18T03:32:55.581 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-07-18T03:32:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.703+0000 7f1aa7644700 1 Processor -- start 2020-07-18T03:32:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa7644700 1 -- start start 2020-07-18T03:32:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa7644700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 0x7f1aa005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa7644700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b4620 0x7f1aa01428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa7644700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa00b3ac0 con 0x7f1aa00b41c0 2020-07-18T03:32:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa7644700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa00b5a10 con 0x7f1aa00b4620 2020-07-18T03:32:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa7644700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00b0040 con 0x7f1aa00b5370 2020-07-18T03:32:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa4bdf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1aa00b41c0 0x7f1aa00ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43534/0 (socket says 172.21.15.47:43534) 2020-07-18T03:32:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.704+0000 7f1aa4bdf700 1 -- 172.21.15.47:0/749218543 learned_addr learned my addr 172.21.15.47:0/749218543 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/749218543 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b4620 0x7f1aa01428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1aa53e0700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 0x7f1aa005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532170472 0 0) 0x7f1aa00b3ac0 con 0x7f1aa00b41c0 2020-07-18T03:32:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a8c003240 con 0x7f1aa00b41c0 2020-07-18T03:32:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 842486978 0 0) 0x7f1a8c003240 con 0x7f1aa00b41c0 2020-07-18T03:32:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 msgr2=0x7f1aa005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 0x7f1aa005fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b4620 msgr2=0x7f1aa01428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 --2- 172.21.15.47:0/749218543 >> v2:172.21.15.18:3300/0 conn(0x7f1aa00b4620 0x7f1aa01428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa00b3d20 con 0x7f1aa00b41c0 2020-07-18T03:32:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f1a940016d0 con 0x7f1aa00b41c0 2020-07-18T03:32:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 >> v1:172.21.15.47:6789/0 conn(0x7f1aa00b41c0 legacy=0x7f1aa00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 0x7f1a8c003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a8c003c40 0x7f1a8c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0049d0 0x7f1a8c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00b5a10 con 0x7f1a8c003c40 2020-07-18T03:32:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00b0040 con 0x7f1a8c0049d0 2020-07-18T03:32:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c004200 con 0x7f1aa00b5370 2020-07-18T03:32:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0049d0 0x7f1a8c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1aa4bdf700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a8c003c40 0x7f1a8c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.705+0000 7f1aa53e0700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 0x7f1a8c003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 -- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 msgr2=0x7f1a8c003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa00b5370 0x7f1a8c003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 -- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a8c003c40 msgr2=0x7f1a8c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a8c003c40 0x7f1a8c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 -- 172.21.15.47:0/749218543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c003240 con 0x7f1a8c0049d0 2020-07-18T03:32:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0049d0 0x7f1a8c004e50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f1a9000c000 tx=0x7f1a90008040).ready entity=mon.1 client_cookie=9ab48a6d1b087663 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9000eb60 con 0x7f1a8c0049d0 2020-07-18T03:32:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9000ecc0 con 0x7f1a8c0049d0 2020-07-18T03:32:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1a9ffff700 1 -- 172.21.15.47:0/749218543 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a900125e0 con 0x7f1a8c0049d0 2020-07-18T03:32:55.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa7644700 1 -- 172.21.15.47:0/749218543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0049d0 msgr2=0x7f1a8c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa7644700 1 --2- 172.21.15.47:0/749218543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0049d0 0x7f1a8c004e50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f1a9000c000 tx=0x7f1a90008040).stop 2020-07-18T03:32:55.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa5be1700 1 -- 172.21.15.47:0/749218543 reap_dead start 2020-07-18T03:32:55.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa7644700 1 -- 172.21.15.47:0/749218543 shutdown_connections 2020-07-18T03:32:55.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa7644700 1 -- 172.21.15.47:0/749218543 >> 172.21.15.47:0/749218543 conn(0x7f1aa00b9340 msgr2=0x7f1aa00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:55.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa7644700 1 -- 172.21.15.47:0/749218543 shutdown_connections 2020-07-18T03:32:55.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.706+0000 7f1aa7644700 1 -- 172.21.15.47:0/749218543 wait complete. 2020-07-18T03:32:55.719 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 Processor -- start 2020-07-18T03:32:55.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 -- start start 2020-07-18T03:32:55.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013ffc0 0x7f1aa0065d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1aa0066250 0x7f1aa0145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1aa0146150 0x7f1aa0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa0113130 con 0x7f1aa0146150 2020-07-18T03:32:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa0115430 con 0x7f1aa0066250 2020-07-18T03:32:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa7644700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa01156d0 con 0x7f1aa013ffc0 2020-07-18T03:32:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.708+0000 7f1aa4bdf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1aa0066250 0x7f1aa0145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1aa0066250 0x7f1aa0145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34228/0 (socket says 172.21.15.47:34228) 2020-07-18T03:32:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 -- 172.21.15.47:0/1717947726 learned_addr learned my addr 172.21.15.47:0/1717947726 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa53e0700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013ffc0 0x7f1aa0065d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/1717947726 >> v2:172.21.15.18:3300/0 conn(0x7f1aa0146150 0x7f1aa0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 -- 172.21.15.47:0/1717947726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013ffc0 msgr2=0x7f1aa0065d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013ffc0 0x7f1aa0065d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 -- 172.21.15.47:0/1717947726 >> v2:172.21.15.18:3300/0 conn(0x7f1aa0146150 msgr2=0x7f1aa0155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 --2- 172.21.15.47:0/1717947726 >> v2:172.21.15.18:3300/0 conn(0x7f1aa0146150 0x7f1aa0155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 -- 172.21.15.47:0/1717947726 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa01403a0 con 0x7f1aa0066250 2020-07-18T03:32:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa5be1700 1 --2- 172.21.15.47:0/1717947726 >> v2:172.21.15.18:3300/0 conn(0x7f1aa0146150 0x7f1aa0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:32:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa4bdf700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa0066250 0x7f1aa0145c60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f1a9400d000 tx=0x7f1aa00b3eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9400fc30 con 0x7f1aa0066250 2020-07-18T03:32:55.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400fd90 con 0x7f1aa0066250 2020-07-18T03:32:55.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa0140590 con 0x7f1aa0066250 2020-07-18T03:32:55.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a94012540 con 0x7f1aa0066250 2020-07-18T03:32:55.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.709+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa0155b00 con 0x7f1aa0066250 2020-07-18T03:32:55.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.710+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa01466d0 con 0x7f1aa0066250 2020-07-18T03:32:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.710+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1a9401b030 con 0x7f1aa0066250 2020-07-18T03:32:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.711+0000 7f1a9d7fa700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a7804ff10 0x7f1a780523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.711+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1a94067490 con 0x7f1aa0066250 2020-07-18T03:32:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.711+0000 7f1aa53e0700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a7804ff10 0x7f1a780523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.711+0000 7f1aa53e0700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a7804ff10 0x7f1a780523d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f1a9800b000 tx=0x7f1a98008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.714+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1a9403ff60 con 0x7f1aa0066250 2020-07-18T03:32:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.934+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1aa0141710 con 0x7f1aa0066250 2020-07-18T03:32:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.935+0000 7f1a9d7fa700 1 -- 172.21.15.47:0/1717947726 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4513 (secure 0 0 0) 0x7f1aa0141710 con 0x7f1aa0066250 2020-07-18T03:32:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:55.937 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:32:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2020-07-18T03:32:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2020-07-18T03:32:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g==", 2020-07-18T03:32:55.939 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.939 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:32:55.939 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2020-07-18T03:32:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ==", 2020-07-18T03:32:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:32:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2020-07-18T03:32:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA==", 2020-07-18T03:32:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2020-07-18T03:32:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ==", 2020-07-18T03:32:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2020-07-18T03:32:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA==", 2020-07-18T03:32:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.947 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.947 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.948 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2020-07-18T03:32:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ==", 2020-07-18T03:32:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2020-07-18T03:32:55.951 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ==", 2020-07-18T03:32:55.951 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2020-07-18T03:32:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA==", 2020-07-18T03:32:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2020-07-18T03:32:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg==", 2020-07-18T03:32:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.957 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2020-07-18T03:32:55.957 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg==", 2020-07-18T03:32:55.957 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2020-07-18T03:32:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw==", 2020-07-18T03:32:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:32:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:32:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:32:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2020-07-18T03:32:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g==", 2020-07-18T03:32:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:32:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:32:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:32:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2020-07-18T03:32:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w==", 2020-07-18T03:32:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:32:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:32:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:32:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2020-07-18T03:32:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog==", 2020-07-18T03:32:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:32:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:32:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:32:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2020-07-18T03:32:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfi9GyDxAACUH4y/92DOhKqZyh9ejEnA==", 2020-07-18T03:32:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:32:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:32:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:32:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2020-07-18T03:32:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ==", 2020-07-18T03:32:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:32:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:32:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2020-07-18T03:32:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw==", 2020-07-18T03:32:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:32:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:32:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:32:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:32:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:32:55.971 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:32:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a7804ff10 msgr2=0x7f1a780523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a7804ff10 0x7f1a780523d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f1a9800b000 tx=0x7f1a98008040).stop 2020-07-18T03:32:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa0066250 msgr2=0x7f1aa0145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa0066250 0x7f1aa0145c60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f1a9400d000 tx=0x7f1aa00b3eb0).stop 2020-07-18T03:32:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 shutdown_connections 2020-07-18T03:32:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1a7804ff10 0x7f1a780523d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 --2- 172.21.15.47:0/1717947726 >> v2:172.21.15.18:3300/0 conn(0x7f1aa0146150 0x7f1aa0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1aa013ffc0 0x7f1aa0065d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 --2- 172.21.15.47:0/1717947726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa0066250 0x7f1aa0145c60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 >> 172.21.15.47:0/1717947726 conn(0x7f1aa00b9340 msgr2=0x7f1aa00b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 shutdown_connections 2020-07-18T03:32:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:55.936+0000 7f1aa7644700 1 -- 172.21.15.47:0/1717947726 wait complete. 2020-07-18T03:32:55.976 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:32:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.090+0000 7f6fcc744700 1 Processor -- start 2020-07-18T03:32:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.090+0000 7f6fcc744700 1 -- start start 2020-07-18T03:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fcc744700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6fc40ad100 0x7f6fc40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fcc744700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 0x7f6fc4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fcc744700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc4147330 con 0x7f6fc4147a00 2020-07-18T03:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fcc744700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40ae530 con 0x7f6fc40ad100 2020-07-18T03:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fcc744700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b57e0 con 0x7f6fc41443f0 2020-07-18T03:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fc9cdf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6fc4147a00 0x7f6fc405f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43552/0 (socket says 172.21.15.47:43552) 2020-07-18T03:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fc9cdf700 1 -- 172.21.15.47:0/1169173593 learned_addr learned my addr 172.21.15.47:0/1169173593 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fcace1700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 0x7f6fc4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463652956 0 0) 0x7f6fc4147330 con 0x7f6fc4147a00 2020-07-18T03:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fac003240 con 0x7f6fc4147a00 2020-07-18T03:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.091+0000 7f6fca4e0700 1 --2- 172.21.15.47:0/1169173593 >> v2:172.21.15.18:3300/0 conn(0x7f6fc40ad100 0x7f6fc40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3271247767 0 0) 0x7f6fac003240 con 0x7f6fc4147a00 2020-07-18T03:32:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 msgr2=0x7f6fc4147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 0x7f6fc4147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 >> v2:172.21.15.18:3300/0 conn(0x7f6fc40ad100 msgr2=0x7f6fc40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 --2- 172.21.15.47:0/1169173593 >> v2:172.21.15.18:3300/0 conn(0x7f6fc40ad100 0x7f6fc40b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40b5a80 con 0x7f6fc4147a00 2020-07-18T03:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f6fc00016d0 con 0x7f6fc4147a00 2020-07-18T03:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 >> v1:172.21.15.47:6789/0 conn(0x7f6fc4147a00 legacy=0x7f6fc405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 0x7f6fac003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac003c40 0x7f6fac0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6fac0049d0 0x7f6fac004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40ae530 con 0x7f6fac0049d0 2020-07-18T03:32:56.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b57e0 con 0x7f6fac003c40 2020-07-18T03:32:56.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac004200 con 0x7f6fc41443f0 2020-07-18T03:32:56.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac003c40 0x7f6fac0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fca4e0700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 0x7f6fac003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fcace1700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6fac0049d0 0x7f6fac004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 -- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 msgr2=0x7f6fac003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc41443f0 0x7f6fac003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 -- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6fac0049d0 msgr2=0x7f6fac004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6fac0049d0 0x7f6fac004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 -- 172.21.15.47:0/1169173593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac003240 con 0x7f6fac003c40 2020-07-18T03:32:56.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac003c40 0x7f6fac0044e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f6fc00013e0 tx=0x7f6fc40b0500).ready entity=mon.1 client_cookie=4ed0cd4b216a101f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc00070d0 con 0x7f6fac003c40 2020-07-18T03:32:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.092+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc0007230 con 0x7f6fac003c40 2020-07-18T03:32:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fc94de700 1 -- 172.21.15.47:0/1169173593 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc00055f0 con 0x7f6fac003c40 2020-07-18T03:32:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcc744700 1 -- 172.21.15.47:0/1169173593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac003c40 msgr2=0x7f6fac0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcc744700 1 --2- 172.21.15.47:0/1169173593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac003c40 0x7f6fac0044e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f6fc00013e0 tx=0x7f6fc40b0500).stop 2020-07-18T03:32:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcace1700 1 -- 172.21.15.47:0/1169173593 reap_dead start 2020-07-18T03:32:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcc744700 1 -- 172.21.15.47:0/1169173593 shutdown_connections 2020-07-18T03:32:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcc744700 1 -- 172.21.15.47:0/1169173593 >> 172.21.15.47:0/1169173593 conn(0x7f6fc40b9340 msgr2=0x7f6fc40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcc744700 1 -- 172.21.15.47:0/1169173593 shutdown_connections 2020-07-18T03:32:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.093+0000 7f6fcc744700 1 -- 172.21.15.47:0/1169173593 wait complete. 2020-07-18T03:32:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.094+0000 7f6fcc744700 1 Processor -- start 2020-07-18T03:32:56.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.094+0000 7f6fcc744700 1 -- start start 2020-07-18T03:32:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcc744700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fc4154320 0x7f6fc4066300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcc744700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc4154700 0x7f6fc4141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcc744700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40b11e0 con 0x7f6fc4061bb0 2020-07-18T03:32:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcc744700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40b1480 con 0x7f6fc4154320 2020-07-18T03:32:56.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcc744700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b1720 con 0x7f6fc4154700 2020-07-18T03:32:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fca4e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6fc4154320 0x7f6fc4066300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcace1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6fc4061bb0 0x7f6fc40b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43568/0 (socket says 172.21.15.47:43568) 2020-07-18T03:32:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcace1700 1 -- 172.21.15.47:0/2872109710 learned_addr learned my addr 172.21.15.47:0/2872109710 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc4154700 0x7f6fc4141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fca4e0700 1 -- 172.21.15.47:0/2872109710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc4154700 msgr2=0x7f6fc4141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fca4e0700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc4154700 0x7f6fc4141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fca4e0700 1 -- 172.21.15.47:0/2872109710 >> v1:172.21.15.47:6789/0 conn(0x7f6fc4061bb0 legacy=0x7f6fc40b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fca4e0700 1 -- 172.21.15.47:0/2872109710 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40667f0 con 0x7f6fc4154320 2020-07-18T03:32:56.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796143273 0 0) 0x7f6fc40b11e0 con 0x7f6fc4061bb0 2020-07-18T03:32:56.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fca4e0700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc4154320 0x7f6fc4066300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f6fb4008000 tx=0x7f6fb4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fb4011ec0 con 0x7f6fc4154320 2020-07-18T03:32:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb400f910 con 0x7f6fc4154320 2020-07-18T03:32:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.095+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc4154c90 con 0x7f6fc4154320 2020-07-18T03:32:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.096+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fb400fa80 con 0x7f6fc4154320 2020-07-18T03:32:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.096+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc41423f0 con 0x7f6fc4154320 2020-07-18T03:32:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.096+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4057450 con 0x7f6fc4154320 2020-07-18T03:32:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.097+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6fb400fc30 con 0x7f6fc4154320 2020-07-18T03:32:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.097+0000 7f6fb27fc700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6f9c04fe50 0x7f6f9c052310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.097+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6fb4067b80 con 0x7f6fc4154320 2020-07-18T03:32:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.097+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6f9c04fe50 0x7f6f9c052310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.097+0000 7f6fc9cdf700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6f9c04fe50 0x7f6f9c052310 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f6fc001d000 tx=0x7f6fc0017000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.100+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6fb4040650 con 0x7f6fc4154320 2020-07-18T03:32:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.257+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fc406a100 con 0x7f6fc4154320 2020-07-18T03:32:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.258+0000 7f6fb27fc700 1 -- 172.21.15.47:0/2872109710 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13418 (secure 0 0 0) 0x7f6fb4043c70 con 0x7f6fc4154320 2020-07-18T03:32:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6f9c04fe50 msgr2=0x7f6f9c052310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6f9c04fe50 0x7f6f9c052310 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f6fc001d000 tx=0x7f6fc0017000).stop 2020-07-18T03:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc4154320 msgr2=0x7f6fc4066300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc4154320 0x7f6fc4066300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f6fb4008000 tx=0x7f6fb4008040).stop 2020-07-18T03:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 shutdown_connections 2020-07-18T03:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6f9c04fe50 0x7f6f9c052310 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fc4154320 0x7f6fc4066300 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 --2- 172.21.15.47:0/2872109710 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6fc4154700 0x7f6fc4141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 >> 172.21.15.47:0/2872109710 conn(0x7f6fc40b9340 msgr2=0x7f6fc405fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 shutdown_connections 2020-07-18T03:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.260+0000 7f6fcc744700 1 -- 172.21.15.47:0/2872109710 wait complete. 2020-07-18T03:32:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:56.264 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":60,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:32:55.559963+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:32:56.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-07-18T03:32:56.272 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:32:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 Processor -- start 2020-07-18T03:32:56.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 -- start start 2020-07-18T03:32:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12c805fce0 0x7f12c8066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f12c80600c0 0x7f12c814a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c805f5a0 con 0x7f12c80670c0 2020-07-18T03:32:56.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c805f840 con 0x7f12c80600c0 2020-07-18T03:32:56.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.390+0000 7f12cd4be700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c8065360 con 0x7f12c805fce0 2020-07-18T03:32:56.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12c805fce0 0x7f12c8066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12c805fce0 0x7f12c8066bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34252/0 (socket says 172.21.15.47:34252) 2020-07-18T03:32:56.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c67fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f12c80670c0 0x7f12c81441b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43572/0 (socket says 172.21.15.47:43572) 2020-07-18T03:32:56.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 -- 172.21.15.47:0/348390593 learned_addr learned my addr 172.21.15.47:0/348390593 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:56.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c77fe700 1 --2- 172.21.15.47:0/348390593 >> v2:172.21.15.18:3300/0 conn(0x7f12c80600c0 0x7f12c814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 -- 172.21.15.47:0/348390593 >> v2:172.21.15.18:3300/0 conn(0x7f12c80600c0 msgr2=0x7f12c814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 --2- 172.21.15.47:0/348390593 >> v2:172.21.15.18:3300/0 conn(0x7f12c80600c0 0x7f12c814a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 -- 172.21.15.47:0/348390593 >> v1:172.21.15.47:6789/0 conn(0x7f12c80670c0 legacy=0x7f12c81441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 -- 172.21.15.47:0/348390593 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c81448c0 con 0x7f12c805fce0 2020-07-18T03:32:56.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c5ffb700 1 -- 172.21.15.47:0/348390593 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174929323 0 0) 0x7f12c805f5a0 con 0x7f12c80670c0 2020-07-18T03:32:56.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c6ffd700 1 --2- 172.21.15.47:0/348390593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c805fce0 0x7f12c8066bd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f12b8002f30 tx=0x7f12b8006e90).ready entity=mon.1 client_cookie=a2db6138562c766d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c5ffb700 1 -- 172.21.15.47:0/348390593 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b80099c0 con 0x7f12c805fce0 2020-07-18T03:32:56.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c5ffb700 1 -- 172.21.15.47:0/348390593 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b8009b20 con 0x7f12c805fce0 2020-07-18T03:32:56.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12c5ffb700 1 -- 172.21.15.47:0/348390593 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b8009e80 con 0x7f12c805fce0 2020-07-18T03:32:56.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 -- 172.21.15.47:0/348390593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c805fce0 msgr2=0x7f12c8066bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 --2- 172.21.15.47:0/348390593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c805fce0 0x7f12c8066bd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f12b8002f30 tx=0x7f12b8006e90).stop 2020-07-18T03:32:56.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 -- 172.21.15.47:0/348390593 shutdown_connections 2020-07-18T03:32:56.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 --2- 172.21.15.47:0/348390593 >> v2:172.21.15.18:3300/0 conn(0x7f12c80600c0 0x7f12c814a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 --2- 172.21.15.47:0/348390593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c805fce0 0x7f12c8066bd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 -- 172.21.15.47:0/348390593 >> 172.21.15.47:0/348390593 conn(0x7f12c80baf50 msgr2=0x7f12c805b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 -- 172.21.15.47:0/348390593 shutdown_connections 2020-07-18T03:32:56.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.391+0000 7f12cd4be700 1 -- 172.21.15.47:0/348390593 wait complete. 2020-07-18T03:32:56.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.393+0000 7f12cd4be700 1 Processor -- start 2020-07-18T03:32:56.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.393+0000 7f12cd4be700 1 -- start start 2020-07-18T03:32:56.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12c8148770 0x7f12c8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f12c80b5a10 0x7f12c8154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c81555c0 con 0x7f12c80600c0 2020-07-18T03:32:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c81491f0 con 0x7f12c80b5a10 2020-07-18T03:32:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c8155860 con 0x7f12c8148770 2020-07-18T03:32:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12c8148770 0x7f12c8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f12c80600c0 0x7f12c8155d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43578/0 (socket says 172.21.15.47:43578) 2020-07-18T03:32:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12c8148770 0x7f12c8159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34260/0 (socket says 172.21.15.47:34260) 2020-07-18T03:32:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c6ffd700 1 -- 172.21.15.47:0/3736303199 learned_addr learned my addr 172.21.15.47:0/3736303199 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:56.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c77fe700 1 --2- 172.21.15.47:0/3736303199 >> v2:172.21.15.18:3300/0 conn(0x7f12c80b5a10 0x7f12c8154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 -- 172.21.15.47:0/3736303199 >> v2:172.21.15.18:3300/0 conn(0x7f12c80b5a10 msgr2=0x7f12c8154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 --2- 172.21.15.47:0/3736303199 >> v2:172.21.15.18:3300/0 conn(0x7f12c80b5a10 0x7f12c8154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 -- 172.21.15.47:0/3736303199 >> v1:172.21.15.47:6789/0 conn(0x7f12c80600c0 legacy=0x7f12c8155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 -- 172.21.15.47:0/3736303199 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80b5ec0 con 0x7f12c8148770 2020-07-18T03:32:56.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12c67fc700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c8148770 0x7f12c8159250 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f12bc000b80 tx=0x7f12bc007920).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12bc0072e0 con 0x7f12c8148770 2020-07-18T03:32:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc005530 con 0x7f12c8148770 2020-07-18T03:32:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12bc005890 con 0x7f12c8148770 2020-07-18T03:32:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c8154cf0 con 0x7f12c8148770 2020-07-18T03:32:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.394+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c81551e0 con 0x7f12c8148770 2020-07-18T03:32:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.395+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c8155400 con 0x7f12c8148770 2020-07-18T03:32:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.396+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f12bc01a060 con 0x7f12c8148770 2020-07-18T03:32:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.396+0000 7f12aeffd700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f12a4050040 0x7f12a4052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.396+0000 7f12c6ffd700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f12a4050040 0x7f12a4052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.396+0000 7f12c6ffd700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f12a4050040 0x7f12a4052500 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f12c8155be0 tx=0x7f12b8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.397+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f12bc0437c0 con 0x7f12c8148770 2020-07-18T03:32:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.401+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f12bc03fb60 con 0x7f12c8148770 2020-07-18T03:32:56.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.558+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f12c80b79c0 con 0x7f12c8148770 2020-07-18T03:32:56.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.558+0000 7f12aeffd700 1 -- 172.21.15.47:0/3736303199 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v60) v1 ==== 110+0+10 (secure 0 0 0) 0x7f12bc018030 con 0x7f12c8148770 2020-07-18T03:32:56.560 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:32:56.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f12a4050040 msgr2=0x7f12a4052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f12a4050040 0x7f12a4052500 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f12c8155be0 tx=0x7f12b8015000).stop 2020-07-18T03:32:56.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c8148770 msgr2=0x7f12c8159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c8148770 0x7f12c8159250 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f12bc000b80 tx=0x7f12bc007920).stop 2020-07-18T03:32:56.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 shutdown_connections 2020-07-18T03:32:56.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 --2- 172.21.15.47:0/3736303199 >> v2:172.21.15.18:3300/0 conn(0x7f12c80b5a10 0x7f12c8154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f12a4050040 0x7f12a4052500 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 --2- 172.21.15.47:0/3736303199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c8148770 0x7f12c8159250 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 >> 172.21.15.47:0/3736303199 conn(0x7f12c80baf50 msgr2=0x7f12c80b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 shutdown_connections 2020-07-18T03:32:56.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.560+0000 7f12cd4be700 1 -- 172.21.15.47:0/3736303199 wait complete. 2020-07-18T03:32:56.572 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:32:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.688+0000 7f9797d79700 1 Processor -- start 2020-07-18T03:32:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.688+0000 7f9797d79700 1 -- start start 2020-07-18T03:32:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.689+0000 7f9797d79700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f97900b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.689+0000 7f9797d79700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9790147de0 con 0x7f97900b6300 2020-07-18T03:32:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.689+0000 7f9797d79700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97900ae000 con 0x7f97900ad330 2020-07-18T03:32:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.689+0000 7f9797d79700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9790148010 con 0x7f97900b59b0 2020-07-18T03:32:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.689+0000 7f9795b15700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f97900ad330 0x7f9790143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43586/0 (socket says 172.21.15.47:43586) 2020-07-18T03:32:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.689+0000 7f9795b15700 1 -- 172.21.15.47:0/242338893 learned_addr learned my addr 172.21.15.47:0/242338893 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795314700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f97900b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935683904 0 0) 0x7f97900ae000 con 0x7f97900ad330 2020-07-18T03:32:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9778003240 con 0x7f97900ad330 2020-07-18T03:32:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1009174414 0 0) 0x7f9778003240 con 0x7f97900ad330 2020-07-18T03:32:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 msgr2=0x7f97900b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f97900b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 >> v1:172.21.15.18:6789/0 conn(0x7f97900b6300 legacy=0x7f97900b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900ad710 con 0x7f97900ad330 2020-07-18T03:32:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f97800016d0 con 0x7f97900ad330 2020-07-18T03:32:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 >> v1:172.21.15.47:6789/0 conn(0x7f97900ad330 legacy=0x7f9790143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9778003730 0x7f9778003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9778004070 0x7f9778004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f9778004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790148010 con 0x7f9778004070 2020-07-18T03:32:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780050e0 con 0x7f9778003730 2020-07-18T03:32:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780052a0 con 0x7f97900b59b0 2020-07-18T03:32:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795b15700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9778003730 0x7f9778003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795314700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9778004070 0x7f9778004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9796316700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f9778004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795b15700 1 -- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 msgr2=0x7f9778004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795b15700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f9778004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795b15700 1 -- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9778004070 msgr2=0x7f9778004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795b15700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9778004070 0x7f9778004540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9795b15700 1 -- 172.21.15.47:0/242338893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778003240 con 0x7f9778003730 2020-07-18T03:32:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.690+0000 7f9796316700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97900b59b0 0x7f9778004ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:32:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9795b15700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9778003730 0x7f9778003b10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f97800013e0 tx=0x7f9780007780).ready entity=mon.1 client_cookie=17bd452f371da83c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978000ce00 con 0x7f9778003730 2020-07-18T03:32:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97800072e0 con 0x7f9778003730 2020-07-18T03:32:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9794b13700 1 -- 172.21.15.47:0/242338893 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97800057e0 con 0x7f9778003730 2020-07-18T03:32:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9797d79700 1 -- 172.21.15.47:0/242338893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9778003730 msgr2=0x7f9778003b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9797d79700 1 --2- 172.21.15.47:0/242338893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9778003730 0x7f9778003b10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f97800013e0 tx=0x7f9780007780).stop 2020-07-18T03:32:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9796316700 1 -- 172.21.15.47:0/242338893 reap_dead start 2020-07-18T03:32:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9797d79700 1 -- 172.21.15.47:0/242338893 shutdown_connections 2020-07-18T03:32:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9797d79700 1 -- 172.21.15.47:0/242338893 >> 172.21.15.47:0/242338893 conn(0x7f979005f5a0 msgr2=0x7f979005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9797d79700 1 -- 172.21.15.47:0/242338893 shutdown_connections 2020-07-18T03:32:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.691+0000 7f9797d79700 1 -- 172.21.15.47:0/242338893 wait complete. 2020-07-18T03:32:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.692+0000 7f9797d79700 1 Processor -- start 2020-07-18T03:32:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.692+0000 7f9797d79700 1 -- start start 2020-07-18T03:32:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9797d79700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979013e480 0x7f9790145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9797d79700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f979013e860 0x7f9790143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9797d79700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9790146030 con 0x7f9790144990 2020-07-18T03:32:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9797d79700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979013faa0 con 0x7f979013e860 2020-07-18T03:32:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9797d79700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979013fd40 con 0x7f979013e480 2020-07-18T03:32:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9796316700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9790144990 0x7f97901401e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43600/0 (socket says 172.21.15.47:43600) 2020-07-18T03:32:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9796316700 1 -- 172.21.15.47:0/1127868697 learned_addr learned my addr 172.21.15.47:0/1127868697 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9795314700 1 --2- 172.21.15.47:0/1127868697 >> v2:172.21.15.18:3300/0 conn(0x7f979013e860 0x7f9790143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f9795b15700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979013e480 0x7f9790145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663316947 0 0) 0x7f9790146030 con 0x7f9790144990 2020-07-18T03:32:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.693+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9768003240 con 0x7f9790144990 2020-07-18T03:32:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 684127062 0 0) 0x7f9768003240 con 0x7f9790144990 2020-07-18T03:32:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9790146030 con 0x7f9790144990 2020-07-18T03:32:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f9788001a00 con 0x7f9790144990 2020-07-18T03:32:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9795314700 1 -- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979013e480 msgr2=0x7f9790145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9795314700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979013e480 0x7f9790145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9795314700 1 -- 172.21.15.47:0/1127868697 >> v1:172.21.15.47:6789/0 conn(0x7f9790144990 legacy=0x7f97901401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9795314700 1 -- 172.21.15.47:0/1127868697 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9790145c80 con 0x7f979013e860 2020-07-18T03:32:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9795b15700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979013e480 0x7f9790145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:32:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9795314700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979013e860 0x7f9790143e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f978c008000 tx=0x7f978c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978c013590 con 0x7f979013e860 2020-07-18T03:32:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c0136f0 con 0x7f979013e860 2020-07-18T03:32:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9790156c80 con 0x7f979013e860 2020-07-18T03:32:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978c013a50 con 0x7f979013e860 2020-07-18T03:32:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.694+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9790144560 con 0x7f979013e860 2020-07-18T03:32:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.695+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979013ff10 con 0x7f979013e860 2020-07-18T03:32:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.695+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f978c013c50 con 0x7f979013e860 2020-07-18T03:32:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.696+0000 7f977dffb700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97680505e0 0x7f9768052aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.696+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f978c067cf0 con 0x7f979013e860 2020-07-18T03:32:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.696+0000 7f9795b15700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97680505e0 0x7f9768052aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.696+0000 7f9795b15700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97680505e0 0x7f9768052aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f978001a000 tx=0x7f9780015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.700+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f978c040740 con 0x7f979013e860 2020-07-18T03:32:56.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.856+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f979005b120 con 0x7f979013e860 2020-07-18T03:32:56.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.856+0000 7f977dffb700 1 -- 172.21.15.47:0/1127868697 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7f978c018070 con 0x7f979013e860 2020-07-18T03:32:56.858 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:32:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97680505e0 msgr2=0x7f9768052aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97680505e0 0x7f9768052aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f978001a000 tx=0x7f9780015000).stop 2020-07-18T03:32:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979013e860 msgr2=0x7f9790143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979013e860 0x7f9790143e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f978c008000 tx=0x7f978c008040).stop 2020-07-18T03:32:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 shutdown_connections 2020-07-18T03:32:56.861 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f97680505e0 0x7f9768052aa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f979013e480 0x7f9790145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 --2- 172.21.15.47:0/1127868697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979013e860 0x7f9790143e30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 >> 172.21.15.47:0/1127868697 conn(0x7f979005f5a0 msgr2=0x7f9790143640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 shutdown_connections 2020-07-18T03:32:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.858+0000 7f9797d79700 1 -- 172.21.15.47:0/1127868697 wait complete. 2020-07-18T03:32:56.870 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-07-18T03:32:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.984+0000 7f0c8e664700 1 Processor -- start 2020-07-18T03:32:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.984+0000 7f0c8e664700 1 -- start start 2020-07-18T03:32:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.984+0000 7f0c8e664700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 0x7f0c880b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.984+0000 7f0c8e664700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880b95a0 con 0x7f0c880b8510 2020-07-18T03:32:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.984+0000 7f0c8e664700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880b5c80 con 0x7f0c880b7420 2020-07-18T03:32:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.984+0000 7f0c8e664700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880b5a90 con 0x7f0c880b42a0 2020-07-18T03:32:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c877fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c880b7420 0x7f0c880b7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43604/0 (socket says 172.21.15.47:43604) 2020-07-18T03:32:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c877fe700 1 -- 172.21.15.47:0/2509459718 learned_addr learned my addr 172.21.15.47:0/2509459718 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c87fff700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 0x7f0c880b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749298511 0 0) 0x7f0c880b5c80 con 0x7f0c880b7420 2020-07-18T03:32:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c74003240 con 0x7f0c880b7420 2020-07-18T03:32:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3643994897 0 0) 0x7f0c74003240 con 0x7f0c880b7420 2020-07-18T03:32:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 msgr2=0x7f0c880b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 0x7f0c880b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 >> v1:172.21.15.18:6789/0 conn(0x7f0c880b8510 legacy=0x7f0c8814a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880b5f20 con 0x7f0c880b7420 2020-07-18T03:32:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f0c7c0016d0 con 0x7f0c880b7420 2020-07-18T03:32:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 >> v1:172.21.15.47:6789/0 conn(0x7f0c880b7420 legacy=0x7f0c880b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 0x7f0c74003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74003c20 0x7f0c740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c740049d0 0x7f0c74004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74004140 con 0x7f0c740049d0 2020-07-18T03:32:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c740042b0 con 0x7f0c74003c20 2020-07-18T03:32:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7400b730 con 0x7f0c880b42a0 2020-07-18T03:32:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c877fe700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74003c20 0x7f0c740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c87fff700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 0x7f0c74003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.985+0000 7f0c8cc01700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c740049d0 0x7f0c74004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c877fe700 1 -- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 msgr2=0x7f0c74003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c877fe700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880b42a0 0x7f0c74003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c877fe700 1 -- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c740049d0 msgr2=0x7f0c74004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c877fe700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c740049d0 0x7f0c74004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c877fe700 1 -- 172.21.15.47:0/2509459718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c74003240 con 0x7f0c74003c20 2020-07-18T03:32:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c877fe700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74003c20 0x7f0c740044e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f0c740054b0 tx=0x7f0c880b5e10).ready entity=mon.1 client_cookie=bcffd8cb8de7de07 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7c0070d0 con 0x7f0c74003c20 2020-07-18T03:32:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c007230 con 0x7f0c74003c20 2020-07-18T03:32:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c86ffd700 1 -- 172.21.15.47:0/2509459718 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7c0055f0 con 0x7f0c74003c20 2020-07-18T03:32:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8e664700 1 -- 172.21.15.47:0/2509459718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74003c20 msgr2=0x7f0c740044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8e664700 1 --2- 172.21.15.47:0/2509459718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74003c20 0x7f0c740044e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f0c740054b0 tx=0x7f0c880b5e10).stop 2020-07-18T03:32:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8cc01700 1 -- 172.21.15.47:0/2509459718 reap_dead start 2020-07-18T03:32:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8e664700 1 -- 172.21.15.47:0/2509459718 shutdown_connections 2020-07-18T03:32:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8e664700 1 -- 172.21.15.47:0/2509459718 >> 172.21.15.47:0/2509459718 conn(0x7f0c8805f5a0 msgr2=0x7f0c8805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8e664700 1 -- 172.21.15.47:0/2509459718 shutdown_connections 2020-07-18T03:32:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.986+0000 7f0c8e664700 1 -- 172.21.15.47:0/2509459718 wait complete. 2020-07-18T03:32:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.988+0000 7f0c8e664700 1 Processor -- start 2020-07-18T03:32:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.988+0000 7f0c8e664700 1 -- start start 2020-07-18T03:32:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.988+0000 7f0c8e664700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0c880b36a0 0x7f0c880b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.988+0000 7f0c8e664700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880ade20 0x7f0c880bbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.988+0000 7f0c8e664700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880ae330 con 0x7f0c880aed10 2020-07-18T03:32:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8e664700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880ae5d0 con 0x7f0c880b36a0 2020-07-18T03:32:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8e664700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880ae870 con 0x7f0c880ade20 2020-07-18T03:32:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c877fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c880aed10 0x7f0c880bb680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43616/0 (socket says 172.21.15.47:43616) 2020-07-18T03:32:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c877fe700 1 -- 172.21.15.47:0/1926808024 learned_addr learned my addr 172.21.15.47:0/1926808024 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8cc01700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880ade20 0x7f0c880bbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c87fff700 1 --2- 172.21.15.47:0/1926808024 >> v2:172.21.15.18:3300/0 conn(0x7f0c880b36a0 0x7f0c880b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486212275 0 0) 0x7f0c880ae330 con 0x7f0c880aed10 2020-07-18T03:32:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c60003240 con 0x7f0c880aed10 2020-07-18T03:32:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8cc01700 1 -- 172.21.15.47:0/1926808024 >> v2:172.21.15.18:3300/0 conn(0x7f0c880b36a0 msgr2=0x7f0c880b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8cc01700 1 --2- 172.21.15.47:0/1926808024 >> v2:172.21.15.18:3300/0 conn(0x7f0c880b36a0 0x7f0c880b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8cc01700 1 -- 172.21.15.47:0/1926808024 >> v1:172.21.15.47:6789/0 conn(0x7f0c880aed10 legacy=0x7f0c880bb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8cc01700 1 -- 172.21.15.47:0/1926808024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88145ca0 con 0x7f0c880ade20 2020-07-18T03:32:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8cc01700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880ade20 0x7f0c880bbd90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0c80000b20 tx=0x7f0c80007e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c80006c70 con 0x7f0c880ade20 2020-07-18T03:32:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c880bc280 con 0x7f0c880ade20 2020-07-18T03:32:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c80006dd0 con 0x7f0c880ade20 2020-07-18T03:32:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c80007260 con 0x7f0c880ade20 2020-07-18T03:32:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.989+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c881586f0 con 0x7f0c880ade20 2020-07-18T03:32:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.990+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c88057450 con 0x7f0c880ade20 2020-07-18T03:32:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.991+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0c800073c0 con 0x7f0c880ade20 2020-07-18T03:32:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.991+0000 7f0c67fff700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c600502e0 0x7f0c600527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.991+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0c800675b0 con 0x7f0c880ade20 2020-07-18T03:32:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.991+0000 7f0c87fff700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c600502e0 0x7f0c600527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.991+0000 7f0c87fff700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c600502e0 0x7f0c600527a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f0c78008000 tx=0x7f0c78008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:56.995+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0c80040000 con 0x7f0c880ade20 2020-07-18T03:32:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.155+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0c8814a5f0 con 0x7f0c880ade20 2020-07-18T03:32:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.156+0000 7f0c67fff700 1 -- 172.21.15.47:0/1926808024 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0c80043620 con 0x7f0c880ade20 2020-07-18T03:32:57.157 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:32:57.158 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2020-07-18T03:32:57.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:57.158 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2020-07-18T03:32:57.158 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2020-07-18T03:32:57.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:32:57.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:32:57.159 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2020-07-18T03:32:57.159 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2020-07-18T03:32:57.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:32:57.159 INFO:teuthology.orchestra.run.smithi047.stdout:] 2020-07-18T03:32:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c600502e0 msgr2=0x7f0c600527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c600502e0 0x7f0c600527a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f0c78008000 tx=0x7f0c78008040).stop 2020-07-18T03:32:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880ade20 msgr2=0x7f0c880bbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880ade20 0x7f0c880bbd90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0c80000b20 tx=0x7f0c80007e40).stop 2020-07-18T03:32:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 shutdown_connections 2020-07-18T03:32:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0c880ade20 0x7f0c880bbd90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 --2- 172.21.15.47:0/1926808024 >> v2:172.21.15.18:3300/0 conn(0x7f0c880b36a0 0x7f0c880b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 --2- 172.21.15.47:0/1926808024 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0c600502e0 0x7f0c600527a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 >> 172.21.15.47:0/1926808024 conn(0x7f0c8805f5a0 msgr2=0x7f0c880bcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 shutdown_connections 2020-07-18T03:32:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.157+0000 7f0c8e664700 1 -- 172.21.15.47:0/1926808024 wait complete. 2020-07-18T03:32:57.168 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-07-18T03:32:57.169 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-07-18T03:32:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.286+0000 7f361147e700 1 Processor -- start 2020-07-18T03:32:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.286+0000 7f361147e700 1 -- start start 2020-07-18T03:32:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.286+0000 7f361147e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f360c142a20 0x7f360c142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.286+0000 7f361147e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c061c60 0x7f360c0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f360c0b5dc0 0x7f360c146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0af4b0 con 0x7f360c0b5dc0 2020-07-18T03:32:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0b6d80 con 0x7f360c142a20 2020-07-18T03:32:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0af710 con 0x7f360c061c60 2020-07-18T03:32:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f360c142a20 0x7f360c142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f360c142a20 0x7f360c142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34302/0 (socket says 172.21.15.47:34302) 2020-07-18T03:32:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 -- 172.21.15.47:0/814286915 learned_addr learned my addr 172.21.15.47:0/814286915 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360a7fc700 1 --2- 172.21.15.47:0/814286915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c061c60 0x7f360c0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360b7fe700 1 --2- 172.21.15.47:0/814286915 >> v2:172.21.15.18:3300/0 conn(0x7f360c0b5dc0 0x7f360c146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 -- 172.21.15.47:0/814286915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c061c60 msgr2=0x7f360c0620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 --2- 172.21.15.47:0/814286915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c061c60 0x7f360c0620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 -- 172.21.15.47:0/814286915 >> v2:172.21.15.18:3300/0 conn(0x7f360c0b5dc0 msgr2=0x7f360c146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 --2- 172.21.15.47:0/814286915 >> v2:172.21.15.18:3300/0 conn(0x7f360c0b5dc0 0x7f360c146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 -- 172.21.15.47:0/814286915 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0af9b0 con 0x7f360c142a20 2020-07-18T03:32:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f360affd700 1 --2- 172.21.15.47:0/814286915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c142a20 0x7f360c142e00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f360c0aea90 tx=0x7f3604009270).ready entity=mon.1 client_cookie=4783bd77960a3aff server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f3609ffb700 1 -- 172.21.15.47:0/814286915 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f360400bbc0 con 0x7f360c142a20 2020-07-18T03:32:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f3609ffb700 1 -- 172.21.15.47:0/814286915 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360400bd20 con 0x7f360c142a20 2020-07-18T03:32:57.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f3609ffb700 1 -- 172.21.15.47:0/814286915 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f360400be90 con 0x7f360c142a20 2020-07-18T03:32:57.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 -- 172.21.15.47:0/814286915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c142a20 msgr2=0x7f360c142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 --2- 172.21.15.47:0/814286915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c142a20 0x7f360c142e00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f360c0aea90 tx=0x7f3604009270).stop 2020-07-18T03:32:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 -- 172.21.15.47:0/814286915 shutdown_connections 2020-07-18T03:32:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 --2- 172.21.15.47:0/814286915 >> v2:172.21.15.18:3300/0 conn(0x7f360c0b5dc0 0x7f360c146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 --2- 172.21.15.47:0/814286915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c142a20 0x7f360c142e00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 --2- 172.21.15.47:0/814286915 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c061c60 0x7f360c0620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.287+0000 7f361147e700 1 -- 172.21.15.47:0/814286915 >> 172.21.15.47:0/814286915 conn(0x7f360c05f5a0 msgr2=0x7f360c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:57.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.288+0000 7f361147e700 1 -- 172.21.15.47:0/814286915 shutdown_connections 2020-07-18T03:32:57.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.288+0000 7f361147e700 1 -- 172.21.15.47:0/814286915 wait complete. 2020-07-18T03:32:57.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.289+0000 7f361147e700 1 Processor -- start 2020-07-18T03:32:57.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.289+0000 7f361147e700 1 -- start start 2020-07-18T03:32:57.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f361147e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f360c061c60 0x7f360c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f361147e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c0b5dc0 0x7f360c133af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f361147e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c145920 con 0x7f360c145450 2020-07-18T03:32:57.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f361147e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c13adf0 con 0x7f360c061c60 2020-07-18T03:32:57.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f361147e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0b2470 con 0x7f360c0b5dc0 2020-07-18T03:32:57.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f360c061c60 0x7f360c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f360c061c60 0x7f360c0b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34306/0 (socket says 172.21.15.47:34306) 2020-07-18T03:32:57.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 -- 172.21.15.47:0/502817629 learned_addr learned my addr 172.21.15.47:0/502817629 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360a7fc700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c0b5dc0 0x7f360c133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 -- 172.21.15.47:0/502817629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c0b5dc0 msgr2=0x7f360c133af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c0b5dc0 0x7f360c133af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 -- 172.21.15.47:0/502817629 >> v1:172.21.15.18:6789/0 conn(0x7f360c145450 legacy=0x7f360c1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 -- 172.21.15.47:0/502817629 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c1466b0 con 0x7f360c061c60 2020-07-18T03:32:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f360affd700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c061c60 0x7f360c0b1f80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f360c0aea90 tx=0x7f3604016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36040108c0 con 0x7f360c061c60 2020-07-18T03:32:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.290+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604010a20 con 0x7f360c061c60 2020-07-18T03:32:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.291+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c13c2a0 con 0x7f360c061c60 2020-07-18T03:32:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.291+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3604010d80 con 0x7f360c061c60 2020-07-18T03:32:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.291+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c145d90 con 0x7f360c061c60 2020-07-18T03:32:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.291+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c05b120 con 0x7f360c061c60 2020-07-18T03:32:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.292+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f360400f070 con 0x7f360c061c60 2020-07-18T03:32:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.292+0000 7f3602ffd700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35e804ffb0 0x7f35e8052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.292+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3604069110 con 0x7f360c061c60 2020-07-18T03:32:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.292+0000 7f360a7fc700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35e804ffb0 0x7f35e8052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:57.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.292+0000 7f360a7fc700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35e804ffb0 0x7f35e8052470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f35f400c000 tx=0x7f35f4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:57.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.295+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3604041b40 con 0x7f360c061c60 2020-07-18T03:32:57.459 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:57.453+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f360c0690d0 con 0x7f360c061c60 2020-07-18T03:32:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.403+0000 7f3602ffd700 1 -- 172.21.15.47:0/502817629 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v61) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3604015070 con 0x7f360c061c60 2020-07-18T03:32:58.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35e804ffb0 msgr2=0x7f35e8052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:58.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35e804ffb0 0x7f35e8052470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f35f400c000 tx=0x7f35f4008040).stop 2020-07-18T03:32:58.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c061c60 msgr2=0x7f360c0b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:58.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c061c60 0x7f360c0b1f80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f360c0aea90 tx=0x7f3604016000).stop 2020-07-18T03:32:58.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 shutdown_connections 2020-07-18T03:32:58.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35e804ffb0 0x7f35e8052470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f360c061c60 0x7f360c0b1f80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 --2- 172.21.15.47:0/502817629 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f360c0b5dc0 0x7f360c133af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.406+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 >> 172.21.15.47:0/502817629 conn(0x7f360c05f5a0 msgr2=0x7f360c05d320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:58.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.407+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 shutdown_connections 2020-07-18T03:32:58.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.407+0000 7f361147e700 1 -- 172.21.15.47:0/502817629 wait complete. 2020-07-18T03:32:58.410 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2020-07-18T03:32:58.421 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-07-18T03:32:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.545+0000 7f9d942bf700 1 Processor -- start 2020-07-18T03:32:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.545+0000 7f9d942bf700 1 -- start start 2020-07-18T03:32:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d942bf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d8c0b59b0 0x7f9d8c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d942bf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9d8c0b6300 0x7f9d8c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d942bf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c147de0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d942bf700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0ae000 con 0x7f9d8c0b6300 2020-07-18T03:32:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d942bf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c148010 con 0x7f9d8c0b59b0 2020-07-18T03:32:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d9185a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d8c0b59b0 0x7f9d8c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d9205b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d8c0ad330 0x7f9d8c143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43634/0 (socket says 172.21.15.47:43634) 2020-07-18T03:32:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d9205b700 1 -- 172.21.15.47:0/287787370 learned_addr learned my addr 172.21.15.47:0/287787370 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.546+0000 7f9d9185a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d8c0b59b0 0x7f9d8c0b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34314/0 (socket says 172.21.15.47:34314) 2020-07-18T03:32:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9285c700 1 --2- 172.21.15.47:0/287787370 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c0b6300 0x7f9d8c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9185a700 1 -- 172.21.15.47:0/287787370 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c0b6300 msgr2=0x7f9d8c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9185a700 1 --2- 172.21.15.47:0/287787370 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c0b6300 0x7f9d8c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9185a700 1 -- 172.21.15.47:0/287787370 >> v1:172.21.15.47:6789/0 conn(0x7f9d8c0ad330 legacy=0x7f9d8c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d91059700 1 -- 172.21.15.47:0/287787370 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159180238 0 0) 0x7f9d8c147de0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9185a700 1 -- 172.21.15.47:0/287787370 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c0ad710 con 0x7f9d8c0b59b0 2020-07-18T03:32:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9285c700 1 --2- 172.21.15.47:0/287787370 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c0b6300 0x7f9d8c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:32:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d9185a700 1 --2- 172.21.15.47:0/287787370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0b59b0 0x7f9d8c0b5e10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f9d88002f30 tx=0x7f9d88006dd0).ready entity=mon.1 client_cookie=7e4c3f768685c58c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d91059700 1 -- 172.21.15.47:0/287787370 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d880097f0 con 0x7f9d8c0b59b0 2020-07-18T03:32:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d91059700 1 -- 172.21.15.47:0/287787370 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d88009950 con 0x7f9d8c0b59b0 2020-07-18T03:32:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d91059700 1 -- 172.21.15.47:0/287787370 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d88009cb0 con 0x7f9d8c0b59b0 2020-07-18T03:32:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 -- 172.21.15.47:0/287787370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0b59b0 msgr2=0x7f9d8c0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 --2- 172.21.15.47:0/287787370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0b59b0 0x7f9d8c0b5e10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f9d88002f30 tx=0x7f9d88006dd0).stop 2020-07-18T03:32:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 -- 172.21.15.47:0/287787370 shutdown_connections 2020-07-18T03:32:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 --2- 172.21.15.47:0/287787370 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c0b6300 0x7f9d8c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 --2- 172.21.15.47:0/287787370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0b59b0 0x7f9d8c0b5e10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 -- 172.21.15.47:0/287787370 >> 172.21.15.47:0/287787370 conn(0x7f9d8c05f5a0 msgr2=0x7f9d8c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 -- 172.21.15.47:0/287787370 shutdown_connections 2020-07-18T03:32:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.547+0000 7f9d942bf700 1 -- 172.21.15.47:0/287787370 wait complete. 2020-07-18T03:32:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 Processor -- start 2020-07-18T03:32:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 -- start start 2020-07-18T03:32:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d8c0ad330 0x7f9d8c064000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9d8c1340d0 0x7f9d8c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c063930 con 0x7f9d8c0b6300 2020-07-18T03:32:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c064510 con 0x7f9d8c1340d0 2020-07-18T03:32:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d942bf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c063b60 con 0x7f9d8c0ad330 2020-07-18T03:32:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d9205b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d8c0ad330 0x7f9d8c064000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d9205b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d8c0ad330 0x7f9d8c064000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34322/0 (socket says 172.21.15.47:34322) 2020-07-18T03:32:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.549+0000 7f9d9205b700 1 -- 172.21.15.47:0/887292138 learned_addr learned my addr 172.21.15.47:0/887292138 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d9285c700 1 --2- 172.21.15.47:0/887292138 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c1340d0 0x7f9d8c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d9205b700 1 -- 172.21.15.47:0/887292138 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c1340d0 msgr2=0x7f9d8c1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d9205b700 1 --2- 172.21.15.47:0/887292138 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c1340d0 0x7f9d8c1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d9205b700 1 -- 172.21.15.47:0/887292138 >> v1:172.21.15.18:6789/0 conn(0x7f9d8c0b6300 legacy=0x7f9d8c1339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d9205b700 1 -- 172.21.15.47:0/887292138 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c146480 con 0x7f9d8c0ad330 2020-07-18T03:32:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116319430 0 0) 0x7f9d8c063930 con 0x7f9d8c0b6300 2020-07-18T03:32:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d9205b700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0ad330 0x7f9d8c064000 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f9d7c000b80 tx=0x7f9d7c007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7c0072e0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c007440 con 0x7f9d8c0ad330 2020-07-18T03:32:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c1563c0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7c0057e0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.550+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c146a60 con 0x7f9d8c0ad330 2020-07-18T03:32:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.551+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c05fed0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.551+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9d7c01a060 con 0x7f9d8c0ad330 2020-07-18T03:32:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.551+0000 7f9d7a7fc700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d6404fec0 0x7f9d64052380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.552+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f9d7c0672b0 con 0x7f9d8c0ad330 2020-07-18T03:32:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.552+0000 7f9d9185a700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d6404fec0 0x7f9d64052380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.552+0000 7f9d9185a700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d6404fec0 0x7f9d64052380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f9d88000b80 tx=0x7f9d88015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.555+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9d7c03fc10 con 0x7f9d8c0ad330 2020-07-18T03:32:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:58.725+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f9d8c0b5150 con 0x7f9d8c0ad330 2020-07-18T03:32:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.566+0000 7f9d7a7fc700 1 -- 172.21.15.47:0/887292138 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v63) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9d7c018020 con 0x7f9d8c0ad330 2020-07-18T03:32:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d6404fec0 msgr2=0x7f9d64052380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d6404fec0 0x7f9d64052380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f9d88000b80 tx=0x7f9d88015000).stop 2020-07-18T03:32:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0ad330 msgr2=0x7f9d8c064000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0ad330 0x7f9d8c064000 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f9d7c000b80 tx=0x7f9d7c007a40).stop 2020-07-18T03:32:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 shutdown_connections 2020-07-18T03:32:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9d6404fec0 0x7f9d64052380 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 --2- 172.21.15.47:0/887292138 >> v2:172.21.15.18:3300/0 conn(0x7f9d8c1340d0 0x7f9d8c1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 --2- 172.21.15.47:0/887292138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d8c0ad330 0x7f9d8c064000 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 >> 172.21.15.47:0/887292138 conn(0x7f9d8c05f5a0 msgr2=0x7f9d8c0b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 shutdown_connections 2020-07-18T03:32:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.569+0000 7f9d942bf700 1 -- 172.21.15.47:0/887292138 wait complete. 2020-07-18T03:32:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2020-07-18T03:32:59.584 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-07-18T03:32:59.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.718+0000 7effb4c50700 1 Processor -- start 2020-07-18T03:32:59.720 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.718+0000 7effb4c50700 1 -- start start 2020-07-18T03:32:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.719+0000 7effb4c50700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb00ad330 0x7effb0143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.719+0000 7effb4c50700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7effb00b3a10 0x7effb00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.719+0000 7effb4c50700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb0147de0 con 0x7effb00b59b0 2020-07-18T03:32:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.719+0000 7effb4c50700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00ae000 con 0x7effb00b3a10 2020-07-18T03:32:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.719+0000 7effb4c50700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0148010 con 0x7effb00ad330 2020-07-18T03:32:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effae59c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb00ad330 0x7effb0143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effaed9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7effb00b3a10 0x7effb00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effadd9b700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7effb00b59b0 0x7effb00b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34184/0 (socket says 172.21.15.47:34184) 2020-07-18T03:32:59.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effadd9b700 1 -- 172.21.15.47:0/3782457430 learned_addr learned my addr 172.21.15.47:0/3782457430 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:32:59.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effad59a700 1 -- 172.21.15.47:0/3782457430 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006885258 0 0) 0x7effb0147de0 con 0x7effb00b59b0 2020-07-18T03:32:59.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effae59c700 1 -- 172.21.15.47:0/3782457430 >> v2:172.21.15.18:3300/0 conn(0x7effb00b3a10 msgr2=0x7effb00b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:32:59.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effae59c700 1 --2- 172.21.15.47:0/3782457430 >> v2:172.21.15.18:3300/0 conn(0x7effb00b3a10 0x7effb00b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effae59c700 1 -- 172.21.15.47:0/3782457430 >> v1:172.21.15.18:6789/0 conn(0x7effb00b59b0 legacy=0x7effb00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effae59c700 1 -- 172.21.15.47:0/3782457430 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00ad710 con 0x7effb00ad330 2020-07-18T03:32:59.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.720+0000 7effae59c700 1 --2- 172.21.15.47:0/3782457430 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb00ad330 0x7effb0143640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7effa4002f30 tx=0x7effa4006dd0).ready entity=mon.2 client_cookie=473a2736d8e188e9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:59.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effad59a700 1 -- 172.21.15.47:0/3782457430 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa4009760 con 0x7effb00ad330 2020-07-18T03:32:59.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effad59a700 1 -- 172.21.15.47:0/3782457430 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa40098c0 con 0x7effb00ad330 2020-07-18T03:32:59.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effad59a700 1 -- 172.21.15.47:0/3782457430 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa4009c20 con 0x7effb00ad330 2020-07-18T03:32:59.728 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 -- 172.21.15.47:0/3782457430 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb00ad330 msgr2=0x7effb0143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 --2- 172.21.15.47:0/3782457430 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb00ad330 0x7effb0143640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7effa4002f30 tx=0x7effa4006dd0).stop 2020-07-18T03:32:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 -- 172.21.15.47:0/3782457430 shutdown_connections 2020-07-18T03:32:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 --2- 172.21.15.47:0/3782457430 >> v2:172.21.15.18:3300/0 conn(0x7effb00b3a10 0x7effb00b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 --2- 172.21.15.47:0/3782457430 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb00ad330 0x7effb0143640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 -- 172.21.15.47:0/3782457430 >> 172.21.15.47:0/3782457430 conn(0x7effb005f5a0 msgr2=0x7effb005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 -- 172.21.15.47:0/3782457430 shutdown_connections 2020-07-18T03:32:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.721+0000 7effb4c50700 1 -- 172.21.15.47:0/3782457430 wait complete. 2020-07-18T03:32:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.722+0000 7effb4c50700 1 Processor -- start 2020-07-18T03:32:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.722+0000 7effb4c50700 1 -- start start 2020-07-18T03:32:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effb4c50700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb00b59b0 0x7effb012f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effb4c50700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7effb00643c0 0x7effb012fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effb4c50700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb0140620 con 0x7effb00b3a10 2020-07-18T03:32:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effb4c50700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb012f650 con 0x7effb00643c0 2020-07-18T03:32:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effb4c50700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00639e0 con 0x7effb00b59b0 2020-07-18T03:32:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb00b59b0 0x7effb012f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb00b59b0 0x7effb012f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34336/0 (socket says 172.21.15.47:34336) 2020-07-18T03:32:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 -- 172.21.15.47:0/1755205771 learned_addr learned my addr 172.21.15.47:0/1755205771 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:32:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effaed9d700 1 --2- 172.21.15.47:0/1755205771 >> v2:172.21.15.18:3300/0 conn(0x7effb00643c0 0x7effb012fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 -- 172.21.15.47:0/1755205771 >> v2:172.21.15.18:3300/0 conn(0x7effb00643c0 msgr2=0x7effb012fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 --2- 172.21.15.47:0/1755205771 >> v2:172.21.15.18:3300/0 conn(0x7effb00643c0 0x7effb012fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 -- 172.21.15.47:0/1755205771 >> v1:172.21.15.18:6789/0 conn(0x7effb00b3a10 legacy=0x7effb013bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 -- 172.21.15.47:0/1755205771 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0063c80 con 0x7effb00b59b0 2020-07-18T03:32:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.723+0000 7effadd9b700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb00b59b0 0x7effb012f7e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7effa0000b80 tx=0x7effa0007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.724+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa00072e0 con 0x7effb00b59b0 2020-07-18T03:32:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.724+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0007440 con 0x7effb00b59b0 2020-07-18T03:32:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.724+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00b31a0 con 0x7effb00b59b0 2020-07-18T03:32:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.724+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa00057e0 con 0x7effb00b59b0 2020-07-18T03:32:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.724+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb0130380 con 0x7effb00b59b0 2020-07-18T03:32:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.725+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0062090 con 0x7effb00b59b0 2020-07-18T03:32:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.725+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7effa001a060 con 0x7effb00b59b0 2020-07-18T03:32:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.725+0000 7eff9e7fc700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8404ffe0 0x7eff840524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:32:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.725+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6304+0+0 (secure 0 0 0) 0x7effa0067560 con 0x7effb00b59b0 2020-07-18T03:32:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.725+0000 7effae59c700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8404ffe0 0x7eff840524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:32:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.725+0000 7effae59c700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8404ffe0 0x7eff840524a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7effb013e8e0 tx=0x7effa4015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:32:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.729+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7effa003fd70 con 0x7effb00b59b0 2020-07-18T03:32:59.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.938+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7effb005b120 con 0x7effb00b59b0 2020-07-18T03:32:59.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.944+0000 7eff9e7fc700 1 -- 172.21.15.47:0/1755205771 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v28) v1 ==== 189+0+0 (secure 0 0 0) 0x7effa0018070 con 0x7effb00b59b0 2020-07-18T03:32:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.946+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8404ffe0 msgr2=0x7eff840524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.946+0000 7effb4c50700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8404ffe0 0x7eff840524a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7effb013e8e0 tx=0x7effa4015000).stop 2020-07-18T03:32:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.946+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb00b59b0 msgr2=0x7effb012f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:32:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.946+0000 7effb4c50700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb00b59b0 0x7effb012f7e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7effa0000b80 tx=0x7effa0007a60).stop 2020-07-18T03:32:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.947+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 shutdown_connections 2020-07-18T03:32:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.947+0000 7effb4c50700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8404ffe0 0x7eff840524a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.947+0000 7effb4c50700 1 --2- 172.21.15.47:0/1755205771 >> v2:172.21.15.18:3300/0 conn(0x7effb00643c0 0x7effb012fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:32:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.947+0000 7effb4c50700 1 --2- 172.21.15.47:0/1755205771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb00b59b0 0x7effb012f7e0 secure :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0x7effa0000b80 tx=0x7effa0007a60).stop 2020-07-18T03:32:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.947+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 >> 172.21.15.47:0/1755205771 conn(0x7effb005f5a0 msgr2=0x7effb00afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:32:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.949+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 shutdown_connections 2020-07-18T03:32:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:32:59.950+0000 7effb4c50700 1 -- 172.21.15.47:0/1755205771 wait complete. 2020-07-18T03:32:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 11 and data pool 12 2020-07-18T03:32:59.968 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:33:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.083+0000 7f45f7d47700 1 Processor -- start 2020-07-18T03:33:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.083+0000 7f45f7d47700 1 -- start start 2020-07-18T03:33:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f7d47700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f45f0062d60 0x7f45f005f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f7d47700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45f014acf0 0x7f45f014b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f7d47700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45f00b15a0 0x7f45f00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f7d47700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f00b7ae0 con 0x7f45f0062d60 2020-07-18T03:33:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f7d47700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f014a620 con 0x7f45f014acf0 2020-07-18T03:33:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f7d47700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f00b7d10 con 0x7f45f00b15a0 2020-07-18T03:33:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45f014acf0 0x7f45f014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45f014acf0 0x7f45f014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34354/0 (socket says 172.21.15.47:34354) 2020-07-18T03:33:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 -- 172.21.15.47:0/1313540744 learned_addr learned my addr 172.21.15.47:0/1313540744 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f62e4700 1 --2- 172.21.15.47:0/1313540744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45f00b15a0 0x7f45f00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 -- 172.21.15.47:0/1313540744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45f00b15a0 msgr2=0x7f45f00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 --2- 172.21.15.47:0/1313540744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45f00b15a0 0x7f45f00b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 -- 172.21.15.47:0/1313540744 >> v2:172.21.15.18:3300/0 conn(0x7f45f0062d60 msgr2=0x7f45f005f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 --2- 172.21.15.47:0/1313540744 >> v2:172.21.15.18:3300/0 conn(0x7f45f0062d60 0x7f45f005f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:33:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 -- 172.21.15.47:0/1313540744 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f014a850 con 0x7f45f014acf0 2020-07-18T03:33:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f52e2700 1 --2- 172.21.15.47:0/1313540744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f014acf0 0x7f45f014b150 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f45e0002f30 tx=0x7f45e0006e90).ready entity=mon.1 client_cookie=511d1d8f588c53c8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f4ae1700 1 -- 172.21.15.47:0/1313540744 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e0009920 con 0x7f45f014acf0 2020-07-18T03:33:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.084+0000 7f45f4ae1700 1 -- 172.21.15.47:0/1313540744 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e0009a80 con 0x7f45f014acf0 2020-07-18T03:33:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f4ae1700 1 -- 172.21.15.47:0/1313540744 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e0009de0 con 0x7f45f014acf0 2020-07-18T03:33:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 -- 172.21.15.47:0/1313540744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f014acf0 msgr2=0x7f45f014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 --2- 172.21.15.47:0/1313540744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f014acf0 0x7f45f014b150 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f45e0002f30 tx=0x7f45e0006e90).stop 2020-07-18T03:33:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 -- 172.21.15.47:0/1313540744 shutdown_connections 2020-07-18T03:33:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 --2- 172.21.15.47:0/1313540744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f45f00b15a0 0x7f45f00b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 --2- 172.21.15.47:0/1313540744 >> v2:172.21.15.18:3300/0 conn(0x7f45f0062d60 0x7f45f005f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:33:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 --2- 172.21.15.47:0/1313540744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f014acf0 0x7f45f014b150 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 -- 172.21.15.47:0/1313540744 >> 172.21.15.47:0/1313540744 conn(0x7f45f005cb10 msgr2=0x7f45f006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 -- 172.21.15.47:0/1313540744 shutdown_connections 2020-07-18T03:33:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.085+0000 7f45f7d47700 1 -- 172.21.15.47:0/1313540744 wait complete. 2020-07-18T03:33:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.086+0000 7f45f7d47700 1 Processor -- start 2020-07-18T03:33:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.086+0000 7f45f7d47700 1 -- start start 2020-07-18T03:33:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f7d47700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45f01144c0 0x7f45f015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f7d47700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f0149520 con 0x7f45f0062d60 2020-07-18T03:33:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f7d47700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f0149280 con 0x7f45f00b15a0 2020-07-18T03:33:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f7d47700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f0155fd0 con 0x7f45f01144c0 2020-07-18T03:33:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f52e2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45f00b15a0 0x7f45f0148660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43680/0 (socket says 172.21.15.47:43680) 2020-07-18T03:33:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f62e4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45f01144c0 0x7f45f015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f52e2700 1 -- 172.21.15.47:0/3763129716 learned_addr learned my addr 172.21.15.47:0/3763129716 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362584426 0 0) 0x7f45f0149280 con 0x7f45f00b15a0 2020-07-18T03:33:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c8003240 con 0x7f45f00b15a0 2020-07-18T03:33:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f62e4700 1 -- 172.21.15.47:0/3763129716 >> v1:172.21.15.47:6789/0 conn(0x7f45f00b15a0 legacy=0x7f45f0148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f62e4700 1 -- 172.21.15.47:0/3763129716 >> v1:172.21.15.18:6789/0 conn(0x7f45f0062d60 legacy=0x7f45f01458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900165676 0 0) 0x7f45f0149520 con 0x7f45f0062d60 2020-07-18T03:33:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f62e4700 1 -- 172.21.15.47:0/3763129716 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0155b30 con 0x7f45f01144c0 2020-07-18T03:33:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45f62e4700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f01144c0 0x7f45f015f700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f45e800a000 tx=0x7f45e8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.087+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e8009980 con 0x7f45f01144c0 2020-07-18T03:33:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.088+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8009ae0 con 0x7f45f01144c0 2020-07-18T03:33:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.088+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f0116b40 con 0x7f45f01144c0 2020-07-18T03:33:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.088+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e8009e40 con 0x7f45f01144c0 2020-07-18T03:33:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.088+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f0148f30 con 0x7f45f01144c0 2020-07-18T03:33:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.089+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f45e8013480 con 0x7f45f01144c0 2020-07-18T03:33:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.089+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f0057450 con 0x7f45f01144c0 2020-07-18T03:33:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.089+0000 7f45ddffb700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45c80502e0 0x7f45c80527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.089+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f45e8068050 con 0x7f45f01144c0 2020-07-18T03:33:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.089+0000 7f45f5ae3700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45c80502e0 0x7f45c80527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.089+0000 7f45f5ae3700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45c80502e0 0x7f45c80527a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f45ec002080 tx=0x7f45ec007990).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.093+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f45e8040780 con 0x7f45f01144c0 2020-07-18T03:33:00.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.250+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f006a100 con 0x7f45f01144c0 2020-07-18T03:33:00.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.251+0000 7f45ddffb700 1 -- 172.21.15.47:0/3763129716 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16290 (secure 0 0 0) 0x7f45e8043da0 con 0x7f45f01144c0 2020-07-18T03:33:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:00.253 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":64,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:32:59.939942+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:32:57.493054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-07-18T03:32:58.764476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:33:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45c80502e0 msgr2=0x7f45c80527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45c80502e0 0x7f45c80527a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f45ec002080 tx=0x7f45ec007990).stop 2020-07-18T03:33:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f01144c0 msgr2=0x7f45f015f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f01144c0 0x7f45f015f700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f45e800a000 tx=0x7f45e8008040).stop 2020-07-18T03:33:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 shutdown_connections 2020-07-18T03:33:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45f01144c0 0x7f45f015f700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 --2- 172.21.15.47:0/3763129716 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45c80502e0 0x7f45c80527a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 >> 172.21.15.47:0/3763129716 conn(0x7f45f005cb10 msgr2=0x7f45f005b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 shutdown_connections 2020-07-18T03:33:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.252+0000 7f45f7d47700 1 -- 172.21.15.47:0/3763129716 wait complete. 2020-07-18T03:33:00.264 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:33:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.379+0000 7fe0bd195700 1 Processor -- start 2020-07-18T03:33:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.379+0000 7fe0bd195700 1 -- start start 2020-07-18T03:33:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0bd195700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 0x7fe0b80b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0bd195700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe0b80b2620 0x7fe0b81454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0bd195700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b80afbd0 con 0x7fe0b80b2e10 2020-07-18T03:33:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0bd195700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b8144e30 con 0x7fe0b80b2620 2020-07-18T03:33:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0bd195700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b81491c0 con 0x7fe0b80b31f0 2020-07-18T03:33:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b6d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe0b80b2e10 0x7fe0b80b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43686/0 (socket says 172.21.15.47:43686) 2020-07-18T03:33:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b6d9d700 1 -- 172.21.15.47:0/2329686947 learned_addr learned my addr 172.21.15.47:0/2329686947 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b659c700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 0x7fe0b80b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b759e700 1 --2- 172.21.15.47:0/2329686947 >> v2:172.21.15.18:3300/0 conn(0x7fe0b80b2620 0x7fe0b81454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920761667 0 0) 0x7fe0ac0010d0 con 0x7fe0b80b2e10 2020-07-18T03:33:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a4003240 con 0x7fe0b80b2e10 2020-07-18T03:33:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1700642015 0 0) 0x7fe0a4003240 con 0x7fe0b80b2e10 2020-07-18T03:33:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 msgr2=0x7fe0b80b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 0x7fe0b80b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 >> v2:172.21.15.18:3300/0 conn(0x7fe0b80b2620 msgr2=0x7fe0b81454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 --2- 172.21.15.47:0/2329686947 >> v2:172.21.15.18:3300/0 conn(0x7fe0b80b2620 0x7fe0b81454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8145050 con 0x7fe0b80b2e10 2020-07-18T03:33:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fe0ac001860 con 0x7fe0b80b2e10 2020-07-18T03:33:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 >> v1:172.21.15.47:6789/0 conn(0x7fe0b80b2e10 legacy=0x7fe0b80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 0x7fe0a4003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0a4003c40 0x7fe0a40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a40049d0 0x7fe0a4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b8144e30 con 0x7fe0a4003c40 2020-07-18T03:33:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b81491c0 con 0x7fe0a40049d0 2020-07-18T03:33:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.380+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4004200 con 0x7fe0b80b31f0 2020-07-18T03:33:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a40049d0 0x7fe0a4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b6d9d700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 0x7fe0a4003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b659c700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0a4003c40 0x7fe0a40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 -- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 msgr2=0x7fe0a4003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b80b31f0 0x7fe0a4003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 -- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0a4003c40 msgr2=0x7fe0a40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0a4003c40 0x7fe0a40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 -- 172.21.15.47:0/2329686947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4003240 con 0x7fe0a40049d0 2020-07-18T03:33:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a40049d0 0x7fe0a4004e50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fe0b000c000 tx=0x7fe0b00091d0).ready entity=mon.1 client_cookie=d0e3432e2d155c0d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b000ec30 con 0x7fe0a40049d0 2020-07-18T03:33:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b000ed90 con 0x7fe0a40049d0 2020-07-18T03:33:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b5d9b700 1 -- 172.21.15.47:0/2329686947 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b00125e0 con 0x7fe0a40049d0 2020-07-18T03:33:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0bd195700 1 -- 172.21.15.47:0/2329686947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a40049d0 msgr2=0x7fe0a4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0bd195700 1 --2- 172.21.15.47:0/2329686947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a40049d0 0x7fe0a4004e50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fe0b000c000 tx=0x7fe0b00091d0).stop 2020-07-18T03:33:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0b759e700 1 -- 172.21.15.47:0/2329686947 reap_dead start 2020-07-18T03:33:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0bd195700 1 -- 172.21.15.47:0/2329686947 shutdown_connections 2020-07-18T03:33:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0bd195700 1 -- 172.21.15.47:0/2329686947 >> 172.21.15.47:0/2329686947 conn(0x7fe0b80b9340 msgr2=0x7fe0b80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0bd195700 1 -- 172.21.15.47:0/2329686947 shutdown_connections 2020-07-18T03:33:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.381+0000 7fe0bd195700 1 -- 172.21.15.47:0/2329686947 wait complete. 2020-07-18T03:33:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.383+0000 7fe0bd195700 1 Processor -- start 2020-07-18T03:33:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.383+0000 7fe0bd195700 1 -- start start 2020-07-18T03:33:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0bd195700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b8141ad0 0x7fe0b8154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0bd195700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe0b8141eb0 0x7fe0b81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0bd195700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b80b8450 con 0x7fe0b80b8e10 2020-07-18T03:33:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0bd195700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b8156460 con 0x7fe0b8141eb0 2020-07-18T03:33:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0bd195700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b81459b0 con 0x7fe0b8141ad0 2020-07-18T03:33:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b759e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe0b80b8e10 0x7fe0b8158b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43702/0 (socket says 172.21.15.47:43702) 2020-07-18T03:33:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b759e700 1 -- 172.21.15.47:0/1959065680 learned_addr learned my addr 172.21.15.47:0/1959065680 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b6d9d700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b8141ad0 0x7fe0b8154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b659c700 1 --2- 172.21.15.47:0/1959065680 >> v2:172.21.15.18:3300/0 conn(0x7fe0b8141eb0 0x7fe0b81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722915375 0 0) 0x7fe0b80b8450 con 0x7fe0b80b8e10 2020-07-18T03:33:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe094003240 con 0x7fe0b80b8e10 2020-07-18T03:33:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b6d9d700 1 -- 172.21.15.47:0/1959065680 >> v2:172.21.15.18:3300/0 conn(0x7fe0b8141eb0 msgr2=0x7fe0b81550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b6d9d700 1 --2- 172.21.15.47:0/1959065680 >> v2:172.21.15.18:3300/0 conn(0x7fe0b8141eb0 0x7fe0b81550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b6d9d700 1 -- 172.21.15.47:0/1959065680 >> v1:172.21.15.47:6789/0 conn(0x7fe0b80b8e10 legacy=0x7fe0b8158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b6d9d700 1 -- 172.21.15.47:0/1959065680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8145af0 con 0x7fe0b8141ad0 2020-07-18T03:33:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe0b6d9d700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b8141ad0 0x7fe0b8154bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe0ac00d000 tx=0x7fe0b80b5d00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.384+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ac00f8d0 con 0x7fe0b8141ad0 2020-07-18T03:33:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.385+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00fa30 con 0x7fe0b8141ad0 2020-07-18T03:33:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.385+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b8156030 con 0x7fe0b8141ad0 2020-07-18T03:33:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.385+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ac00fd90 con 0x7fe0b8141ad0 2020-07-18T03:33:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.385+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b8148140 con 0x7fe0b8141ad0 2020-07-18T03:33:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.385+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b80b4d70 con 0x7fe0b8141ad0 2020-07-18T03:33:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.386+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe0ac01b030 con 0x7fe0b8141ad0 2020-07-18T03:33:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.386+0000 7fe09effd700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe094050360 0x7fe094052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.386+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe0ac067440 con 0x7fe0b8141ad0 2020-07-18T03:33:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.386+0000 7fe0b659c700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe094050360 0x7fe094052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.387+0000 7fe0b659c700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe094050360 0x7fe094052820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fe0a8008000 tx=0x7fe0a8008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.390+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe0ac03fb50 con 0x7fe0b8141ad0 2020-07-18T03:33:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.550+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0b80afbd0 con 0x7fe0b8141ad0 2020-07-18T03:33:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.552+0000 7fe09effd700 1 -- 172.21.15.47:0/1959065680 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16290 (secure 0 0 0) 0x7fe0b80afbd0 con 0x7fe0b8141ad0 2020-07-18T03:33:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:00.553 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":64,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:32:59.939942+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:32:57.493054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-07-18T03:32:58.764476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:33:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.552+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe094050360 msgr2=0x7fe094052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.552+0000 7fe0bd195700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe094050360 0x7fe094052820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fe0a8008000 tx=0x7fe0a8008040).stop 2020-07-18T03:33:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.552+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b8141ad0 msgr2=0x7fe0b8154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.552+0000 7fe0bd195700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b8141ad0 0x7fe0b8154bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe0ac00d000 tx=0x7fe0b80b5d00).stop 2020-07-18T03:33:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 shutdown_connections 2020-07-18T03:33:00.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe094050360 0x7fe094052820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 --2- 172.21.15.47:0/1959065680 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe0b8141ad0 0x7fe0b8154bc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 --2- 172.21.15.47:0/1959065680 >> v2:172.21.15.18:3300/0 conn(0x7fe0b8141eb0 0x7fe0b81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 >> 172.21.15.47:0/1959065680 conn(0x7fe0b80b9340 msgr2=0x7fe0b80b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 shutdown_connections 2020-07-18T03:33:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.553+0000 7fe0bd195700 1 -- 172.21.15.47:0/1959065680 wait complete. 2020-07-18T03:33:00.565 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-07-18T03:33:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 Processor -- start 2020-07-18T03:33:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 -- start start 2020-07-18T03:33:00.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 0x7fc13c143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc13c0b59b0 0x7fc13c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c147de0 con 0x7fc13c0b6300 2020-07-18T03:33:00.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13c0ae000 con 0x7fc13c0b59b0 2020-07-18T03:33:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.682+0000 7fc144029700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c148010 con 0x7fc13c0ad330 2020-07-18T03:33:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc141dc5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 0x7fc13c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1415c4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc13c0b59b0 0x7fc13c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc141dc5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 0x7fc13c143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55216/0 (socket says 172.21.15.47:55216) 2020-07-18T03:33:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc141dc5700 1 -- 172.21.15.47:0/3289938933 learned_addr learned my addr 172.21.15.47:0/3289938933 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1415c4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc13c0b59b0 0x7fc13c0b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:48298/0 (socket says 172.21.15.47:48298) 2020-07-18T03:33:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1425c6700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fc13c0b6300 0x7fc13c0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34240/0 (socket says 172.21.15.47:34240) 2020-07-18T03:33:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1415c4700 1 -- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 msgr2=0x7fc13c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1415c4700 1 --2- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 0x7fc13c143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1415c4700 1 -- 172.21.15.47:0/3289938933 >> v1:172.21.15.18:6789/0 conn(0x7fc13c0b6300 legacy=0x7fc13c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc140dc3700 1 -- 172.21.15.47:0/3289938933 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915954298 0 0) 0x7fc13c147de0 con 0x7fc13c0b6300 2020-07-18T03:33:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc1415c4700 1 -- 172.21.15.47:0/3289938933 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c0ad710 con 0x7fc13c0b59b0 2020-07-18T03:33:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.683+0000 7fc141dc5700 1 --2- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 0x7fc13c143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc1415c4700 1 --2- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0b59b0 0x7fc13c0b5e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc1340032b0 tx=0x7fc1340091d0).ready entity=mon.0 client_cookie=6a7ff470c313ef81 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc140dc3700 1 -- 172.21.15.47:0/3289938933 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13400b9a0 con 0x7fc13c0b59b0 2020-07-18T03:33:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc140dc3700 1 -- 172.21.15.47:0/3289938933 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13400bb00 con 0x7fc13c0b59b0 2020-07-18T03:33:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc140dc3700 1 -- 172.21.15.47:0/3289938933 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13400bc70 con 0x7fc13c0b59b0 2020-07-18T03:33:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 -- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0b59b0 msgr2=0x7fc13c0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 --2- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0b59b0 0x7fc13c0b5e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc1340032b0 tx=0x7fc1340091d0).stop 2020-07-18T03:33:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 -- 172.21.15.47:0/3289938933 shutdown_connections 2020-07-18T03:33:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 --2- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c0ad330 0x7fc13c143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 --2- 172.21.15.47:0/3289938933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0b59b0 0x7fc13c0b5e10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 -- 172.21.15.47:0/3289938933 >> 172.21.15.47:0/3289938933 conn(0x7fc13c05f5a0 msgr2=0x7fc13c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 -- 172.21.15.47:0/3289938933 shutdown_connections 2020-07-18T03:33:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.684+0000 7fc144029700 1 -- 172.21.15.47:0/3289938933 wait complete. 2020-07-18T03:33:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.685+0000 7fc144029700 1 Processor -- start 2020-07-18T03:33:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.685+0000 7fc144029700 1 -- start start 2020-07-18T03:33:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc144029700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc13c0ad330 0x7fc13c064700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc144029700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 0x7fc13c139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc144029700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c1475a0 con 0x7fc13c0b6300 2020-07-18T03:33:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc144029700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13c0668a0 con 0x7fc13c0ad330 2020-07-18T03:33:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc144029700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c066b40 con 0x7fc13c146d40 2020-07-18T03:33:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc1425c6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 0x7fc13c139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc141dc5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc13c0ad330 0x7fc13c064700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc1425c6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 0x7fc13c139be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55226/0 (socket says 172.21.15.47:55226) 2020-07-18T03:33:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc141dc5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc13c0ad330 0x7fc13c064700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:48304/0 (socket says 172.21.15.47:48304) 2020-07-18T03:33:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.686+0000 7fc1425c6700 1 -- 172.21.15.47:0/4113280768 learned_addr learned my addr 172.21.15.47:0/4113280768 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc141dc5700 1 -- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 msgr2=0x7fc13c139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc141dc5700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 0x7fc13c139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc141dc5700 1 -- 172.21.15.47:0/4113280768 >> v1:172.21.15.18:6789/0 conn(0x7fc13c0b6300 legacy=0x7fc13c066040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90797115 0 0) 0x7fc13c1475a0 con 0x7fc13c0b6300 2020-07-18T03:33:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc141dc5700 1 -- 172.21.15.47:0/4113280768 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c063850 con 0x7fc13c0ad330 2020-07-18T03:33:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc1425c6700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 0x7fc13c139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc141dc5700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0ad330 0x7fc13c064700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fc13800c000 tx=0x7fc138006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13800ca00 con 0x7fc13c0ad330 2020-07-18T03:33:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13800cb60 con 0x7fc13c0ad330 2020-07-18T03:33:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c063c80 con 0x7fc13c0ad330 2020-07-18T03:33:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1380123f0 con 0x7fc13c0ad330 2020-07-18T03:33:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.687+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c13a290 con 0x7fc13c0ad330 2020-07-18T03:33:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.688+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c05b120 con 0x7fc13c0ad330 2020-07-18T03:33:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.688+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc1380125f0 con 0x7fc13c0ad330 2020-07-18T03:33:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.689+0000 7fc129ffb700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc11404fd80 0x7fc114052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.689+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc138068570 con 0x7fc13c0ad330 2020-07-18T03:33:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.689+0000 7fc1415c4700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc11404fd80 0x7fc114052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.689+0000 7fc1415c4700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc11404fd80 0x7fc114052240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fc134003270 tx=0x7fc134016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.693+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc138040b80 con 0x7fc13c0ad330 2020-07-18T03:33:00.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.902+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc13c0690d0 con 0x7fc13c0ad330 2020-07-18T03:33:00.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.943+0000 7fc129ffb700 1 -- 172.21.15.47:0/4113280768 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc1380441a0 con 0x7fc13c0ad330 2020-07-18T03:33:00.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc11404fd80 msgr2=0x7fc114052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc11404fd80 0x7fc114052240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fc134003270 tx=0x7fc134016000).stop 2020-07-18T03:33:00.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0ad330 msgr2=0x7fc13c064700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:00.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0ad330 0x7fc13c064700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fc13800c000 tx=0x7fc138006e90).stop 2020-07-18T03:33:00.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 shutdown_connections 2020-07-18T03:33:00.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc11404fd80 0x7fc114052240 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc13c146d40 0x7fc13c139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:00.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 --2- 172.21.15.47:0/4113280768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc13c0ad330 0x7fc13c064700 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0x7fc13800c000 tx=0x7fc138006e90).stop 2020-07-18T03:33:00.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 >> 172.21.15.47:0/4113280768 conn(0x7fc13c05f5a0 msgr2=0x7fc13c1477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:00.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.945+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 shutdown_connections 2020-07-18T03:33:00.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:00.946+0000 7fc144029700 1 -- 172.21.15.47:0/4113280768 wait complete. 2020-07-18T03:33:00.958 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 Processor -- start 2020-07-18T03:33:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 -- start start 2020-07-18T03:33:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6e4147a00 0x7fc6e4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e41433e0 0x7fc6e4143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e4147330 con 0x7fc6e40ad100 2020-07-18T03:33:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e40ae530 con 0x7fc6e41433e0 2020-07-18T03:33:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e9560700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e40b57e0 con 0x7fc6e4147a00 2020-07-18T03:33:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.073+0000 7fc6e37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e41433e0 0x7fc6e4143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e41433e0 0x7fc6e4143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34400/0 (socket says 172.21.15.47:34400) 2020-07-18T03:33:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e2ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc6e40ad100 0x7fc6e40b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43720/0 (socket says 172.21.15.47:43720) 2020-07-18T03:33:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 -- 172.21.15.47:0/1069794624 learned_addr learned my addr 172.21.15.47:0/1069794624 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e27fc700 1 --2- 172.21.15.47:0/1069794624 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6e4147a00 0x7fc6e4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 -- 172.21.15.47:0/1069794624 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6e4147a00 msgr2=0x7fc6e4147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 --2- 172.21.15.47:0/1069794624 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6e4147a00 0x7fc6e4147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 -- 172.21.15.47:0/1069794624 >> v1:172.21.15.47:6789/0 conn(0x7fc6e40ad100 legacy=0x7fc6e40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e1ffb700 1 -- 172.21.15.47:0/1069794624 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898002239 0 0) 0x7fc6e4147330 con 0x7fc6e40ad100 2020-07-18T03:33:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 -- 172.21.15.47:0/1069794624 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e40b5a80 con 0x7fc6e41433e0 2020-07-18T03:33:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e37fe700 1 --2- 172.21.15.47:0/1069794624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e41433e0 0x7fc6e4143850 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fc6dc0032b0 tx=0x7fc6dc009270).ready entity=mon.1 client_cookie=b8b37ea3bdb3520e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e1ffb700 1 -- 172.21.15.47:0/1069794624 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6dc00bc00 con 0x7fc6e41433e0 2020-07-18T03:33:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e1ffb700 1 -- 172.21.15.47:0/1069794624 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc00bd60 con 0x7fc6e41433e0 2020-07-18T03:33:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e1ffb700 1 -- 172.21.15.47:0/1069794624 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6dc011400 con 0x7fc6e41433e0 2020-07-18T03:33:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 -- 172.21.15.47:0/1069794624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e41433e0 msgr2=0x7fc6e4143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 --2- 172.21.15.47:0/1069794624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e41433e0 0x7fc6e4143850 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fc6dc0032b0 tx=0x7fc6dc009270).stop 2020-07-18T03:33:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 -- 172.21.15.47:0/1069794624 shutdown_connections 2020-07-18T03:33:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 --2- 172.21.15.47:0/1069794624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e41433e0 0x7fc6e4143850 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 --2- 172.21.15.47:0/1069794624 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6e4147a00 0x7fc6e4147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 -- 172.21.15.47:0/1069794624 >> 172.21.15.47:0/1069794624 conn(0x7fc6e40b9340 msgr2=0x7fc6e40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 -- 172.21.15.47:0/1069794624 shutdown_connections 2020-07-18T03:33:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.074+0000 7fc6e9560700 1 -- 172.21.15.47:0/1069794624 wait complete. 2020-07-18T03:33:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.076+0000 7fc6e9560700 1 Processor -- start 2020-07-18T03:33:01.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.076+0000 7fc6e9560700 1 -- start start 2020-07-18T03:33:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e9560700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e40ad100 0x7fc6e414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e9560700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc6e40b3010 0x7fc6e4139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e9560700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e414d580 con 0x7fc6e4147a00 2020-07-18T03:33:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e9560700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e414d820 con 0x7fc6e40b3010 2020-07-18T03:33:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e9560700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e414dac0 con 0x7fc6e40ad100 2020-07-18T03:33:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e40ad100 0x7fc6e414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e40ad100 0x7fc6e414cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34404/0 (socket says 172.21.15.47:34404) 2020-07-18T03:33:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 -- 172.21.15.47:0/3757208458 learned_addr learned my addr 172.21.15.47:0/3757208458 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e37fe700 1 --2- 172.21.15.47:0/3757208458 >> v2:172.21.15.18:3300/0 conn(0x7fc6e40b3010 0x7fc6e4139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 -- 172.21.15.47:0/3757208458 >> v2:172.21.15.18:3300/0 conn(0x7fc6e40b3010 msgr2=0x7fc6e4139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 --2- 172.21.15.47:0/3757208458 >> v2:172.21.15.18:3300/0 conn(0x7fc6e40b3010 0x7fc6e4139c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 -- 172.21.15.47:0/3757208458 >> v1:172.21.15.18:6789/0 conn(0x7fc6e4147a00 legacy=0x7fc6e4066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 -- 172.21.15.47:0/3757208458 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e414d220 con 0x7fc6e40ad100 2020-07-18T03:33:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e2ffd700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e40ad100 0x7fc6e414cd30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fc6d4000e90 tx=0x7fc6d4007a10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d40072e0 con 0x7fc6e40ad100 2020-07-18T03:33:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d4005530 con 0x7fc6e40ad100 2020-07-18T03:33:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e413a180 con 0x7fc6e40ad100 2020-07-18T03:33:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.077+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d4005890 con 0x7fc6e40ad100 2020-07-18T03:33:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.078+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e414b9c0 con 0x7fc6e40ad100 2020-07-18T03:33:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.078+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e4057450 con 0x7fc6e40ad100 2020-07-18T03:33:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.079+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc6d401a060 con 0x7fc6e40ad100 2020-07-18T03:33:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.079+0000 7fc6daffd700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc6c0050040 0x7fc6c0052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.079+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc6d40674b0 con 0x7fc6e40ad100 2020-07-18T03:33:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.079+0000 7fc6e27fc700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc6c0050040 0x7fc6c0052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.079+0000 7fc6e27fc700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc6c0050040 0x7fc6c0052500 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc6e414a830 tx=0x7fc6cc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.082+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc6d403fc40 con 0x7fc6e40ad100 2020-07-18T03:33:01.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2020-07-18T03:33:01.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.291+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6e40ae990 con 0x7fc6e40ad100 2020-07-18T03:33:01.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.292+0000 7fc6daffd700 1 -- 172.21.15.47:0/3757208458 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2133 (secure 0 0 0) 0x7fc6d4018020 con 0x7fc6e40ad100 2020-07-18T03:33:01.294 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:01.294 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/848402287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":848402287},{"type":"v1","addr":"172.21.15.18:6833","nonce":848402287}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:00.941352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2273595707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2273595707},{"type":"v1","addr":"172.21.15.47:6835","nonce":2273595707}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc6c0050040 msgr2=0x7fc6c0052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc6c0050040 0x7fc6c0052500 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc6e414a830 tx=0x7fc6cc008040).stop 2020-07-18T03:33:01.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e40ad100 msgr2=0x7fc6e414cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e40ad100 0x7fc6e414cd30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fc6d4000e90 tx=0x7fc6d4007a10).stop 2020-07-18T03:33:01.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 shutdown_connections 2020-07-18T03:33:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc6c0050040 0x7fc6c0052500 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 --2- 172.21.15.47:0/3757208458 >> v2:172.21.15.18:3300/0 conn(0x7fc6e40b3010 0x7fc6e4139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 --2- 172.21.15.47:0/3757208458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6e40ad100 0x7fc6e414cd30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 >> 172.21.15.47:0/3757208458 conn(0x7fc6e40b9340 msgr2=0x7fc6e41469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:01.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 shutdown_connections 2020-07-18T03:33:01.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.293+0000 7fc6e9560700 1 -- 172.21.15.47:0/3757208458 wait complete. 2020-07-18T03:33:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2020-07-18T03:33:01.305 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:33:01.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.420+0000 7f1d99ef7700 1 Processor -- start 2020-07-18T03:33:01.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.420+0000 7f1d99ef7700 1 -- start start 2020-07-18T03:33:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d99ef7700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1d940ade90 0x7f1d94142f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d99ef7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b0950 0x7f1d94147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d99ef7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d94143400 con 0x7f1d940b04c0 2020-07-18T03:33:01.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d99ef7700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940afbd0 con 0x7f1d940ade90 2020-07-18T03:33:01.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d99ef7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940afe70 con 0x7f1d940b0950 2020-07-18T03:33:01.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d93fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d940b04c0 0x7f1d94146b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43732/0 (socket says 172.21.15.47:43732) 2020-07-18T03:33:01.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d93fff700 1 -- 172.21.15.47:0/330814371 learned_addr learned my addr 172.21.15.47:0/330814371 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:01.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d996f6700 1 --2- 172.21.15.47:0/330814371 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b0950 0x7f1d94147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d98ef5700 1 --2- 172.21.15.47:0/330814371 >> v2:172.21.15.18:3300/0 conn(0x7f1d940ade90 0x7f1d94142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d937fe700 1 -- 172.21.15.47:0/330814371 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104519642 0 0) 0x7f1d94143400 con 0x7f1d940b04c0 2020-07-18T03:33:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d937fe700 1 -- 172.21.15.47:0/330814371 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d78003240 con 0x7f1d940b04c0 2020-07-18T03:33:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d937fe700 1 -- 172.21.15.47:0/330814371 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2529024390 0 0) 0x7f1d78003240 con 0x7f1d940b04c0 2020-07-18T03:33:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d996f6700 1 -- 172.21.15.47:0/330814371 >> v2:172.21.15.18:3300/0 conn(0x7f1d940ade90 msgr2=0x7f1d94142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d996f6700 1 --2- 172.21.15.47:0/330814371 >> v2:172.21.15.18:3300/0 conn(0x7f1d940ade90 0x7f1d94142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d996f6700 1 -- 172.21.15.47:0/330814371 >> v1:172.21.15.47:6789/0 conn(0x7f1d940b04c0 legacy=0x7f1d94146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d996f6700 1 -- 172.21.15.47:0/330814371 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940ad9d0 con 0x7f1d940b0950 2020-07-18T03:33:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.421+0000 7f1d98ef5700 1 --2- 172.21.15.47:0/330814371 >> v2:172.21.15.18:3300/0 conn(0x7f1d940ade90 0x7f1d94142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d996f6700 1 --2- 172.21.15.47:0/330814371 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b0950 0x7f1d94147220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1d940b0110 tx=0x7f1d8c0091d0).ready entity=mon.2 client_cookie=52f8105cec15ad96 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d937fe700 1 -- 172.21.15.47:0/330814371 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d800016d0 con 0x7f1d940b0950 2020-07-18T03:33:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d937fe700 1 -- 172.21.15.47:0/330814371 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00b7e0 con 0x7f1d940b0950 2020-07-18T03:33:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d937fe700 1 -- 172.21.15.47:0/330814371 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c00b950 con 0x7f1d940b0950 2020-07-18T03:33:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 -- 172.21.15.47:0/330814371 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b0950 msgr2=0x7f1d94147220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/330814371 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b0950 0x7f1d94147220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1d940b0110 tx=0x7f1d8c0091d0).stop 2020-07-18T03:33:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 -- 172.21.15.47:0/330814371 shutdown_connections 2020-07-18T03:33:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/330814371 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b0950 0x7f1d94147220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/330814371 >> v2:172.21.15.18:3300/0 conn(0x7f1d940ade90 0x7f1d94142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 -- 172.21.15.47:0/330814371 >> 172.21.15.47:0/330814371 conn(0x7f1d940b9e50 msgr2=0x7f1d940b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 -- 172.21.15.47:0/330814371 shutdown_connections 2020-07-18T03:33:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.422+0000 7f1d99ef7700 1 -- 172.21.15.47:0/330814371 wait complete. 2020-07-18T03:33:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.423+0000 7f1d99ef7700 1 Processor -- start 2020-07-18T03:33:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d99ef7700 1 -- start start 2020-07-18T03:33:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d99ef7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b04c0 0x7f1d94064f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d99ef7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d940666a0 0x7f1d9413bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d99ef7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9413d720 con 0x7f1d940ade90 2020-07-18T03:33:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d99ef7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9414d370 con 0x7f1d940666a0 2020-07-18T03:33:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d99ef7700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9414d610 con 0x7f1d940b04c0 2020-07-18T03:33:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d98ef5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d940ade90 0x7f1d94147290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43738/0 (socket says 172.21.15.47:43738) 2020-07-18T03:33:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d98ef5700 1 -- 172.21.15.47:0/3930215138 learned_addr learned my addr 172.21.15.47:0/3930215138 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d996f6700 1 --2- 172.21.15.47:0/3930215138 >> v2:172.21.15.47:3300/0 conn(0x7f1d940666a0 0x7f1d9413bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d93fff700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b04c0 0x7f1d94064f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.424+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630727464 0 0) 0x7f1d9413d720 con 0x7f1d940ade90 2020-07-18T03:33:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d6c003240 con 0x7f1d940ade90 2020-07-18T03:33:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d996f6700 1 -- 172.21.15.47:0/3930215138 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b04c0 msgr2=0x7f1d94064f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d996f6700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b04c0 0x7f1d94064f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d996f6700 1 -- 172.21.15.47:0/3930215138 >> v1:172.21.15.47:6789/0 conn(0x7f1d940ade90 legacy=0x7f1d94147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d996f6700 1 -- 172.21.15.47:0/3930215138 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9413d3e0 con 0x7f1d940666a0 2020-07-18T03:33:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d996f6700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940666a0 0x7f1d9413bf10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f1d940b0110 tx=0x7f1d8c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c021450 con 0x7f1d940666a0 2020-07-18T03:33:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c019070 con 0x7f1d940666a0 2020-07-18T03:33:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9413c400 con 0x7f1d940666a0 2020-07-18T03:33:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c0217a0 con 0x7f1d940666a0 2020-07-18T03:33:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.425+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d94061ad0 con 0x7f1d940666a0 2020-07-18T03:33:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.426+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d94057450 con 0x7f1d940666a0 2020-07-18T03:33:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.426+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1d8c01e070 con 0x7f1d940666a0 2020-07-18T03:33:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.426+0000 7f1d90ff9700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6c0503b0 0x7f1d6c052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.426+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d8c0688a0 con 0x7f1d940666a0 2020-07-18T03:33:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.426+0000 7f1d98ef5700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6c0503b0 0x7f1d6c052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.427+0000 7f1d98ef5700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6c0503b0 0x7f1d6c052870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1d8800a000 tx=0x7f1d88008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.430+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1d8c041030 con 0x7f1d940666a0 2020-07-18T03:33:01.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.591+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d9414d780 con 0x7f1d940666a0 2020-07-18T03:33:01.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.592+0000 7f1d90ff9700 1 -- 172.21.15.47:0/3930215138 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16290 (secure 0 0 0) 0x7f1d8c01b090 con 0x7f1d940666a0 2020-07-18T03:33:01.593 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:01.593 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":65,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:33:00.941659+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:32:57.493054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-07-18T03:32:58.764476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:33:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6c0503b0 msgr2=0x7f1d6c052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6c0503b0 0x7f1d6c052870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1d8800a000 tx=0x7f1d88008040).stop 2020-07-18T03:33:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940666a0 msgr2=0x7f1d9413bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940666a0 0x7f1d9413bf10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f1d940b0110 tx=0x7f1d8c016000).stop 2020-07-18T03:33:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 shutdown_connections 2020-07-18T03:33:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940666a0 0x7f1d9413bf10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6c0503b0 0x7f1d6c052870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 --2- 172.21.15.47:0/3930215138 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1d940b04c0 0x7f1d94064f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 >> 172.21.15.47:0/3930215138 conn(0x7f1d940b9e50 msgr2=0x7f1d940b8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 shutdown_connections 2020-07-18T03:33:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.593+0000 7f1d99ef7700 1 -- 172.21.15.47:0/3930215138 wait complete. 2020-07-18T03:33:01.605 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:33:01.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.725+0000 7f75dff3a700 1 Processor -- start 2020-07-18T03:33:01.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.725+0000 7f75dff3a700 1 -- start start 2020-07-18T03:33:01.729 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75dff3a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f75d8153ad0 0x7f75d81575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75dff3a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f75d8153f80 0x7f75d8157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75dff3a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d8066c20 con 0x7f75d8080510 2020-07-18T03:33:01.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75dff3a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d8080bb0 con 0x7f75d8153ad0 2020-07-18T03:33:01.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75dff3a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8157140 con 0x7f75d8153f80 2020-07-18T03:33:01.730 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75ddcd6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f75d8153ad0 0x7f75d81575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75decd8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f75d8153f80 0x7f75d8157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75ddcd6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f75d8153ad0 0x7f75d81575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:48338/0 (socket says 172.21.15.47:48338) 2020-07-18T03:33:01.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75ddcd6700 1 -- 172.21.15.47:0/2095226200 learned_addr learned my addr 172.21.15.47:0/2095226200 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:01.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75decd8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f75d8153f80 0x7f75d8157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55260/0 (socket says 172.21.15.47:55260) 2020-07-18T03:33:01.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.726+0000 7f75de4d7700 1 --1- 172.21.15.47:0/2095226200 >> v1:172.21.15.18:6789/0 conn(0x7f75d8080510 0x7f75d80b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34280/0 (socket says 172.21.15.47:34280) 2020-07-18T03:33:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75ddcd6700 1 -- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f75d8153f80 msgr2=0x7f75d8157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75ddcd6700 1 --2- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f75d8153f80 0x7f75d8157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75ddcd6700 1 -- 172.21.15.47:0/2095226200 >> v1:172.21.15.18:6789/0 conn(0x7f75d8080510 legacy=0x7f75d80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75ddcd6700 1 -- 172.21.15.47:0/2095226200 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80808f0 con 0x7f75d8153ad0 2020-07-18T03:33:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75ddcd6700 1 --2- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d8153ad0 0x7f75d81575e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f75c8002f30 tx=0x7f75c8006e90).ready entity=mon.0 client_cookie=45e20f6ae13d4290 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dd4d5700 1 -- 172.21.15.47:0/2095226200 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c8009850 con 0x7f75d8153ad0 2020-07-18T03:33:01.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dd4d5700 1 -- 172.21.15.47:0/2095226200 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c80099b0 con 0x7f75d8153ad0 2020-07-18T03:33:01.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dd4d5700 1 -- 172.21.15.47:0/2095226200 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c8009d10 con 0x7f75d8153ad0 2020-07-18T03:33:01.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 -- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d8153ad0 msgr2=0x7f75d81575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.733 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 --2- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d8153ad0 0x7f75d81575e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f75c8002f30 tx=0x7f75c8006e90).stop 2020-07-18T03:33:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 -- 172.21.15.47:0/2095226200 shutdown_connections 2020-07-18T03:33:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 --2- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f75d8153f80 0x7f75d8157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 --2- 172.21.15.47:0/2095226200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d8153ad0 0x7f75d81575e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 -- 172.21.15.47:0/2095226200 >> 172.21.15.47:0/2095226200 conn(0x7f75d805c620 msgr2=0x7f75d80b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 -- 172.21.15.47:0/2095226200 shutdown_connections 2020-07-18T03:33:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.727+0000 7f75dff3a700 1 -- 172.21.15.47:0/2095226200 wait complete. 2020-07-18T03:33:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.729+0000 7f75dff3a700 1 Processor -- start 2020-07-18T03:33:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.729+0000 7f75dff3a700 1 -- start start 2020-07-18T03:33:01.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.729+0000 7f75dff3a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f75d8080510 0x7f75d8129430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.729+0000 7f75dff3a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d8158050 con 0x7f75d8153f80 2020-07-18T03:33:01.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.729+0000 7f75dff3a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d812d270 con 0x7f75d812ce60 2020-07-18T03:33:01.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.729+0000 7f75dff3a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d812d500 con 0x7f75d8080510 2020-07-18T03:33:01.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75de4d7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f75d8080510 0x7f75d8129430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75decd8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f75d812ce60 0x7f75d8127a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43754/0 (socket says 172.21.15.47:43754) 2020-07-18T03:33:01.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75decd8700 1 -- 172.21.15.47:0/883183765 learned_addr learned my addr 172.21.15.47:0/883183765 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:01.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101452690 0 0) 0x7f75d8158050 con 0x7f75d8153f80 2020-07-18T03:33:01.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75de4d7700 1 -- 172.21.15.47:0/883183765 >> v1:172.21.15.47:6789/0 conn(0x7f75d812ce60 legacy=0x7f75d8127a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75de4d7700 1 -- 172.21.15.47:0/883183765 >> v1:172.21.15.18:6789/0 conn(0x7f75d8153f80 legacy=0x7f75d8158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75de4d7700 1 -- 172.21.15.47:0/883183765 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8158280 con 0x7f75d8080510 2020-07-18T03:33:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75de4d7700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d8080510 0x7f75d8129430 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f75d000a000 tx=0x7f75d4007b20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d40072e0 con 0x7f75d8080510 2020-07-18T03:33:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d4005530 con 0x7f75d8080510 2020-07-18T03:33:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d4005890 con 0x7f75d8080510 2020-07-18T03:33:01.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d8124860 con 0x7f75d8080510 2020-07-18T03:33:01.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.730+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d8124d30 con 0x7f75d8080510 2020-07-18T03:33:01.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.731+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f75d401a060 con 0x7f75d8080510 2020-07-18T03:33:01.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.731+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d812d640 con 0x7f75d8080510 2020-07-18T03:33:01.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.732+0000 7f75cdffb700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f75b804fe20 0x7f75b80522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:01.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.732+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f75d4067330 con 0x7f75d8080510 2020-07-18T03:33:01.742 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.732+0000 7f75ddcd6700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f75b804fe20 0x7f75b80522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:01.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.732+0000 7f75ddcd6700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f75b804fe20 0x7f75b80522e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f75c8006b60 tx=0x7f75c8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.735+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f75d403fa40 con 0x7f75d8080510 2020-07-18T03:33:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.956+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f75d8158410 con 0x7f75d8080510 2020-07-18T03:33:01.961 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.960+0000 7f75cdffb700 1 -- 172.21.15.47:0/883183765 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f75d4018070 con 0x7f75d8080510 2020-07-18T03:33:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f75b804fe20 msgr2=0x7f75b80522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f75b804fe20 0x7f75b80522e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f75c8006b60 tx=0x7f75c8015000).stop 2020-07-18T03:33:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d8080510 msgr2=0x7f75d8129430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d8080510 0x7f75d8129430 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f75d000a000 tx=0x7f75d4007b20).stop 2020-07-18T03:33:01.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 shutdown_connections 2020-07-18T03:33:01.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f75b804fe20 0x7f75b80522e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 --2- 172.21.15.47:0/883183765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d8080510 0x7f75d8129430 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:01.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 >> 172.21.15.47:0/883183765 conn(0x7f75d805c620 msgr2=0x7f75d8060030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 shutdown_connections 2020-07-18T03:33:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:01.961+0000 7f75dff3a700 1 -- 172.21.15.47:0/883183765 wait complete. 2020-07-18T03:33:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2020-07-18T03:33:01.973 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:33:02.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.094+0000 7f69ffe6f700 1 Processor -- start 2020-07-18T03:33:02.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.094+0000 7f69ffe6f700 1 -- start start 2020-07-18T03:33:02.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69ffe6f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f69f8157cf0 0x7f69f8063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.100 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69ffe6f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f8080bb0 con 0x7f69f8080510 2020-07-18T03:33:02.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69ffe6f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f8157140 con 0x7f69f8153ad0 2020-07-18T03:33:02.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69ffe6f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f80808f0 con 0x7f69f8157cf0 2020-07-18T03:33:02.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69fdc0b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f69f8153ad0 0x7f69f81575e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43760/0 (socket says 172.21.15.47:43760) 2020-07-18T03:33:02.101 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69fdc0b700 1 -- 172.21.15.47:0/4085927642 learned_addr learned my addr 172.21.15.47:0/4085927642 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:02.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69fec0d700 1 --2- 172.21.15.47:0/4085927642 >> v2:172.21.15.18:3300/0 conn(0x7f69f8157cf0 0x7f69f8063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203329891 0 0) 0x7f69f8157140 con 0x7f69f8153ad0 2020-07-18T03:33:02.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.095+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e4003240 con 0x7f69f8153ad0 2020-07-18T03:33:02.102 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1850213210 0 0) 0x7f69e4003240 con 0x7f69f8153ad0 2020-07-18T03:33:02.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 >> v2:172.21.15.18:3300/0 conn(0x7f69f8157cf0 msgr2=0x7f69f8063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 --2- 172.21.15.47:0/4085927642 >> v2:172.21.15.18:3300/0 conn(0x7f69f8157cf0 0x7f69f8063450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 >> v1:172.21.15.18:6789/0 conn(0x7f69f8080510 legacy=0x7f69f80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8080e50 con 0x7f69f8153ad0 2020-07-18T03:33:02.103 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f69e80016d0 con 0x7f69f8153ad0 2020-07-18T03:33:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 >> v1:172.21.15.47:6789/0 conn(0x7f69f8153ad0 legacy=0x7f69f81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e4003730 0x7f69e4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69e40044e0 0x7f69e4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4004e50 0x7f69e400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80808f0 con 0x7f69e4004e50 2020-07-18T03:33:02.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4004320 con 0x7f69e4003730 2020-07-18T03:33:02.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e404fc80 con 0x7f69e40044e0 2020-07-18T03:33:02.105 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e4003730 0x7f69e4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.106 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fec0d700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4004e50 0x7f69e400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fdc0b700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69e40044e0 0x7f69e4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 -- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69e40044e0 msgr2=0x7f69e4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69e40044e0 0x7f69e4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 -- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4004e50 msgr2=0x7f69e400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.107 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4004e50 0x7f69e400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 -- 172.21.15.47:0/4085927642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4003240 con 0x7f69e4003730 2020-07-18T03:33:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fec0d700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4004e50 0x7f69e400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fec0d700 1 -- 172.21.15.47:0/4085927642 reap_dead start 2020-07-18T03:33:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fe40c700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e4003730 0x7f69e4003b50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f69e40054b0 tx=0x7f69f4007b70).ready entity=mon.1 client_cookie=fa90a8c2743022e0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f400c830 con 0x7f69e4003730 2020-07-18T03:33:02.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400c990 con 0x7f69e4003730 2020-07-18T03:33:02.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69fd40a700 1 -- 172.21.15.47:0/4085927642 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f400ccf0 con 0x7f69e4003730 2020-07-18T03:33:02.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69ffe6f700 1 -- 172.21.15.47:0/4085927642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e4003730 msgr2=0x7f69e4003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.109 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.096+0000 7f69ffe6f700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e4003730 0x7f69e4003b50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f69e40054b0 tx=0x7f69f4007b70).stop 2020-07-18T03:33:02.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.097+0000 7f69ffe6f700 1 -- 172.21.15.47:0/4085927642 shutdown_connections 2020-07-18T03:33:02.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.097+0000 7f69ffe6f700 1 --2- 172.21.15.47:0/4085927642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e4003730 0x7f69e4003b50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.097+0000 7f69ffe6f700 1 -- 172.21.15.47:0/4085927642 >> 172.21.15.47:0/4085927642 conn(0x7f69f805c620 msgr2=0x7f69f80b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:02.110 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.097+0000 7f69ffe6f700 1 -- 172.21.15.47:0/4085927642 shutdown_connections 2020-07-18T03:33:02.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.097+0000 7f69ffe6f700 1 -- 172.21.15.47:0/4085927642 wait complete. 2020-07-18T03:33:02.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.098+0000 7f69ffe6f700 1 Processor -- start 2020-07-18T03:33:02.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.098+0000 7f69ffe6f700 1 -- start start 2020-07-18T03:33:02.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69ffe6f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 0x7f69f80b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69ffe6f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f69f80b3a10 0x7f69f80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69ffe6f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f80b1e10 con 0x7f69f80b8260 2020-07-18T03:33:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69ffe6f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f80b3620 con 0x7f69f80b3a10 2020-07-18T03:33:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69ffe6f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8141970 con 0x7f69f8155900 2020-07-18T03:33:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69fe40c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f69f80b8260 0x7f69f8141260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43772/0 (socket says 172.21.15.47:43772) 2020-07-18T03:33:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69fe40c700 1 -- 172.21.15.47:0/2956185390 learned_addr learned my addr 172.21.15.47:0/2956185390 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69fdc0b700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 0x7f69f80b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69fec0d700 1 --2- 172.21.15.47:0/2956185390 >> v2:172.21.15.18:3300/0 conn(0x7f69f80b3a10 0x7f69f80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243941571 0 0) 0x7f69f80b1e10 con 0x7f69f80b8260 2020-07-18T03:33:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.099+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d8003240 con 0x7f69f80b8260 2020-07-18T03:33:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2664138343 0 0) 0x7f69d8003240 con 0x7f69f80b8260 2020-07-18T03:33:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f69f80b1e10 con 0x7f69f80b8260 2020-07-18T03:33:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f69f400c830 con 0x7f69f80b8260 2020-07-18T03:33:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 msgr2=0x7f69f80b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 0x7f69f80b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 >> v2:172.21.15.18:3300/0 conn(0x7f69f80b3a10 msgr2=0x7f69f80b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 --2- 172.21.15.47:0/2956185390 >> v2:172.21.15.18:3300/0 conn(0x7f69f80b3a10 0x7f69f80b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 >> v1:172.21.15.47:6789/0 conn(0x7f69f80b8260 legacy=0x7f69f8141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d8003800 0x7f69d8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 0x7f69d80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69d80049d0 0x7f69d8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8141970 con 0x7f69d80049d0 2020-07-18T03:33:02.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8004210 con 0x7f69d8003800 2020-07-18T03:33:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d800b730 con 0x7f69f8155900 2020-07-18T03:33:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fec0d700 1 --2- 172.21.15.47:0/2956185390 >> v2:172.21.15.18:3300/0 conn(0x7f69f80b3a10 0x7f69f80b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d8003800 0x7f69d8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fdc0b700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 0x7f69d80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fec0d700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69d80049d0 0x7f69d8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 -- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 msgr2=0x7f69d80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f69f8155900 0x7f69d80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 -- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69d80049d0 msgr2=0x7f69d8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69d80049d0 0x7f69d8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80b20b0 con 0x7f69d8003800 2020-07-18T03:33:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f80b4bb0 con 0x7f69d8003800 2020-07-18T03:33:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69fe40c700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d8003800 0x7f69d8003c20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f69d80054b0 tx=0x7f69f4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f80b8640 con 0x7f69d8003800 2020-07-18T03:33:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f4007220 con 0x7f69d8003800 2020-07-18T03:33:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4007380 con 0x7f69d8003800 2020-07-18T03:33:02.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.100+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f40066e0 con 0x7f69d8003800 2020-07-18T03:33:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.101+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f80b8920 con 0x7f69d8003800 2020-07-18T03:33:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.101+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f69f4010030 con 0x7f69d8003800 2020-07-18T03:33:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.102+0000 7f69edffb700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69d8058170 0x7f69d805a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.102+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f69f4069a40 con 0x7f69d8003800 2020-07-18T03:33:02.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.102+0000 7f69fdc0b700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69d8058170 0x7f69d805a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.102+0000 7f69fdc0b700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69d8058170 0x7f69d805a630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f69e800d000 tx=0x7f69e8009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.105+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f69f4042210 con 0x7f69d8003800 2020-07-18T03:33:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.322+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f69f80b8920 con 0x7f69d8003800 2020-07-18T03:33:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.326+0000 7f69edffb700 1 -- 172.21.15.47:0/2956185390 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7f69f4023070 con 0x7f69d8003800 2020-07-18T03:33:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69d8058170 msgr2=0x7f69d805a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f69d8058170 0x7f69d805a630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f69e800d000 tx=0x7f69e8009040).stop 2020-07-18T03:33:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d8003800 msgr2=0x7f69d8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 --2- 172.21.15.47:0/2956185390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d8003800 0x7f69d8003c20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f69d80054b0 tx=0x7f69f4016000).stop 2020-07-18T03:33:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69fec0d700 1 -- 172.21.15.47:0/2956185390 reap_dead start 2020-07-18T03:33:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 shutdown_connections 2020-07-18T03:33:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 >> 172.21.15.47:0/2956185390 conn(0x7f69f805c620 msgr2=0x7f69f805d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 shutdown_connections 2020-07-18T03:33:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.327+0000 7f69ffe6f700 1 -- 172.21.15.47:0/2956185390 wait complete. 2020-07-18T03:33:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2020-07-18T03:33:02.339 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:33:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.461+0000 7f9711086700 1 Processor -- start 2020-07-18T03:33:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.461+0000 7f9711086700 1 -- start start 2020-07-18T03:33:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f970c080510 0x7f970c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153ad0 0x7f970c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f970c153f80 0x7f970c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c080bb0 con 0x7f970c153f80 2020-07-18T03:33:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c157140 con 0x7f970c080510 2020-07-18T03:33:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0808f0 con 0x7f970c153ad0 2020-07-18T03:33:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f970c080510 0x7f970c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f970c080510 0x7f970c0b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34466/0 (socket says 172.21.15.47:34466) 2020-07-18T03:33:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 -- 172.21.15.47:0/1329235559 learned_addr learned my addr 172.21.15.47:0/1329235559 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970ad9d700 1 --2- 172.21.15.47:0/1329235559 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153ad0 0x7f970c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9710885700 1 --2- 172.21.15.47:0/1329235559 >> v2:172.21.15.18:3300/0 conn(0x7f970c153f80 0x7f970c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 -- 172.21.15.47:0/1329235559 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153ad0 msgr2=0x7f970c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 --2- 172.21.15.47:0/1329235559 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153ad0 0x7f970c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 -- 172.21.15.47:0/1329235559 >> v2:172.21.15.18:3300/0 conn(0x7f970c153f80 msgr2=0x7f970c157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 --2- 172.21.15.47:0/1329235559 >> v2:172.21.15.18:3300/0 conn(0x7f970c153f80 0x7f970c157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 -- 172.21.15.47:0/1329235559 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c080e50 con 0x7f970c080510 2020-07-18T03:33:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970b59e700 1 --2- 172.21.15.47:0/1329235559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c080510 0x7f970c0b9aa0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f96f4002f30 tx=0x7f96f4006e90).ready entity=mon.1 client_cookie=e072f6b1357b9a87 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970a59c700 1 -- 172.21.15.47:0/1329235559 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96f4009920 con 0x7f970c080510 2020-07-18T03:33:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970a59c700 1 -- 172.21.15.47:0/1329235559 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4009a80 con 0x7f970c080510 2020-07-18T03:33:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f970a59c700 1 -- 172.21.15.47:0/1329235559 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96f4009de0 con 0x7f970c080510 2020-07-18T03:33:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- 172.21.15.47:0/1329235559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c080510 msgr2=0x7f970c0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- 172.21.15.47:0/1329235559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c080510 0x7f970c0b9aa0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f96f4002f30 tx=0x7f96f4006e90).stop 2020-07-18T03:33:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- 172.21.15.47:0/1329235559 shutdown_connections 2020-07-18T03:33:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- 172.21.15.47:0/1329235559 >> v2:172.21.15.18:3300/0 conn(0x7f970c153f80 0x7f970c157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- 172.21.15.47:0/1329235559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c080510 0x7f970c0b9aa0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 --2- 172.21.15.47:0/1329235559 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153ad0 0x7f970c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- 172.21.15.47:0/1329235559 >> 172.21.15.47:0/1329235559 conn(0x7f970c05c620 msgr2=0x7f970c0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:02.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.462+0000 7f9711086700 1 -- 172.21.15.47:0/1329235559 shutdown_connections 2020-07-18T03:33:02.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.463+0000 7f9711086700 1 -- 172.21.15.47:0/1329235559 wait complete. 2020-07-18T03:33:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.464+0000 7f9711086700 1 Processor -- start 2020-07-18T03:33:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.464+0000 7f9711086700 1 -- start start 2020-07-18T03:33:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9711086700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f970c153ad0 0x7f970c05f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9711086700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153f80 0x7f970c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9711086700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970c0b3bd0 con 0x7f970c140210 2020-07-18T03:33:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9711086700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c0b39c0 con 0x7f970c153ad0 2020-07-18T03:33:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9711086700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0b2e60 con 0x7f970c153f80 2020-07-18T03:33:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f970c153ad0 0x7f970c05f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9710885700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f970c140210 0x7f970c1406c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43792/0 (socket says 172.21.15.47:43792) 2020-07-18T03:33:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9710885700 1 -- 172.21.15.47:0/542917343 learned_addr learned my addr 172.21.15.47:0/542917343 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f970c153ad0 0x7f970c05f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34474/0 (socket says 172.21.15.47:34474) 2020-07-18T03:33:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970ad9d700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153f80 0x7f970c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970b59e700 1 -- 172.21.15.47:0/542917343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153f80 msgr2=0x7f970c0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970b59e700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153f80 0x7f970c0b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970b59e700 1 -- 172.21.15.47:0/542917343 >> v1:172.21.15.47:6789/0 conn(0x7f970c140210 legacy=0x7f970c1406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726100611 0 0) 0x7f970c0b3bd0 con 0x7f970c140210 2020-07-18T03:33:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.465+0000 7f970b59e700 1 -- 172.21.15.47:0/542917343 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c0b3e70 con 0x7f970c153ad0 2020-07-18T03:33:02.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.466+0000 7f970b59e700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c153ad0 0x7f970c05f760 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f96f4002f30 tx=0x7f96f4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.466+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96f4009ea0 con 0x7f970c153ad0 2020-07-18T03:33:02.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.466+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f401d5d0 con 0x7f970c153ad0 2020-07-18T03:33:02.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.466+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c05fc50 con 0x7f970c153ad0 2020-07-18T03:33:02.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.466+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96f401d930 con 0x7f970c153ad0 2020-07-18T03:33:02.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.466+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970c0b0700 con 0x7f970c153ad0 2020-07-18T03:33:02.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.467+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970c0b4000 con 0x7f970c153ad0 2020-07-18T03:33:02.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.467+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f96f4025030 con 0x7f970c153ad0 2020-07-18T03:33:02.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.467+0000 7f9702ffd700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f96e804fe00 0x7f96e80522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.467+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96f4069320 con 0x7f970c153ad0 2020-07-18T03:33:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.467+0000 7f970ad9d700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f96e804fe00 0x7f96e80522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.467+0000 7f970ad9d700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f96e804fe00 0x7f96e80522c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f96fc00c000 tx=0x7f96fc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.470+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f970c0b4000 con 0x7f970c153ad0 2020-07-18T03:33:02.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.685+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f970c0b4000 con 0x7f970c153ad0 2020-07-18T03:33:02.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.689+0000 7f9702ffd700 1 -- 172.21.15.47:0/542917343 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7f970c0b4000 con 0x7f970c153ad0 2020-07-18T03:33:02.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f96e804fe00 msgr2=0x7f96e80522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f96e804fe00 0x7f96e80522c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f96fc00c000 tx=0x7f96fc008040).stop 2020-07-18T03:33:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c153ad0 msgr2=0x7f970c05f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c153ad0 0x7f970c05f760 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f96f4002f30 tx=0x7f96f4015000).stop 2020-07-18T03:33:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 shutdown_connections 2020-07-18T03:33:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f96e804fe00 0x7f96e80522c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f970c153ad0 0x7f970c05f760 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 --2- 172.21.15.47:0/542917343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f970c153f80 0x7f970c0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 >> 172.21.15.47:0/542917343 conn(0x7f970c05c620 msgr2=0x7f970c062440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 shutdown_connections 2020-07-18T03:33:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.690+0000 7f9711086700 1 -- 172.21.15.47:0/542917343 wait complete. 2020-07-18T03:33:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.2 2020-07-18T03:33:02.702 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:33:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 Processor -- start 2020-07-18T03:33:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 -- start start 2020-07-18T03:33:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0490080510 0x7f04900b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0490153ad0 0x7f04901575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0490080bb0 con 0x7f0490153f80 2020-07-18T03:33:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0490157140 con 0x7f0490080510 2020-07-18T03:33:02.826 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.822+0000 7f0498070700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04900808f0 con 0x7f0490153ad0 2020-07-18T03:33:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0490153ad0 0x7f04901575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0490153ad0 0x7f04901575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34480/0 (socket says 172.21.15.47:34480) 2020-07-18T03:33:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 -- 172.21.15.47:0/31511342 learned_addr learned my addr 172.21.15.47:0/31511342 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f049660d700 1 --2- 172.21.15.47:0/31511342 >> v2:172.21.15.18:3300/0 conn(0x7f0490080510 0x7f04900b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 -- 172.21.15.47:0/31511342 >> v2:172.21.15.18:3300/0 conn(0x7f0490080510 msgr2=0x7f04900b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 --2- 172.21.15.47:0/31511342 >> v2:172.21.15.18:3300/0 conn(0x7f0490080510 0x7f04900b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 -- 172.21.15.47:0/31511342 >> v1:172.21.15.18:6789/0 conn(0x7f0490153f80 legacy=0x7f0490063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 -- 172.21.15.47:0/31511342 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490080e50 con 0x7f0490153ad0 2020-07-18T03:33:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0495e0c700 1 --2- 172.21.15.47:0/31511342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0490153ad0 0x7f04901575e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f04900b90e0 tx=0x7f0480006e90).ready entity=mon.1 client_cookie=262f9440e29e869c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f049560b700 1 -- 172.21.15.47:0/31511342 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0480009930 con 0x7f0490153ad0 2020-07-18T03:33:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f049560b700 1 -- 172.21.15.47:0/31511342 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480009a90 con 0x7f0490153ad0 2020-07-18T03:33:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f049560b700 1 -- 172.21.15.47:0/31511342 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0480009e30 con 0x7f0490153ad0 2020-07-18T03:33:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 -- 172.21.15.47:0/31511342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0490153ad0 msgr2=0x7f04901575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 --2- 172.21.15.47:0/31511342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0490153ad0 0x7f04901575e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f04900b90e0 tx=0x7f0480006e90).stop 2020-07-18T03:33:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 -- 172.21.15.47:0/31511342 shutdown_connections 2020-07-18T03:33:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 --2- 172.21.15.47:0/31511342 >> v2:172.21.15.18:3300/0 conn(0x7f0490080510 0x7f04900b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 --2- 172.21.15.47:0/31511342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0490153ad0 0x7f04901575e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 -- 172.21.15.47:0/31511342 >> 172.21.15.47:0/31511342 conn(0x7f049005c620 msgr2=0x7f04900b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 -- 172.21.15.47:0/31511342 shutdown_connections 2020-07-18T03:33:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.823+0000 7f0498070700 1 -- 172.21.15.47:0/31511342 wait complete. 2020-07-18T03:33:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.825+0000 7f0498070700 1 Processor -- start 2020-07-18T03:33:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.825+0000 7f0498070700 1 -- start start 2020-07-18T03:33:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0498070700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0490080510 0x7f0490156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0498070700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0490153f80 0x7f0490151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0498070700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04900b0070 con 0x7f04900afb60 2020-07-18T03:33:02.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0498070700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0490156a70 con 0x7f0490153f80 2020-07-18T03:33:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0498070700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04900b0310 con 0x7f0490080510 2020-07-18T03:33:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f049660d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0490080510 0x7f0490156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f049660d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0490080510 0x7f0490156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55316/0 (socket says 172.21.15.47:55316) 2020-07-18T03:33:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0495e0c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0490153f80 0x7f0490151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f049660d700 1 -- 172.21.15.47:0/1936683851 learned_addr learned my addr 172.21.15.47:0/1936683851 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:02.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0495e0c700 1 -- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0490080510 msgr2=0x7f0490156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0495e0c700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0490080510 0x7f0490156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:02.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0495e0c700 1 -- 172.21.15.47:0/1936683851 >> v1:172.21.15.18:6789/0 conn(0x7f04900afb60 legacy=0x7f049013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:02.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0495e0c700 1 -- 172.21.15.47:0/1936683851 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490139230 con 0x7f0490153f80 2020-07-18T03:33:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544061296 0 0) 0x7f04900b0070 con 0x7f04900afb60 2020-07-18T03:33:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.826+0000 7f0495e0c700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0490153f80 0x7f0490151cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f04900b90e0 tx=0x7f0480015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.827+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0480011e80 con 0x7f0490153f80 2020-07-18T03:33:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.827+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048001d5d0 con 0x7f0490153f80 2020-07-18T03:33:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.827+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04901521c0 con 0x7f0490153f80 2020-07-18T03:33:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.827+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048001d930 con 0x7f0490153f80 2020-07-18T03:33:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.827+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049013d570 con 0x7f0490153f80 2020-07-18T03:33:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.828+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049013cba0 con 0x7f0490153f80 2020-07-18T03:33:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.828+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0480025030 con 0x7f0490153f80 2020-07-18T03:33:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.828+0000 7f0485ffb700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f047004fdd0 0x7f0470052290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:02.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.829+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0480069400 con 0x7f0490153f80 2020-07-18T03:33:02.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.829+0000 7f049660d700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f047004fdd0 0x7f0470052290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:02.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.829+0000 7f049660d700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f047004fdd0 0x7f0470052290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f048c008000 tx=0x7f048c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:02.833+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f049013cba0 con 0x7f0490153f80 2020-07-18T03:33:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.048+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f049013cba0 con 0x7f0490153f80 2020-07-18T03:33:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.051+0000 7f0485ffb700 1 -- 172.21.15.47:0/1936683851 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f049013cba0 con 0x7f0490153f80 2020-07-18T03:33:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f047004fdd0 msgr2=0x7f0470052290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f047004fdd0 0x7f0470052290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f048c008000 tx=0x7f048c008040).stop 2020-07-18T03:33:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0490153f80 msgr2=0x7f0490151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0490153f80 0x7f0490151cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f04900b90e0 tx=0x7f0480015000).stop 2020-07-18T03:33:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 shutdown_connections 2020-07-18T03:33:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f047004fdd0 0x7f0470052290 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0490080510 0x7f0490156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 --2- 172.21.15.47:0/1936683851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0490153f80 0x7f0490151cd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 >> 172.21.15.47:0/1936683851 conn(0x7f049005c620 msgr2=0x7f0490063770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 shutdown_connections 2020-07-18T03:33:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.053+0000 7f0498070700 1 -- 172.21.15.47:0/1936683851 wait complete. 2020-07-18T03:33:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.3 2020-07-18T03:33:03.064 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.179+0000 7fe1b991d700 1 Processor -- start 2020-07-18T03:33:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.179+0000 7fe1b991d700 1 -- start start 2020-07-18T03:33:03.182 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.179+0000 7fe1b991d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe1b40ad100 0x7fe1b40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.179+0000 7fe1b991d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b4147330 con 0x7fe1b41443f0 2020-07-18T03:33:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.179+0000 7fe1b991d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b40603c0 con 0x7fe1b4147a00 2020-07-18T03:33:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.179+0000 7fe1b991d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b40b57e0 con 0x7fe1b40ad100 2020-07-18T03:33:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b27fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe1b4147a00 0x7fe1b405f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43814/0 (socket says 172.21.15.47:43814) 2020-07-18T03:33:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b27fc700 1 -- 172.21.15.47:0/66076444 learned_addr learned my addr 172.21.15.47:0/66076444 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b2ffd700 1 --2- 172.21.15.47:0/66076444 >> v2:172.21.15.18:3300/0 conn(0x7fe1b40ad100 0x7fe1b40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b1ffb700 1 -- 172.21.15.47:0/66076444 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219201191 0 0) 0x7fe1b40603c0 con 0x7fe1b4147a00 2020-07-18T03:33:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b1ffb700 1 -- 172.21.15.47:0/66076444 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe194003240 con 0x7fe1b4147a00 2020-07-18T03:33:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b1ffb700 1 -- 172.21.15.47:0/66076444 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2647228787 0 0) 0x7fe194003240 con 0x7fe1b4147a00 2020-07-18T03:33:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b2ffd700 1 -- 172.21.15.47:0/66076444 >> v1:172.21.15.47:6789/0 conn(0x7fe1b4147a00 legacy=0x7fe1b405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b2ffd700 1 -- 172.21.15.47:0/66076444 >> v1:172.21.15.18:6789/0 conn(0x7fe1b41443f0 legacy=0x7fe1b405fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b2ffd700 1 -- 172.21.15.47:0/66076444 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40b5a80 con 0x7fe1b40ad100 2020-07-18T03:33:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b2ffd700 1 --2- 172.21.15.47:0/66076444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b40ad100 0x7fe1b40b4c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fe1ac001450 tx=0x7fe19c006dd0).ready entity=mon.0 client_cookie=78df355e28d68d0c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b1ffb700 1 -- 172.21.15.47:0/66076444 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1a40016d0 con 0x7fe1b40ad100 2020-07-18T03:33:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b1ffb700 1 -- 172.21.15.47:0/66076444 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c00d5c0 con 0x7fe1b40ad100 2020-07-18T03:33:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.180+0000 7fe1b1ffb700 1 -- 172.21.15.47:0/66076444 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe19c00d730 con 0x7fe1b40ad100 2020-07-18T03:33:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 -- 172.21.15.47:0/66076444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b40ad100 msgr2=0x7fe1b40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 --2- 172.21.15.47:0/66076444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b40ad100 0x7fe1b40b4c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fe1ac001450 tx=0x7fe19c006dd0).stop 2020-07-18T03:33:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 -- 172.21.15.47:0/66076444 shutdown_connections 2020-07-18T03:33:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 --2- 172.21.15.47:0/66076444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b40ad100 0x7fe1b40b4c10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 -- 172.21.15.47:0/66076444 >> 172.21.15.47:0/66076444 conn(0x7fe1b40b9340 msgr2=0x7fe1b40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 -- 172.21.15.47:0/66076444 shutdown_connections 2020-07-18T03:33:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.181+0000 7fe1b991d700 1 -- 172.21.15.47:0/66076444 wait complete. 2020-07-18T03:33:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.182+0000 7fe1b991d700 1 Processor -- start 2020-07-18T03:33:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.182+0000 7fe1b991d700 1 -- start start 2020-07-18T03:33:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b991d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe1b4147a00 0x7fe1b4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b991d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1b40b29c0 0x7fe1b4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b991d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b4153700 con 0x7fe1b41443f0 2020-07-18T03:33:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b991d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b4153520 con 0x7fe1b40b29c0 2020-07-18T03:33:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b991d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b41539a0 con 0x7fe1b4147a00 2020-07-18T03:33:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1b40b29c0 0x7fe1b4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b2ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe1b41443f0 0x7fe1b40b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43820/0 (socket says 172.21.15.47:43820) 2020-07-18T03:33:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1b40b29c0 0x7fe1b4156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34504/0 (socket says 172.21.15.47:34504) 2020-07-18T03:33:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b2ffd700 1 -- 172.21.15.47:0/2581423209 learned_addr learned my addr 172.21.15.47:0/2581423209 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b27fc700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe1b4147a00 0x7fe1b4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 -- 172.21.15.47:0/2581423209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe1b4147a00 msgr2=0x7fe1b4153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe1b4147a00 0x7fe1b4153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 -- 172.21.15.47:0/2581423209 >> v1:172.21.15.47:6789/0 conn(0x7fe1b41443f0 legacy=0x7fe1b40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 -- 172.21.15.47:0/2581423209 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40b4050 con 0x7fe1b40b29c0 2020-07-18T03:33:03.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.183+0000 7fe1b37fe700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1b40b29c0 0x7fe1b4156b40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fe19c00c000 tx=0x7fe1ac007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:03.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.184+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1ac00ce60 con 0x7fe1b40b29c0 2020-07-18T03:33:03.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.184+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac00a6b0 con 0x7fe1b40b29c0 2020-07-18T03:33:03.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.184+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b4157030 con 0x7fe1b40b29c0 2020-07-18T03:33:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.184+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1ac00a840 con 0x7fe1b40b29c0 2020-07-18T03:33:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.184+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b4157580 con 0x7fe1b40b29c0 2020-07-18T03:33:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.185+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe1ac010070 con 0x7fe1b40b29c0 2020-07-18T03:33:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.185+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b40aec80 con 0x7fe1b40b29c0 2020-07-18T03:33:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.185+0000 7fe1aaffd700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe19004ff60 0x7fe190052420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.185+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe1ac0677d0 con 0x7fe1b40b29c0 2020-07-18T03:33:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.185+0000 7fe1b2ffd700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe19004ff60 0x7fe190052420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.185+0000 7fe1b2ffd700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe19004ff60 0x7fe190052420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fe19c006920 tx=0x7fe1b40b03e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.188+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe1ac03fe40 con 0x7fe1b40b29c0 2020-07-18T03:33:03.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.395+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1b4057450 con 0x7fe1b40b29c0 2020-07-18T03:33:03.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.396+0000 7fe1aaffd700 1 -- 172.21.15.47:0/2581423209 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2133 (secure 0 0 0) 0x7fe1ac043460 con 0x7fe1b40b29c0 2020-07-18T03:33:03.398 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:03.398 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/848402287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":848402287},{"type":"v1","addr":"172.21.15.18:6833","nonce":848402287}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:00.941352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2273595707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2273595707},{"type":"v1","addr":"172.21.15.47:6835","nonce":2273595707}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:03.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe19004ff60 msgr2=0x7fe190052420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe19004ff60 0x7fe190052420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fe19c006920 tx=0x7fe1b40b03e0).stop 2020-07-18T03:33:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1b40b29c0 msgr2=0x7fe1b4156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1b40b29c0 0x7fe1b4156b40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fe19c00c000 tx=0x7fe1ac007da0).stop 2020-07-18T03:33:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 shutdown_connections 2020-07-18T03:33:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1b40b29c0 0x7fe1b4156b40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe19004ff60 0x7fe190052420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 --2- 172.21.15.47:0/2581423209 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe1b4147a00 0x7fe1b4153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 >> 172.21.15.47:0/2581423209 conn(0x7fe1b40b9340 msgr2=0x7fe1b40b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 shutdown_connections 2020-07-18T03:33:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.397+0000 7fe1b991d700 1 -- 172.21.15.47:0/2581423209 wait complete. 2020-07-18T03:33:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2020-07-18T03:33:03.611 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:32:59.940158+0000', 'modified': '2020-07-18T03:33:00.941352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6295': {'gid': 6295, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/2273595707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 2273595707}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 2273595707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:33:03.611 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:33:03.612 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:33:03.748 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6295,"want_state":"up:active","state":"up:active","rank_uptime":3.7978125089999999,"mdsmap_epoch":29,"osdmap_epoch":66,"osdmap_epoch_barrier":64,"uptime":9.1866163410000006} 2020-07-18T03:33:03.749 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6295, "mdsmap_epoch": 29, "osdmap_epoch": 66, "osdmap_epoch_barrier": 64, "rank_uptime": 3.797812509, "state": "up:active", "uptime": 9.186616341, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:33:03.750 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:33:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.863+0000 7fa321a44700 1 Processor -- start 2020-07-18T03:33:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.863+0000 7fa321a44700 1 -- start start 2020-07-18T03:33:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa321a44700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 0x7fa31c145940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa321a44700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c0afbd0 con 0x7fa31c0b2e10 2020-07-18T03:33:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa321a44700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c145e30 con 0x7fa31c0b31f0 2020-07-18T03:33:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa321a44700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c144e30 con 0x7fa31c1454f0 2020-07-18T03:33:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa31a7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa31c0b31f0 0x7fa31c0b2130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43828/0 (socket says 172.21.15.47:43828) 2020-07-18T03:33:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3778571359 learned_addr learned my addr 172.21.15.47:0/3778571359 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727224760 0 0) 0x7fa31c145e30 con 0x7fa31c0b31f0 2020-07-18T03:33:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2fc003240 con 0x7fa31c0b31f0 2020-07-18T03:33:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.864+0000 7fa31b7fe700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 0x7fa31c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4184216717 0 0) 0x7fa2fc003240 con 0x7fa31c0b31f0 2020-07-18T03:33:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 msgr2=0x7fa31c145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 0x7fa31c145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 >> v1:172.21.15.18:6789/0 conn(0x7fa31c0b2e10 legacy=0x7fa31c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c1491c0 con 0x7fa31c0b31f0 2020-07-18T03:33:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fa30c0016d0 con 0x7fa31c0b31f0 2020-07-18T03:33:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 >> v1:172.21.15.47:6789/0 conn(0x7fa31c0b31f0 legacy=0x7fa31c0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2fc003760 0x7fa2fc003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2fc0044e0 0x7fa2fc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 0x7fa2fc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c144e30 con 0x7fa2fc003760 2020-07-18T03:33:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc005390 con 0x7fa2fc0044e0 2020-07-18T03:33:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc004320 con 0x7fa31c1454f0 2020-07-18T03:33:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2fc0044e0 0x7fa2fc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31b7fe700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 0x7fa2fc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31affd700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2fc003760 0x7fa2fc003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 msgr2=0x7fa2fc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c1454f0 0x7fa2fc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2fc003760 msgr2=0x7fa2fc003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2fc003760 0x7fa2fc003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3778571359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc003240 con 0x7fa2fc0044e0 2020-07-18T03:33:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.865+0000 7fa31a7fc700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2fc0044e0 0x7fa2fc004960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fa30c0013e0 tx=0x7fa31c0b5d00).ready entity=mon.1 client_cookie=48e377f1f822d351 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa30c0070d0 con 0x7fa2fc0044e0 2020-07-18T03:33:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c007230 con 0x7fa2fc0044e0 2020-07-18T03:33:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa319ffb700 1 -- 172.21.15.47:0/3778571359 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa30c0055f0 con 0x7fa2fc0044e0 2020-07-18T03:33:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa321a44700 1 -- 172.21.15.47:0/3778571359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2fc0044e0 msgr2=0x7fa2fc004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa321a44700 1 --2- 172.21.15.47:0/3778571359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2fc0044e0 0x7fa2fc004960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fa30c0013e0 tx=0x7fa31c0b5d00).stop 2020-07-18T03:33:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa31b7fe700 1 -- 172.21.15.47:0/3778571359 reap_dead start 2020-07-18T03:33:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa321a44700 1 -- 172.21.15.47:0/3778571359 shutdown_connections 2020-07-18T03:33:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa321a44700 1 -- 172.21.15.47:0/3778571359 >> 172.21.15.47:0/3778571359 conn(0x7fa31c0b9340 msgr2=0x7fa31c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa321a44700 1 -- 172.21.15.47:0/3778571359 shutdown_connections 2020-07-18T03:33:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.866+0000 7fa321a44700 1 -- 172.21.15.47:0/3778571359 wait complete. 2020-07-18T03:33:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.867+0000 7fa321a44700 1 Processor -- start 2020-07-18T03:33:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.867+0000 7fa321a44700 1 -- start start 2020-07-18T03:33:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa321a44700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa31c0b8db0 0x7fa31c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa321a44700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c066040 0x7fa31c0653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa321a44700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c148fa0 con 0x7fa31c0b89d0 2020-07-18T03:33:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa321a44700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c062690 con 0x7fa31c0b8db0 2020-07-18T03:33:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa321a44700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c062930 con 0x7fa31c066040 2020-07-18T03:33:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa31c0b8db0 0x7fa31c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa31c0b8db0 0x7fa31c148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34522/0 (socket says 172.21.15.47:34522) 2020-07-18T03:33:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3956654927 learned_addr learned my addr 172.21.15.47:0/3956654927 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3956654927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c066040 msgr2=0x7fa31c0653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31b7fe700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c066040 0x7fa31c0653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c066040 0x7fa31c0653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3956654927 >> v1:172.21.15.18:6789/0 conn(0x7fa31c0b89d0 legacy=0x7fa31c064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.868+0000 7fa31a7fc700 1 -- 172.21.15.47:0/3956654927 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c0658d0 con 0x7fa31c0b8db0 2020-07-18T03:33:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.869+0000 7fa31a7fc700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c0b8db0 0x7fa31c148ab0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fa31c147c10 tx=0x7fa30c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:03.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.869+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa30c013640 con 0x7fa31c0b8db0 2020-07-18T03:33:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.869+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0137a0 con 0x7fa31c0b8db0 2020-07-18T03:33:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.869+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c062f80 con 0x7fa31c0b8db0 2020-07-18T03:33:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.869+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa30c013b00 con 0x7fa31c0b8db0 2020-07-18T03:33:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.869+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c141bc0 con 0x7fa31c0b8db0 2020-07-18T03:33:03.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.870+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c0aff00 con 0x7fa31c0b8db0 2020-07-18T03:33:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.870+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa30c01d070 con 0x7fa31c0b8db0 2020-07-18T03:33:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.870+0000 7fa312ffd700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2f804fee0 0x7fa2f80523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.870+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa30c068e30 con 0x7fa31c0b8db0 2020-07-18T03:33:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.870+0000 7fa31affd700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2f804fee0 0x7fa2f80523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.871+0000 7fa31affd700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2f804fee0 0x7fa2f80523a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fa30400d000 tx=0x7fa304007b70).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:03.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:03.874+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa30c0415c0 con 0x7fa31c0b8db0 2020-07-18T03:33:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.037+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa31c063110 con 0x7fa31c0b8db0 2020-07-18T03:33:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.038+0000 7fa312ffd700 1 -- 172.21.15.47:0/3956654927 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fa30c030fb0 con 0x7fa31c0b8db0 2020-07-18T03:33:04.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:04.040 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":66,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:33:01.945196+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:32:57.493054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-07-18T03:32:58.764476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:33:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2f804fee0 msgr2=0x7fa2f80523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2f804fee0 0x7fa2f80523a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fa30400d000 tx=0x7fa304007b70).stop 2020-07-18T03:33:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c0b8db0 msgr2=0x7fa31c148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c0b8db0 0x7fa31c148ab0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fa31c147c10 tx=0x7fa30c015000).stop 2020-07-18T03:33:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 shutdown_connections 2020-07-18T03:33:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa2f804fee0 0x7fa2f80523a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa31c066040 0x7fa31c0653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 --2- 172.21.15.47:0/3956654927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c0b8db0 0x7fa31c148ab0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 >> 172.21.15.47:0/3956654927 conn(0x7fa31c0b9340 msgr2=0x7fa31c142980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 shutdown_connections 2020-07-18T03:33:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.040+0000 7fa321a44700 1 -- 172.21.15.47:0/3956654927 wait complete. 2020-07-18T03:33:04.051 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:32:57.493054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:32:58.764476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:33:04.053 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:33:04.170 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.169+0000 7fdc37b52700 1 Processor -- start 2020-07-18T03:33:04.171 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.169+0000 7fdc37b52700 1 -- start start 2020-07-18T03:33:04.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc37b52700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdc300ad330 0x7fdc30143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.173 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc37b52700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc300b3a10 0x7fdc300b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc37b52700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc30147de0 con 0x7fdc300b59b0 2020-07-18T03:33:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc37b52700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc300ae000 con 0x7fdc300ad330 2020-07-18T03:33:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc37b52700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc30148010 con 0x7fdc300b3a10 2020-07-18T03:33:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc360ef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc300b3a10 0x7fdc300b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc350ed700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdc300b59b0 0x7fdc300b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43848/0 (socket says 172.21.15.47:43848) 2020-07-18T03:33:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc350ed700 1 -- 172.21.15.47:0/3137435382 learned_addr learned my addr 172.21.15.47:0/3137435382 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc358ee700 1 --2- 172.21.15.47:0/3137435382 >> v2:172.21.15.18:3300/0 conn(0x7fdc300ad330 0x7fdc30143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc360ef700 1 -- 172.21.15.47:0/3137435382 >> v2:172.21.15.18:3300/0 conn(0x7fdc300ad330 msgr2=0x7fdc30143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc360ef700 1 --2- 172.21.15.47:0/3137435382 >> v2:172.21.15.18:3300/0 conn(0x7fdc300ad330 0x7fdc30143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc360ef700 1 -- 172.21.15.47:0/3137435382 >> v1:172.21.15.47:6789/0 conn(0x7fdc300b59b0 legacy=0x7fdc300b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc360ef700 1 -- 172.21.15.47:0/3137435382 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300ad710 con 0x7fdc300b3a10 2020-07-18T03:33:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc348ec700 1 -- 172.21.15.47:0/3137435382 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136473685 0 0) 0x7fdc30147de0 con 0x7fdc300b59b0 2020-07-18T03:33:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.170+0000 7fdc360ef700 1 --2- 172.21.15.47:0/3137435382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b3a10 0x7fdc300b3e80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fdc280032b0 tx=0x7fdc280091d0).ready entity=mon.1 client_cookie=8aabd59603fd2ea server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc348ec700 1 -- 172.21.15.47:0/3137435382 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc2800ba50 con 0x7fdc300b3a10 2020-07-18T03:33:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc348ec700 1 -- 172.21.15.47:0/3137435382 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2800bbb0 con 0x7fdc300b3a10 2020-07-18T03:33:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc348ec700 1 -- 172.21.15.47:0/3137435382 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc2800bd20 con 0x7fdc300b3a10 2020-07-18T03:33:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 -- 172.21.15.47:0/3137435382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b3a10 msgr2=0x7fdc300b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 --2- 172.21.15.47:0/3137435382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b3a10 0x7fdc300b3e80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fdc280032b0 tx=0x7fdc280091d0).stop 2020-07-18T03:33:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 -- 172.21.15.47:0/3137435382 shutdown_connections 2020-07-18T03:33:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 --2- 172.21.15.47:0/3137435382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b3a10 0x7fdc300b3e80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 --2- 172.21.15.47:0/3137435382 >> v2:172.21.15.18:3300/0 conn(0x7fdc300ad330 0x7fdc30143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 -- 172.21.15.47:0/3137435382 >> 172.21.15.47:0/3137435382 conn(0x7fdc3005f5a0 msgr2=0x7fdc3005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 -- 172.21.15.47:0/3137435382 shutdown_connections 2020-07-18T03:33:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.171+0000 7fdc37b52700 1 -- 172.21.15.47:0/3137435382 wait complete. 2020-07-18T03:33:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.172+0000 7fdc37b52700 1 Processor -- start 2020-07-18T03:33:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.172+0000 7fdc37b52700 1 -- start start 2020-07-18T03:33:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc37b52700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc300b59b0 0x7fdc30062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc37b52700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdc30155130 0x7fdc30155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc37b52700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc30063560 con 0x7fdc300ad330 2020-07-18T03:33:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc37b52700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc30063340 con 0x7fdc30155130 2020-07-18T03:33:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc37b52700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc30063800 con 0x7fdc300b59b0 2020-07-18T03:33:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc300b59b0 0x7fdc30062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc300b59b0 0x7fdc30062e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34534/0 (socket says 172.21.15.47:34534) 2020-07-18T03:33:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 -- 172.21.15.47:0/1327492215 learned_addr learned my addr 172.21.15.47:0/1327492215 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc360ef700 1 --2- 172.21.15.47:0/1327492215 >> v2:172.21.15.18:3300/0 conn(0x7fdc30155130 0x7fdc30155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 -- 172.21.15.47:0/1327492215 >> v2:172.21.15.18:3300/0 conn(0x7fdc30155130 msgr2=0x7fdc30155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 --2- 172.21.15.47:0/1327492215 >> v2:172.21.15.18:3300/0 conn(0x7fdc30155130 0x7fdc30155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 -- 172.21.15.47:0/1327492215 >> v1:172.21.15.18:6789/0 conn(0x7fdc300ad330 legacy=0x7fdc301454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc350ed700 1 -- 172.21.15.47:0/1327492215 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30063aa0 con 0x7fdc300b59b0 2020-07-18T03:33:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.173+0000 7fdc360ef700 1 --2- 172.21.15.47:0/1327492215 >> v2:172.21.15.18:3300/0 conn(0x7fdc30155130 0x7fdc30155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.174+0000 7fdc350ed700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b59b0 0x7fdc30062e50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fdc2c000e90 tx=0x7fdc2c007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.174+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc2c0072e0 con 0x7fdc300b59b0 2020-07-18T03:33:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.174+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c007440 con 0x7fdc300b59b0 2020-07-18T03:33:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.174+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc30155d30 con 0x7fdc300b59b0 2020-07-18T03:33:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.174+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc2c0057e0 con 0x7fdc300b59b0 2020-07-18T03:33:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.174+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc30143640 con 0x7fdc300b59b0 2020-07-18T03:33:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.175+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc30063c30 con 0x7fdc300b59b0 2020-07-18T03:33:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.175+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fdc2c01a060 con 0x7fdc300b59b0 2020-07-18T03:33:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.175+0000 7fdc1dffb700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdc0804ffb0 0x7fdc08052470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.175+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdc2c067610 con 0x7fdc300b59b0 2020-07-18T03:33:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.175+0000 7fdc358ee700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdc0804ffb0 0x7fdc08052470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.176+0000 7fdc358ee700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdc0804ffb0 0x7fdc08052470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fdc30147120 tx=0x7fdc20008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.178+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fdc2c03fda0 con 0x7fdc300b59b0 2020-07-18T03:33:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.337+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdc3005b120 con 0x7fdc300b59b0 2020-07-18T03:33:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.337+0000 7fdc1dffb700 1 -- 172.21.15.47:0/1327492215 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdc2c018070 con 0x7fdc300b59b0 2020-07-18T03:33:04.340 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:33:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdc0804ffb0 msgr2=0x7fdc08052470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdc0804ffb0 0x7fdc08052470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fdc30147120 tx=0x7fdc20008040).stop 2020-07-18T03:33:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b59b0 msgr2=0x7fdc30062e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b59b0 0x7fdc30062e50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fdc2c000e90 tx=0x7fdc2c007aa0).stop 2020-07-18T03:33:04.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 shutdown_connections 2020-07-18T03:33:04.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdc0804ffb0 0x7fdc08052470 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 --2- 172.21.15.47:0/1327492215 >> v2:172.21.15.18:3300/0 conn(0x7fdc30155130 0x7fdc30155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 --2- 172.21.15.47:0/1327492215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc300b59b0 0x7fdc30062e50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 >> 172.21.15.47:0/1327492215 conn(0x7fdc3005f5a0 msgr2=0x7fdc300b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 shutdown_connections 2020-07-18T03:33:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.339+0000 7fdc37b52700 1 -- 172.21.15.47:0/1327492215 wait complete. 2020-07-18T03:33:04.351 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:33:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.470+0000 7ff906731700 1 Processor -- start 2020-07-18T03:33:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.470+0000 7ff906731700 1 -- start start 2020-07-18T03:33:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff906731700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9000b59b0 0x7ff9000b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff906731700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff900147de0 con 0x7ff9000ad330 2020-07-18T03:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff906731700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9000ae000 con 0x7ff9000b6300 2020-07-18T03:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff906731700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff900148010 con 0x7ff9000b59b0 2020-07-18T03:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff8ff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9000b59b0 0x7ff9000b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff904cce700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff9000b6300 0x7ff9000b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43862/0 (socket says 172.21.15.47:43862) 2020-07-18T03:33:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff904cce700 1 -- 172.21.15.47:0/3550435054 learned_addr learned my addr 172.21.15.47:0/3550435054 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff8ff7fe700 1 -- 172.21.15.47:0/3550435054 >> v1:172.21.15.47:6789/0 conn(0x7ff9000b6300 legacy=0x7ff9000b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff8ff7fe700 1 -- 172.21.15.47:0/3550435054 >> v1:172.21.15.18:6789/0 conn(0x7ff9000ad330 legacy=0x7ff900143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff8ff7fe700 1 -- 172.21.15.47:0/3550435054 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000ad710 con 0x7ff9000b59b0 2020-07-18T03:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff8feffd700 1 -- 172.21.15.47:0/3550435054 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132602337 0 0) 0x7ff9000ae000 con 0x7ff9000b6300 2020-07-18T03:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.471+0000 7ff8ff7fe700 1 --2- 172.21.15.47:0/3550435054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b59b0 0x7ff9000b5e10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7ff8f4002f30 tx=0x7ff8f4006dd0).ready entity=mon.1 client_cookie=90989ec2e55e0d3b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff8feffd700 1 -- 172.21.15.47:0/3550435054 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8f40099e0 con 0x7ff9000b59b0 2020-07-18T03:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff8feffd700 1 -- 172.21.15.47:0/3550435054 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f4009b40 con 0x7ff9000b59b0 2020-07-18T03:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff8feffd700 1 -- 172.21.15.47:0/3550435054 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8f4009ea0 con 0x7ff9000b59b0 2020-07-18T03:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 -- 172.21.15.47:0/3550435054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b59b0 msgr2=0x7ff9000b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 --2- 172.21.15.47:0/3550435054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b59b0 0x7ff9000b5e10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7ff8f4002f30 tx=0x7ff8f4006dd0).stop 2020-07-18T03:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 -- 172.21.15.47:0/3550435054 shutdown_connections 2020-07-18T03:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 --2- 172.21.15.47:0/3550435054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b59b0 0x7ff9000b5e10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 -- 172.21.15.47:0/3550435054 >> 172.21.15.47:0/3550435054 conn(0x7ff90005f5a0 msgr2=0x7ff90005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 -- 172.21.15.47:0/3550435054 shutdown_connections 2020-07-18T03:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.472+0000 7ff906731700 1 -- 172.21.15.47:0/3550435054 wait complete. 2020-07-18T03:33:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.473+0000 7ff906731700 1 Processor -- start 2020-07-18T03:33:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.473+0000 7ff906731700 1 -- start start 2020-07-18T03:33:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff906731700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff9000ad330 0x7ff900146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff906731700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9000b07c0 0x7ff9000b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff906731700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9000b0f00 con 0x7ff9000b6300 2020-07-18T03:33:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff906731700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9000b11a0 con 0x7ff9000ad330 2020-07-18T03:33:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff906731700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff900066d40 con 0x7ff9000b07c0 2020-07-18T03:33:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff8ff7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff9000b6300 0x7ff90013bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43866/0 (socket says 172.21.15.47:43866) 2020-07-18T03:33:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff904cce700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff9000b07c0 0x7ff9000b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff8ff7fe700 1 -- 172.21.15.47:0/2956179315 learned_addr learned my addr 172.21.15.47:0/2956179315 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff8fffff700 1 --2- 172.21.15.47:0/2956179315 >> v2:172.21.15.18:3300/0 conn(0x7ff9000ad330 0x7ff900146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff904cce700 1 -- 172.21.15.47:0/2956179315 >> v2:172.21.15.18:3300/0 conn(0x7ff9000ad330 msgr2=0x7ff900146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff904cce700 1 --2- 172.21.15.47:0/2956179315 >> v2:172.21.15.18:3300/0 conn(0x7ff9000ad330 0x7ff900146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff904cce700 1 -- 172.21.15.47:0/2956179315 >> v1:172.21.15.47:6789/0 conn(0x7ff9000b6300 legacy=0x7ff90013bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff904cce700 1 -- 172.21.15.47:0/2956179315 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900146af0 con 0x7ff9000b07c0 2020-07-18T03:33:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800713527 0 0) 0x7ff9000b0f00 con 0x7ff9000b6300 2020-07-18T03:33:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.474+0000 7ff8fffff700 1 --2- 172.21.15.47:0/2956179315 >> v2:172.21.15.18:3300/0 conn(0x7ff9000ad330 0x7ff900146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:33:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.475+0000 7ff904cce700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b07c0 0x7ff9000b2390 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7ff8f8001210 tx=0x7ff8f8007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.475+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8f8005b80 con 0x7ff9000b07c0 2020-07-18T03:33:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.475+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f8005ce0 con 0x7ff9000b07c0 2020-07-18T03:33:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.475+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90013c220 con 0x7ff9000b07c0 2020-07-18T03:33:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.475+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8f8004a70 con 0x7ff9000b07c0 2020-07-18T03:33:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.475+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9000b1800 con 0x7ff9000b07c0 2020-07-18T03:33:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.476+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900066e80 con 0x7ff9000b07c0 2020-07-18T03:33:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.476+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff8f801b060 con 0x7ff9000b07c0 2020-07-18T03:33:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.476+0000 7ff8dffff700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8d804fe90 0x7ff8d8052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.476+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff8f8067700 con 0x7ff9000b07c0 2020-07-18T03:33:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.476+0000 7ff8fffff700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8d804fe90 0x7ff8d8052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.477+0000 7ff8fffff700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8d804fe90 0x7ff8d8052350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7ff8f0008000 tx=0x7ff8f0009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.480+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff8f803fe90 con 0x7ff9000b07c0 2020-07-18T03:33:04.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.638+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff90005b120 con 0x7ff9000b07c0 2020-07-18T03:33:04.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.639+0000 7ff8dffff700 1 -- 172.21.15.47:0/2956179315 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8f80190a0 con 0x7ff9000b07c0 2020-07-18T03:33:04.642 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:33:04.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8d804fe90 msgr2=0x7ff8d8052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8d804fe90 0x7ff8d8052350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7ff8f0008000 tx=0x7ff8f0009040).stop 2020-07-18T03:33:04.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b07c0 msgr2=0x7ff9000b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b07c0 0x7ff9000b2390 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7ff8f8001210 tx=0x7ff8f8007c40).stop 2020-07-18T03:33:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 shutdown_connections 2020-07-18T03:33:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9000b07c0 0x7ff9000b2390 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 --2- 172.21.15.47:0/2956179315 >> v2:172.21.15.18:3300/0 conn(0x7ff9000ad330 0x7ff900146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 --2- 172.21.15.47:0/2956179315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff8d804fe90 0x7ff8d8052350 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 >> 172.21.15.47:0/2956179315 conn(0x7ff90005f5a0 msgr2=0x7ff90013cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:04.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.640+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 shutdown_connections 2020-07-18T03:33:04.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.641+0000 7ff906731700 1 -- 172.21.15.47:0/2956179315 wait complete. 2020-07-18T03:33:04.653 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:33:04.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.768+0000 7fb4998c1700 1 Processor -- start 2020-07-18T03:33:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.768+0000 7fb4998c1700 1 -- start start 2020-07-18T03:33:04.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4998c1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb4940b59b0 0x7fb4940b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4998c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4940b6300 0x7fb4940b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4998c1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb494147de0 con 0x7fb4940ad330 2020-07-18T03:33:04.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4998c1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4940ae000 con 0x7fb4940b59b0 2020-07-18T03:33:04.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4998c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb494148010 con 0x7fb4940b6300 2020-07-18T03:33:04.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4940b6300 0x7fb4940b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4940b6300 0x7fb4940b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34556/0 (socket says 172.21.15.47:34556) 2020-07-18T03:33:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 -- 172.21.15.47:0/133564211 learned_addr learned my addr 172.21.15.47:0/133564211 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4927fc700 1 --2- 172.21.15.47:0/133564211 >> v2:172.21.15.18:3300/0 conn(0x7fb4940b59b0 0x7fb4940b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 -- 172.21.15.47:0/133564211 >> v2:172.21.15.18:3300/0 conn(0x7fb4940b59b0 msgr2=0x7fb4940b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:04.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 --2- 172.21.15.47:0/133564211 >> v2:172.21.15.18:3300/0 conn(0x7fb4940b59b0 0x7fb4940b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 -- 172.21.15.47:0/133564211 >> v1:172.21.15.18:6789/0 conn(0x7fb4940ad330 legacy=0x7fb494143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:04.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.769+0000 7fb4937fe700 1 -- 172.21.15.47:0/133564211 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940ad710 con 0x7fb4940b6300 2020-07-18T03:33:04.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4937fe700 1 --2- 172.21.15.47:0/133564211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b6300 0x7fb4940b3a10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fb48c0032b0 tx=0x7fb48c009270).ready entity=mon.1 client_cookie=403c11cdce7a9663 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb491ffb700 1 -- 172.21.15.47:0/133564211 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb48c00bc00 con 0x7fb4940b6300 2020-07-18T03:33:04.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb491ffb700 1 -- 172.21.15.47:0/133564211 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c00bd60 con 0x7fb4940b6300 2020-07-18T03:33:04.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb491ffb700 1 -- 172.21.15.47:0/133564211 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb48c011400 con 0x7fb4940b6300 2020-07-18T03:33:04.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 -- 172.21.15.47:0/133564211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b6300 msgr2=0x7fb4940b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 --2- 172.21.15.47:0/133564211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b6300 0x7fb4940b3a10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fb48c0032b0 tx=0x7fb48c009270).stop 2020-07-18T03:33:04.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 -- 172.21.15.47:0/133564211 shutdown_connections 2020-07-18T03:33:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 --2- 172.21.15.47:0/133564211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b6300 0x7fb4940b3a10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 --2- 172.21.15.47:0/133564211 >> v2:172.21.15.18:3300/0 conn(0x7fb4940b59b0 0x7fb4940b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 -- 172.21.15.47:0/133564211 >> 172.21.15.47:0/133564211 conn(0x7fb49405f5a0 msgr2=0x7fb49405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 -- 172.21.15.47:0/133564211 shutdown_connections 2020-07-18T03:33:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.770+0000 7fb4998c1700 1 -- 172.21.15.47:0/133564211 wait complete. 2020-07-18T03:33:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.771+0000 7fb4998c1700 1 Processor -- start 2020-07-18T03:33:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.771+0000 7fb4998c1700 1 -- start start 2020-07-18T03:33:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4998c1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb4940ad330 0x7fb494155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4998c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4940b59b0 0x7fb494061d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4998c1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb49413cc90 0x7fb49413d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4998c1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb494158dc0 con 0x7fb49413cc90 2020-07-18T03:33:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4998c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb494158fd0 con 0x7fb4940b59b0 2020-07-18T03:33:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4998c1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb494159270 con 0x7fb4940ad330 2020-07-18T03:33:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4927fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4940b59b0 0x7fb494061d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4927fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4940b59b0 0x7fb494061d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34562/0 (socket says 172.21.15.47:34562) 2020-07-18T03:33:04.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4927fc700 1 -- 172.21.15.47:0/376174198 learned_addr learned my addr 172.21.15.47:0/376174198 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:04.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb492ffd700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb4940ad330 0x7fb494155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.772+0000 7fb4937fe700 1 --2- 172.21.15.47:0/376174198 >> v2:172.21.15.18:3300/0 conn(0x7fb49413cc90 0x7fb49413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4927fc700 1 -- 172.21.15.47:0/376174198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb4940ad330 msgr2=0x7fb494155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4927fc700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb4940ad330 0x7fb494155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4927fc700 1 -- 172.21.15.47:0/376174198 >> v2:172.21.15.18:3300/0 conn(0x7fb49413cc90 msgr2=0x7fb49413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:04.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4927fc700 1 --2- 172.21.15.47:0/376174198 >> v2:172.21.15.18:3300/0 conn(0x7fb49413cc90 0x7fb49413d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:04.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4927fc700 1 -- 172.21.15.47:0/376174198 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb494062230 con 0x7fb4940b59b0 2020-07-18T03:33:04.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4927fc700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b59b0 0x7fb494061d40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fb484003070 tx=0x7fb484008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb484009980 con 0x7fb4940b59b0 2020-07-18T03:33:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb484009ae0 con 0x7fb4940b59b0 2020-07-18T03:33:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb494159710 con 0x7fb4940b59b0 2020-07-18T03:33:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb484009e40 con 0x7fb4940b59b0 2020-07-18T03:33:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.773+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb494158900 con 0x7fb4940b59b0 2020-07-18T03:33:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.774+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb494158b20 con 0x7fb4940b59b0 2020-07-18T03:33:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.774+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb484013480 con 0x7fb4940b59b0 2020-07-18T03:33:04.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.774+0000 7fb47affd700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb47004ffd0 0x7fb470052490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:04.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.775+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb4840681c0 con 0x7fb4940b59b0 2020-07-18T03:33:04.793 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.775+0000 7fb492ffd700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb47004ffd0 0x7fb470052490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:04.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.775+0000 7fb492ffd700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb47004ffd0 0x7fb470052490 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fb48800b000 tx=0x7fb488009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:04.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.778+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb4840408f0 con 0x7fb4940b59b0 2020-07-18T03:33:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.932+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4940adb40 con 0x7fb4940b59b0 2020-07-18T03:33:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.933+0000 7fb47affd700 1 -- 172.21.15.47:0/376174198 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb484043f10 con 0x7fb4940b59b0 2020-07-18T03:33:05.450 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:33:05.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb47004ffd0 msgr2=0x7fb470052490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb47004ffd0 0x7fb470052490 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fb48800b000 tx=0x7fb488009040).stop 2020-07-18T03:33:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b59b0 msgr2=0x7fb494061d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b59b0 0x7fb494061d40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fb484003070 tx=0x7fb484008040).stop 2020-07-18T03:33:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 shutdown_connections 2020-07-18T03:33:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb47004ffd0 0x7fb470052490 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 --2- 172.21.15.47:0/376174198 >> v2:172.21.15.18:3300/0 conn(0x7fb49413cc90 0x7fb49413d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb4940ad330 0x7fb494155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 --2- 172.21.15.47:0/376174198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4940b59b0 0x7fb494061d40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 >> 172.21.15.47:0/376174198 conn(0x7fb49405f5a0 msgr2=0x7fb494147460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 shutdown_connections 2020-07-18T03:33:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:04.934+0000 7fb4998c1700 1 -- 172.21.15.47:0/376174198 wait complete. 2020-07-18T03:33:05.458 INFO:tasks.ceph.mds.a.smithi047.stderr:2020-07-18T03:33:05.396+0000 7f5fd4ea3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:33:05.459 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:33:05.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.585+0000 7f5370578700 1 Processor -- start 2020-07-18T03:33:05.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.585+0000 7f5370578700 1 -- start start 2020-07-18T03:33:05.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f5370578700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53680b59b0 0x7f53680b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f5370578700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f53680b6300 0x7f53680b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f5370578700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5368147de0 con 0x7f53680ad330 2020-07-18T03:33:05.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f5370578700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53680ae000 con 0x7f53680b59b0 2020-07-18T03:33:05.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f5370578700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5368148010 con 0x7f53680b6300 2020-07-18T03:33:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53680b59b0 0x7f53680b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53680b59b0 0x7f53680b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34568/0 (socket says 172.21.15.47:34568) 2020-07-18T03:33:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 -- 172.21.15.47:0/1862298907 learned_addr learned my addr 172.21.15.47:0/1862298907 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536eb15700 1 --2- 172.21.15.47:0/1862298907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f53680b6300 0x7f53680b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 -- 172.21.15.47:0/1862298907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f53680b6300 msgr2=0x7f53680b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:05.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 --2- 172.21.15.47:0/1862298907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f53680b6300 0x7f53680b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 -- 172.21.15.47:0/1862298907 >> v1:172.21.15.18:6789/0 conn(0x7f53680ad330 legacy=0x7f5368143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:05.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 -- 172.21.15.47:0/1862298907 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680ad710 con 0x7f53680b59b0 2020-07-18T03:33:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.586+0000 7f536db13700 1 --2- 172.21.15.47:0/1862298907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53680b59b0 0x7f53680b5e10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f5368067970 tx=0x7f5364006e90).ready entity=mon.1 client_cookie=c8489f7f0e9f75a4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f536d312700 1 -- 172.21.15.47:0/1862298907 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53640098e0 con 0x7f53680b59b0 2020-07-18T03:33:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f536d312700 1 -- 172.21.15.47:0/1862298907 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5364009a40 con 0x7f53680b59b0 2020-07-18T03:33:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f536d312700 1 -- 172.21.15.47:0/1862298907 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5364009de0 con 0x7f53680b59b0 2020-07-18T03:33:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 -- 172.21.15.47:0/1862298907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53680b59b0 msgr2=0x7f53680b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 --2- 172.21.15.47:0/1862298907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53680b59b0 0x7f53680b5e10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f5368067970 tx=0x7f5364006e90).stop 2020-07-18T03:33:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 -- 172.21.15.47:0/1862298907 shutdown_connections 2020-07-18T03:33:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 --2- 172.21.15.47:0/1862298907 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f53680b6300 0x7f53680b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 --2- 172.21.15.47:0/1862298907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53680b59b0 0x7f53680b5e10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 -- 172.21.15.47:0/1862298907 >> 172.21.15.47:0/1862298907 conn(0x7f536805f5a0 msgr2=0x7f536805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 -- 172.21.15.47:0/1862298907 shutdown_connections 2020-07-18T03:33:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.587+0000 7f5370578700 1 -- 172.21.15.47:0/1862298907 wait complete. 2020-07-18T03:33:05.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.588+0000 7f5370578700 1 Processor -- start 2020-07-18T03:33:05.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.588+0000 7f5370578700 1 -- start start 2020-07-18T03:33:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f5370578700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f53680ad330 0x7f5368142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f5370578700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5368158670 0x7f536813ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f5370578700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5368158bf0 con 0x7f53680b6300 2020-07-18T03:33:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f5370578700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53680b9030 con 0x7f53680ad330 2020-07-18T03:33:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f5370578700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5368158e90 con 0x7f5368158670 2020-07-18T03:33:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f536db13700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53680b6300 0x7f5368139010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43894/0 (socket says 172.21.15.47:43894) 2020-07-18T03:33:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f536db13700 1 -- 172.21.15.47:0/243871020 learned_addr learned my addr 172.21.15.47:0/243871020 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f536eb15700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5368158670 0x7f536813ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f536e314700 1 --2- 172.21.15.47:0/243871020 >> v2:172.21.15.18:3300/0 conn(0x7f53680ad330 0x7f5368142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097045073 0 0) 0x7f5368158bf0 con 0x7f53680b6300 2020-07-18T03:33:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.589+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5340003240 con 0x7f53680b6300 2020-07-18T03:33:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3440087090 0 0) 0x7f5340003240 con 0x7f53680b6300 2020-07-18T03:33:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5368158bf0 con 0x7f53680b6300 2020-07-18T03:33:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f53640098e0 con 0x7f53680b6300 2020-07-18T03:33:05.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f536e314700 1 -- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5368158670 msgr2=0x7f536813ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f536e314700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5368158670 0x7f536813ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f536e314700 1 -- 172.21.15.47:0/243871020 >> v1:172.21.15.47:6789/0 conn(0x7f53680b6300 legacy=0x7f5368139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f536e314700 1 -- 172.21.15.47:0/243871020 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368159130 con 0x7f53680ad330 2020-07-18T03:33:05.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f536eb15700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5368158670 0x7f536813ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:05.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f536e314700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53680ad330 0x7f5368142070 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5358001f50 tx=0x7f5358007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:05.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5358007030 con 0x7f53680ad330 2020-07-18T03:33:05.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5358007190 con 0x7f53680ad330 2020-07-18T03:33:05.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5368142560 con 0x7f53680ad330 2020-07-18T03:33:05.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53580064f0 con 0x7f53680ad330 2020-07-18T03:33:05.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.590+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53681398e0 con 0x7f53680ad330 2020-07-18T03:33:05.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.591+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53681592c0 con 0x7f53680ad330 2020-07-18T03:33:05.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.591+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5358010070 con 0x7f53680ad330 2020-07-18T03:33:05.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.592+0000 7f53567fc700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f53400505e0 0x7f5340052aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.592+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5358066ff0 con 0x7f53680ad330 2020-07-18T03:33:05.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.592+0000 7f536db13700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f53400505e0 0x7f5340052aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.592+0000 7f536db13700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f53400505e0 0x7f5340052aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f5364009fc0 tx=0x7f5364015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:05.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.596+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f535803f760 con 0x7f53680ad330 2020-07-18T03:33:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.755+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f536805b120 con 0x7f53680ad330 2020-07-18T03:33:05.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.755+0000 7f53567fc700 1 -- 172.21.15.47:0/243871020 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5358042d80 con 0x7f53680ad330 2020-07-18T03:33:05.758 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:33:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f53400505e0 msgr2=0x7f5340052aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f53400505e0 0x7f5340052aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f5364009fc0 tx=0x7f5364015000).stop 2020-07-18T03:33:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53680ad330 msgr2=0x7f5368142070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53680ad330 0x7f5368142070 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5358001f50 tx=0x7f5358007b50).stop 2020-07-18T03:33:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 shutdown_connections 2020-07-18T03:33:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5368158670 0x7f536813ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53680ad330 0x7f5368142070 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 --2- 172.21.15.47:0/243871020 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f53400505e0 0x7f5340052aa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.756+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 >> 172.21.15.47:0/243871020 conn(0x7f536805f5a0 msgr2=0x7f5368147460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.757+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 shutdown_connections 2020-07-18T03:33:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.757+0000 7f5370578700 1 -- 172.21.15.47:0/243871020 wait complete. 2020-07-18T03:33:05.769 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:33:05.770 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.882+0000 7f1b1d8b6700 1 Processor -- start 2020-07-18T03:33:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.882+0000 7f1b1d8b6700 1 -- start start 2020-07-18T03:33:05.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b1d8b6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1b18147e40 0x7f1b18143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b1d8b6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b18147330 con 0x7f1b180ad100 2020-07-18T03:33:05.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b1d8b6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180b57e0 con 0x7f1b18147a00 2020-07-18T03:33:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b1d8b6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180ae530 con 0x7f1b18147e40 2020-07-18T03:33:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b167fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1b18147a00 0x7f1b181433e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43902/0 (socket says 172.21.15.47:43902) 2020-07-18T03:33:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b167fc700 1 -- 172.21.15.47:0/145213594 learned_addr learned my addr 172.21.15.47:0/145213594 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b177fe700 1 --2- 172.21.15.47:0/145213594 >> v2:172.21.15.18:3300/0 conn(0x7f1b18147e40 0x7f1b18143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091537933 0 0) 0x7f1b180b57e0 con 0x7f1b18147a00 2020-07-18T03:33:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b04003240 con 0x7f1b18147a00 2020-07-18T03:33:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3136234715 0 0) 0x7f1b04003240 con 0x7f1b18147a00 2020-07-18T03:33:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 >> v2:172.21.15.18:3300/0 conn(0x7f1b18147e40 msgr2=0x7f1b18143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 --2- 172.21.15.47:0/145213594 >> v2:172.21.15.18:3300/0 conn(0x7f1b18147e40 0x7f1b18143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 >> v1:172.21.15.18:6789/0 conn(0x7f1b180ad100 legacy=0x7f1b180b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180b5a80 con 0x7f1b18147a00 2020-07-18T03:33:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f1b08001a50 con 0x7f1b18147a00 2020-07-18T03:33:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 >> v1:172.21.15.47:6789/0 conn(0x7f1b18147a00 legacy=0x7f1b181433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.883+0000 7f1b15ffb700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b04003730 0x7f1b04003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b040044e0 0x7f1b04004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b04004e50 0x7f1b0400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180ae530 con 0x7f1b04003730 2020-07-18T03:33:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04004350 con 0x7f1b040044e0 2020-07-18T03:33:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0400bc50 con 0x7f1b04004e50 2020-07-18T03:33:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b040044e0 0x7f1b04004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b177fe700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b04004e50 0x7f1b0400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b16ffd700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b04003730 0x7f1b04003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 -- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b04004e50 msgr2=0x7f1b0400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b04004e50 0x7f1b0400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 -- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b04003730 msgr2=0x7f1b04003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b04003730 0x7f1b04003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 -- 172.21.15.47:0/145213594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04003240 con 0x7f1b040044e0 2020-07-18T03:33:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b16ffd700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b04003730 0x7f1b04003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b177fe700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b04004e50 0x7f1b0400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b177fe700 1 -- 172.21.15.47:0/145213594 reap_dead start 2020-07-18T03:33:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b167fc700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b040044e0 0x7f1b04004960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f1b08001760 tx=0x7f1b180b9e60).ready entity=mon.1 client_cookie=7c1479bf5839f760 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b08005b80 con 0x7f1b040044e0 2020-07-18T03:33:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08005ce0 con 0x7f1b040044e0 2020-07-18T03:33:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b15ffb700 1 -- 172.21.15.47:0/145213594 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b080072c0 con 0x7f1b040044e0 2020-07-18T03:33:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/145213594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b040044e0 msgr2=0x7f1b04004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b040044e0 0x7f1b04004960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f1b08001760 tx=0x7f1b180b9e60).stop 2020-07-18T03:33:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/145213594 shutdown_connections 2020-07-18T03:33:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/145213594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b040044e0 0x7f1b04004960 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/145213594 >> 172.21.15.47:0/145213594 conn(0x7f1b180b9340 msgr2=0x7f1b180b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/145213594 shutdown_connections 2020-07-18T03:33:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.884+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/145213594 wait complete. 2020-07-18T03:33:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.886+0000 7f1b1d8b6700 1 Processor -- start 2020-07-18T03:33:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.886+0000 7f1b1d8b6700 1 -- start start 2020-07-18T03:33:05.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b18143a30 0x7f1b181359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b18143e10 0x7f1b180b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b181360b0 con 0x7f1b180b2180 2020-07-18T03:33:05.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b18139720 con 0x7f1b18143a30 2020-07-18T03:33:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18136310 con 0x7f1b18143e10 2020-07-18T03:33:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b18143a30 0x7f1b181359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b18143a30 0x7f1b181359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34592/0 (socket says 172.21.15.47:34592) 2020-07-18T03:33:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 -- 172.21.15.47:0/2549656218 learned_addr learned my addr 172.21.15.47:0/2549656218 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b167fc700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b18143e10 0x7f1b180b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 -- 172.21.15.47:0/2549656218 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b18143e10 msgr2=0x7f1b180b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b18143e10 0x7f1b180b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:05.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 -- 172.21.15.47:0/2549656218 >> v1:172.21.15.47:6789/0 conn(0x7f1b180b2180 legacy=0x7f1b180b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:05.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 -- 172.21.15.47:0/2549656218 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180b32b0 con 0x7f1b18143a30 2020-07-18T03:33:05.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162497060 0 0) 0x7f1b181360b0 con 0x7f1b180b2180 2020-07-18T03:33:05.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b16ffd700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b18143a30 0x7f1b181359f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f1b0c007630 tx=0x7f1b0c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:05.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b0c010680 con 0x7f1b18143a30 2020-07-18T03:33:05.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c010040 con 0x7f1b18143a30 2020-07-18T03:33:05.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b18137210 con 0x7f1b18143a30 2020-07-18T03:33:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b0c0109d0 con 0x7f1b18143a30 2020-07-18T03:33:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.887+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b180b12f0 con 0x7f1b18143a30 2020-07-18T03:33:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.888+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b18057450 con 0x7f1b18143a30 2020-07-18T03:33:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.888+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1b0c010b30 con 0x7f1b18143a30 2020-07-18T03:33:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.889+0000 7f1afeffd700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1af404fe60 0x7f1af4052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.889+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1b0c067840 con 0x7f1b18143a30 2020-07-18T03:33:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.889+0000 7f1b167fc700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1af404fe60 0x7f1af4052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.889+0000 7f1b167fc700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1af404fe60 0x7f1af4052320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f1b08019000 tx=0x7f1b08015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:05.892+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1b0c03ffd0 con 0x7f1b18143a30 2020-07-18T03:33:06.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.109+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b1806a100 con 0x7f1b18143a30 2020-07-18T03:33:06.111 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.109+0000 7f1afeffd700 1 -- 172.21.15.47:0/2549656218 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f1b0c0435f0 con 0x7f1b18143a30 2020-07-18T03:33:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:06.112 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/848402287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":848402287},{"type":"v1","addr":"172.21.15.18:6833","nonce":848402287}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:00.941352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2273595707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2273595707},{"type":"v1","addr":"172.21.15.47:6835","nonce":2273595707}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:06.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1af404fe60 msgr2=0x7f1af4052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:06.113 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1af404fe60 0x7f1af4052320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f1b08019000 tx=0x7f1b08015000).stop 2020-07-18T03:33:06.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b18143a30 msgr2=0x7f1b181359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:06.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b18143a30 0x7f1b181359f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f1b0c007630 tx=0x7f1b0c009040).stop 2020-07-18T03:33:06.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 shutdown_connections 2020-07-18T03:33:06.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1af404fe60 0x7f1af4052320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:06.114 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b18143a30 0x7f1b181359f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:06.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 --2- 172.21.15.47:0/2549656218 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1b18143e10 0x7f1b180b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:06.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.110+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 >> 172.21.15.47:0/2549656218 conn(0x7f1b180b9340 msgr2=0x7f1b1805f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:06.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.111+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 shutdown_connections 2020-07-18T03:33:06.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:06.111+0000 7f1b1d8b6700 1 -- 172.21.15.47:0/2549656218 wait complete. 2020-07-18T03:33:06.115 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2020-07-18T03:33:06.324 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:32:59.940158+0000', 'modified': '2020-07-18T03:33:00.941352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6295': {'gid': 6295, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/2273595707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 2273595707}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 2273595707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:33:06.326 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:33:06.326 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:33:06.466 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6295,"want_state":"up:active","state":"up:active","rank_uptime":6.5144822810000003,"mdsmap_epoch":29,"osdmap_epoch":66,"osdmap_epoch_barrier":64,"uptime":11.903288908} 2020-07-18T03:33:06.467 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6295, "mdsmap_epoch": 29, "osdmap_epoch": 66, "osdmap_epoch_barrier": 64, "rank_uptime": 6.514482281, "state": "up:active", "uptime": 11.903288908, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:33:06.468 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:33:06.468 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:33:06.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.470 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:33:06.487 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:33:06.488 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:06.488 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:33:06.488 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:06.489 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:33:06.489 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:06.489 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:06.490 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:06.490 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:33:06.491 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:06.491 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:33:06.491 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42404sec preferred_lft 42404sec 2020-07-18T03:33:06.492 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:33:06.492 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:06.492 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:06.493 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:06.493 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:06.493 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:06.494 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:33:06.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.496 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:33:06.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.561 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:33:06.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.636 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:33:06.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.708 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:33:06.780 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:33:06.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.782 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:33:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:33:06.839 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:33:06.839 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:33:06.840 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:33:06.840 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:33:06.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.841 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:33:06.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.912 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:33:06.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:06.985 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:33:07.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.059 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:33:07.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.114 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:33:07.168 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 2020-07-18T03:33:07.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.170 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:33:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 2020-07-18T03:33:07.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.225 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:33:07.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.297 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 1' 2020-07-18T03:33:07.369 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:33:07.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.371 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.1' 2020-07-18T03:33:07.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.442 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:33:07.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.535 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:33:07.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.594 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:33:07.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.645 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:33:07.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.695 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 up' 2020-07-18T03:33:07.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:07.737 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.1 master ceph-brx' 2020-07-18T03:33:07.849 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:33:07.850 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:33:07.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:07.851 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:33:07.872 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:33:07.943 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:33:07.945 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:33:08.025 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:33:08.029 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:33:08.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:08.031 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:33:08.084 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:33:08.085 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:33:08.127 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:33:08.159 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:33:08.159 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:33:08.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:33:08.223+0000 7f533d8951c0 -1 init, newargv = 0x55f3d356f850 newargc=15 2020-07-18T03:33:08.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[46088]: starting ceph client 2020-07-18T03:33:08.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[46088]: starting fuse 2020-07-18T03:33:08.234 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:33:08.236 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:33:08.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:08.237 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:33:08.291 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:33:08.292 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:33:08.293 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:33:08.293 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:33:08.294 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:33:08.294 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:33:08.295 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:33:08.296 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:08.296 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:33:08.297 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:33:08.297 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:33:08.300 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:33:08.301 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:08.301 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:33:08.302 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:33:08.303 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:33:08.303 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:33:08.304 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:08.304 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:33:08.305 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:33:08.306 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:33:08.307 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:33:08.307 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:33:08.308 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:33:08.308 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:33:08.309 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:33:08.309 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:08.310 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:33:08.310 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:33:08.423 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.46088.asok 2020-07-18T03:33:08.604 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46088.asok 2020-07-18T03:33:08.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:08.606 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46088.asok status 2020-07-18T03:33:08.757 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:33:08.757 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:33:08.758 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:33:08.758 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:33:08.758 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:33:08.759 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:33:08.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:33:08.759 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "46088", 2020-07-18T03:33:08.759 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:33:08.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:08.760 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:33:08.760 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:33:08.760 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6484, 2020-07-18T03:33:08.760 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:33:08.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:33:08.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:33:08.761 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6484 2020-07-18T03:33:08.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:08.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:33:08.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:33:08.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:33:08.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1325450171 2020-07-18T03:33:08.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:33:08.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:08.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:33:08.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:33:08.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:33:08.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1325450171 2020-07-18T03:33:08.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:08.769 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6484 192.168.0.1:0/1325450171", 2020-07-18T03:33:08.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/1325450171", 2020-07-18T03:33:08.769 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:33:08.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 29, 2020-07-18T03:33:08.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 66, 2020-07-18T03:33:08.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:33:08.770 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:33:08.771 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:33:08.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:08.773 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:33:08.774 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:33:08.789 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:33:08.789 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:33:08.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:08.791 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:33:08.861 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:33:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.014+0000 7f5d464ca700 1 Processor -- start 2020-07-18T03:33:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.014+0000 7f5d464ca700 1 -- start start 2020-07-18T03:33:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d464ca700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 0x7f5d40143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d464ca700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d40147330 con 0x7f5d40147a00 2020-07-18T03:33:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d464ca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d400b57e0 con 0x7f5d400ad100 2020-07-18T03:33:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d464ca700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400ae530 con 0x7f5d40147e40 2020-07-18T03:33:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5d400ad100 0x7f5d400b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43940/0 (socket says 172.21.15.47:43940) 2020-07-18T03:33:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3ffff700 1 -- 172.21.15.47:0/1084310258 learned_addr learned my addr 172.21.15.47:0/1084310258 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:09.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d44a67700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 0x7f5d40143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817444356 0 0) 0x7f5d400b57e0 con 0x7f5d400ad100 2020-07-18T03:33:09.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c003240 con 0x7f5d400ad100 2020-07-18T03:33:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1152417947 0 0) 0x7f5d2c003240 con 0x7f5d400ad100 2020-07-18T03:33:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 msgr2=0x7f5d40143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 0x7f5d40143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 >> v1:172.21.15.18:6789/0 conn(0x7f5d40147a00 legacy=0x7f5d401433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400b5a80 con 0x7f5d400ad100 2020-07-18T03:33:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5d300016d0 con 0x7f5d400ad100 2020-07-18T03:33:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 >> v1:172.21.15.47:6789/0 conn(0x7f5d400ad100 legacy=0x7f5d400b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c003730 0x7f5d2c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 0x7f5d2c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c0049d0 0x7f5d2c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400ae530 con 0x7f5d2c003730 2020-07-18T03:33:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0041d0 con 0x7f5d2c0049d0 2020-07-18T03:33:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.015+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c00b730 con 0x7f5d40147e40 2020-07-18T03:33:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c0049d0 0x7f5d2c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d3ffff700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c003730 0x7f5d2c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d3f7fe700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 0x7f5d2c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 -- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 msgr2=0x7f5d2c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5d40147e40 0x7f5d2c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 -- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c003730 msgr2=0x7f5d2c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c003730 0x7f5d2c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 -- 172.21.15.47:0/1084310258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c003240 con 0x7f5d2c0049d0 2020-07-18T03:33:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c0049d0 0x7f5d2c004e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f5d3800c000 tx=0x7f5d380091d0).ready entity=mon.1 client_cookie=635003fca72d0a9a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d3800eb80 con 0x7f5d2c0049d0 2020-07-18T03:33:09.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800ece0 con 0x7f5d2c0049d0 2020-07-18T03:33:09.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d3effd700 1 -- 172.21.15.47:0/1084310258 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d380125e0 con 0x7f5d2c0049d0 2020-07-18T03:33:09.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d464ca700 1 -- 172.21.15.47:0/1084310258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c0049d0 msgr2=0x7f5d2c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d464ca700 1 --2- 172.21.15.47:0/1084310258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c0049d0 0x7f5d2c004e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f5d3800c000 tx=0x7f5d380091d0).stop 2020-07-18T03:33:09.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d44a67700 1 -- 172.21.15.47:0/1084310258 reap_dead start 2020-07-18T03:33:09.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d464ca700 1 -- 172.21.15.47:0/1084310258 shutdown_connections 2020-07-18T03:33:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d464ca700 1 -- 172.21.15.47:0/1084310258 >> 172.21.15.47:0/1084310258 conn(0x7f5d400b9340 msgr2=0x7f5d400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d464ca700 1 -- 172.21.15.47:0/1084310258 shutdown_connections 2020-07-18T03:33:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.016+0000 7f5d464ca700 1 -- 172.21.15.47:0/1084310258 wait complete. 2020-07-18T03:33:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.018+0000 7f5d464ca700 1 Processor -- start 2020-07-18T03:33:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.018+0000 7f5d464ca700 1 -- start start 2020-07-18T03:33:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.018+0000 7f5d464ca700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d40066450 0x7f5d400b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d464ca700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4013cfe0 con 0x7f5d400b66e0 2020-07-18T03:33:09.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d464ca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4013cb40 con 0x7f5d40066070 2020-07-18T03:33:09.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d464ca700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d400b6b80 con 0x7f5d40066450 2020-07-18T03:33:09.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d40066450 0x7f5d400b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5d40066450 0x7f5d400b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34634/0 (socket says 172.21.15.47:34634) 2020-07-18T03:33:09.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 -- 172.21.15.47:0/697210821 learned_addr learned my addr 172.21.15.47:0/697210821 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:09.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 -- 172.21.15.47:0/697210821 >> v1:172.21.15.47:6789/0 conn(0x7f5d40066070 legacy=0x7f5d40138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 -- 172.21.15.47:0/697210821 >> v1:172.21.15.18:6789/0 conn(0x7f5d400b66e0 legacy=0x7f5d400b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254633937 0 0) 0x7f5d4013cb40 con 0x7f5d40066070 2020-07-18T03:33:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 -- 172.21.15.47:0/697210821 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40139540 con 0x7f5d40066450 2020-07-18T03:33:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d3f7fe700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d40066450 0x7f5d400b1530 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f5d3400d000 tx=0x7f5d34007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d34007400 con 0x7f5d40066450 2020-07-18T03:33:09.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d340064f0 con 0x7f5d40066450 2020-07-18T03:33:09.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d34006850 con 0x7f5d40066450 2020-07-18T03:33:09.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d40139050 con 0x7f5d40066450 2020-07-18T03:33:09.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.019+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d400b3000 con 0x7f5d40066450 2020-07-18T03:33:09.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.020+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4013ccd0 con 0x7f5d40066450 2020-07-18T03:33:09.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.020+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5d340069b0 con 0x7f5d40066450 2020-07-18T03:33:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.021+0000 7f5d1ffff700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d1804fe40 0x7f5d18052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.021+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5d340672f0 con 0x7f5d40066450 2020-07-18T03:33:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.021+0000 7f5d3ffff700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d1804fe40 0x7f5d18052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.021+0000 7f5d3ffff700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d1804fe40 0x7f5d18052300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f5d30001b20 tx=0x7f5d400b03e0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.024+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5d3403fa50 con 0x7f5d40066450 2020-07-18T03:33:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.181+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d40057450 con 0x7f5d40066450 2020-07-18T03:33:09.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.182+0000 7f5d1ffff700 1 -- 172.21.15.47:0/697210821 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16277 (secure 0 0 0) 0x7f5d34043070 con 0x7f5d40066450 2020-07-18T03:33:09.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:09.185 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":66,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:33:01.945196+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:32:57.493054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-07-18T03:32:58.764476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:33:09.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.183+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d1804fe40 msgr2=0x7f5d18052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.183+0000 7f5d464ca700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d1804fe40 0x7f5d18052300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f5d30001b20 tx=0x7f5d400b03e0).stop 2020-07-18T03:33:09.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.183+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d40066450 msgr2=0x7f5d400b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.183+0000 7f5d464ca700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d40066450 0x7f5d400b1530 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f5d3400d000 tx=0x7f5d34007b70).stop 2020-07-18T03:33:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.184+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 shutdown_connections 2020-07-18T03:33:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.184+0000 7f5d464ca700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5d1804fe40 0x7f5d18052300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.184+0000 7f5d464ca700 1 --2- 172.21.15.47:0/697210821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d40066450 0x7f5d400b1530 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.184+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 >> 172.21.15.47:0/697210821 conn(0x7f5d400b9340 msgr2=0x7f5d4005f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.184+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 shutdown_connections 2020-07-18T03:33:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.184+0000 7f5d464ca700 1 -- 172.21.15.47:0/697210821 wait complete. 2020-07-18T03:33:09.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:32:57.493054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:32:58.764476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:33:09.198 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:33:09.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.315+0000 7ff865bf4700 1 Processor -- start 2020-07-18T03:33:09.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff865bf4700 1 -- start start 2020-07-18T03:33:09.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff865bf4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8600ad330 0x7ff860143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff865bf4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 0x7ff8600b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff865bf4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff860147de0 con 0x7ff8600b59b0 2020-07-18T03:33:09.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff865bf4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8600ae000 con 0x7ff8600b3a10 2020-07-18T03:33:09.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff865bf4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff860148010 con 0x7ff8600ad330 2020-07-18T03:33:09.322 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff85f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8600ad330 0x7ff860143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff85f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8600ad330 0x7ff860143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34640/0 (socket says 172.21.15.47:34640) 2020-07-18T03:33:09.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.316+0000 7ff85f7fe700 1 -- 172.21.15.47:0/1913476669 learned_addr learned my addr 172.21.15.47:0/1913476669 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:09.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85ffff700 1 --2- 172.21.15.47:0/1913476669 >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 0x7ff8600b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85f7fe700 1 -- 172.21.15.47:0/1913476669 >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 msgr2=0x7ff8600b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85f7fe700 1 --2- 172.21.15.47:0/1913476669 >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 0x7ff8600b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85f7fe700 1 -- 172.21.15.47:0/1913476669 >> v1:172.21.15.18:6789/0 conn(0x7ff8600b59b0 legacy=0x7ff8600b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85f7fe700 1 -- 172.21.15.47:0/1913476669 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600ad710 con 0x7ff8600ad330 2020-07-18T03:33:09.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85f7fe700 1 --2- 172.21.15.47:0/1913476669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8600ad330 0x7ff860143640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7ff8600ae4e0 tx=0x7ff850006e90).ready entity=mon.1 client_cookie=40e7615629b26e08 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85e7fc700 1 -- 172.21.15.47:0/1913476669 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff850009930 con 0x7ff8600ad330 2020-07-18T03:33:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85e7fc700 1 -- 172.21.15.47:0/1913476669 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850009a90 con 0x7ff8600ad330 2020-07-18T03:33:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff85e7fc700 1 -- 172.21.15.47:0/1913476669 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff850009e30 con 0x7ff8600ad330 2020-07-18T03:33:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 -- 172.21.15.47:0/1913476669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8600ad330 msgr2=0x7ff860143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 --2- 172.21.15.47:0/1913476669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8600ad330 0x7ff860143640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7ff8600ae4e0 tx=0x7ff850006e90).stop 2020-07-18T03:33:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 -- 172.21.15.47:0/1913476669 shutdown_connections 2020-07-18T03:33:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 --2- 172.21.15.47:0/1913476669 >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 0x7ff8600b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 --2- 172.21.15.47:0/1913476669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8600ad330 0x7ff860143640 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 -- 172.21.15.47:0/1913476669 >> 172.21.15.47:0/1913476669 conn(0x7ff86005f5a0 msgr2=0x7ff86005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 -- 172.21.15.47:0/1913476669 shutdown_connections 2020-07-18T03:33:09.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.317+0000 7ff865bf4700 1 -- 172.21.15.47:0/1913476669 wait complete. 2020-07-18T03:33:09.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 Processor -- start 2020-07-18T03:33:09.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 -- start start 2020-07-18T03:33:09.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 0x7ff8601471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8600b59b0 0x7ff860146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff860146e20 con 0x7ff860156ee0 2020-07-18T03:33:09.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff860153f20 con 0x7ff8600b3a10 2020-07-18T03:33:09.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.319+0000 7ff865bf4700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff860065900 con 0x7ff8600b59b0 2020-07-18T03:33:09.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff860156ee0 0x7ff860061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43970/0 (socket says 172.21.15.47:43970) 2020-07-18T03:33:09.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85ffff700 1 -- 172.21.15.47:0/3657565572 learned_addr learned my addr 172.21.15.47:0/3657565572 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:09.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85f7fe700 1 --2- 172.21.15.47:0/3657565572 >> v2:172.21.15.18:3300/0 conn(0x7ff8600b3a10 0x7ff8601471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85effd700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8600b59b0 0x7ff860146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167177495 0 0) 0x7ff858001210 con 0x7ff860156ee0 2020-07-18T03:33:09.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff838003240 con 0x7ff860156ee0 2020-07-18T03:33:09.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85f7fe700 1 -- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8600b59b0 msgr2=0x7ff860146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85f7fe700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8600b59b0 0x7ff860146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85f7fe700 1 -- 172.21.15.47:0/3657565572 >> v1:172.21.15.47:6789/0 conn(0x7ff860156ee0 legacy=0x7ff860061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3214634216 0 0) 0x7ff838003240 con 0x7ff860156ee0 2020-07-18T03:33:09.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85f7fe700 1 -- 172.21.15.47:0/3657565572 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff860065ba0 con 0x7ff8600b3a10 2020-07-18T03:33:09.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85effd700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8600b59b0 0x7ff860146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:09.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff85f7fe700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8600b3a10 0x7ff8601471f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7ff858001b40 tx=0x7ff850015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8580019a0 con 0x7ff8600b3a10 2020-07-18T03:33:09.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8500067a0 con 0x7ff8600b3a10 2020-07-18T03:33:09.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff860062290 con 0x7ff8600b3a10 2020-07-18T03:33:09.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.320+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff850011ea0 con 0x7ff8600b3a10 2020-07-18T03:33:09.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.321+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff860145c30 con 0x7ff8600b3a10 2020-07-18T03:33:09.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.322+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86005b120 con 0x7ff8600b3a10 2020-07-18T03:33:09.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.322+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff85001a040 con 0x7ff8600b3a10 2020-07-18T03:33:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.322+0000 7ff83f7fe700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff838050310 0x7ff8380527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.322+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff850068980 con 0x7ff8600b3a10 2020-07-18T03:33:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.322+0000 7ff85effd700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff838050310 0x7ff8380527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.322+0000 7ff85effd700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff838050310 0x7ff8380527d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7ff85400d000 tx=0x7ff854007b50).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.326+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff850041140 con 0x7ff8600b3a10 2020-07-18T03:33:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.484+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff8600690d0 con 0x7ff8600b3a10 2020-07-18T03:33:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.484+0000 7ff83f7fe700 1 -- 172.21.15.47:0/3657565572 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff850023070 con 0x7ff8600b3a10 2020-07-18T03:33:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff838050310 msgr2=0x7ff8380527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff838050310 0x7ff8380527d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7ff85400d000 tx=0x7ff854007b50).stop 2020-07-18T03:33:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8600b3a10 msgr2=0x7ff8601471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8600b3a10 0x7ff8601471f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7ff858001b40 tx=0x7ff850015000).stop 2020-07-18T03:33:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 shutdown_connections 2020-07-18T03:33:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff838050310 0x7ff8380527d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8600b3a10 0x7ff8601471f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 --2- 172.21.15.47:0/3657565572 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff8600b59b0 0x7ff860146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 >> 172.21.15.47:0/3657565572 conn(0x7ff86005f5a0 msgr2=0x7ff8601427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 shutdown_connections 2020-07-18T03:33:09.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.486+0000 7ff865bf4700 1 -- 172.21.15.47:0/3657565572 wait complete. 2020-07-18T03:33:09.496 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:33:09.499 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:33:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.615+0000 7f384e3ec700 1 Processor -- start 2020-07-18T03:33:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.615+0000 7f384e3ec700 1 -- start start 2020-07-18T03:33:09.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384e3ec700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3848143cd0 0x7f3848061c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384e3ec700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f38481448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384e3ec700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3848062c30 con 0x7f38481472e0 2020-07-18T03:33:09.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384e3ec700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38481440b0 con 0x7f3848143cd0 2020-07-18T03:33:09.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384e3ec700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38480ad840 con 0x7f3848144460 2020-07-18T03:33:09.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384c989700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38481472e0 0x7f3848147760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43974/0 (socket says 172.21.15.47:43974) 2020-07-18T03:33:09.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f384c989700 1 -- 172.21.15.47:0/3744968386 learned_addr learned my addr 172.21.15.47:0/3744968386 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:09.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f38477fe700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f38481448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3847fff700 1 --2- 172.21.15.47:0/3744968386 >> v2:172.21.15.18:3300/0 conn(0x7f3848143cd0 0x7f3848061c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516282567 0 0) 0x7f3848062c30 con 0x7f38481472e0 2020-07-18T03:33:09.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3834003240 con 0x7f38481472e0 2020-07-18T03:33:09.623 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1291288999 0 0) 0x7f3834003240 con 0x7f38481472e0 2020-07-18T03:33:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 msgr2=0x7f38481448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f38481448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 >> v2:172.21.15.18:3300/0 conn(0x7f3848143cd0 msgr2=0x7f3848061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 --2- 172.21.15.47:0/3744968386 >> v2:172.21.15.18:3300/0 conn(0x7f3848143cd0 0x7f3848061c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480adae0 con 0x7f38481472e0 2020-07-18T03:33:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f3840001a50 con 0x7f38481472e0 2020-07-18T03:33:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f38477fe700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f38481448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3847fff700 1 --2- 172.21.15.47:0/3744968386 >> v2:172.21.15.18:3300/0 conn(0x7f3848143cd0 0x7f3848061c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 >> v1:172.21.15.47:6789/0 conn(0x7f38481472e0 legacy=0x7f3848147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.616+0000 7f3846ffd700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3834003730 0x7f3834003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38340044e0 0x7f3834004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f3834004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38481440b0 con 0x7f3834003730 2020-07-18T03:33:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480ad840 con 0x7f38340044e0 2020-07-18T03:33:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3834004250 con 0x7f3848144460 2020-07-18T03:33:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38340044e0 0x7f3834004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3847fff700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3834003730 0x7f3834003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384c989700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f3834004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 -- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 msgr2=0x7f3834004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3848144460 0x7f3834004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 -- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3834003730 msgr2=0x7f3834003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3834003730 0x7f3834003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 -- 172.21.15.47:0/3744968386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3834003240 con 0x7f38340044e0 2020-07-18T03:33:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f38477fe700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38340044e0 0x7f3834004960 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f383c00c000 tx=0x7f383c008040).ready entity=mon.1 client_cookie=54e319f73ff085b7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383c00ed70 con 0x7f38340044e0 2020-07-18T03:33:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00be90 con 0x7f38340044e0 2020-07-18T03:33:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f3846ffd700 1 -- 172.21.15.47:0/3744968386 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383c01a660 con 0x7f38340044e0 2020-07-18T03:33:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384e3ec700 1 -- 172.21.15.47:0/3744968386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38340044e0 msgr2=0x7f3834004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384e3ec700 1 --2- 172.21.15.47:0/3744968386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38340044e0 0x7f3834004960 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f383c00c000 tx=0x7f383c008040).stop 2020-07-18T03:33:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384c989700 1 -- 172.21.15.47:0/3744968386 reap_dead start 2020-07-18T03:33:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384e3ec700 1 -- 172.21.15.47:0/3744968386 shutdown_connections 2020-07-18T03:33:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384e3ec700 1 -- 172.21.15.47:0/3744968386 >> 172.21.15.47:0/3744968386 conn(0x7f384805f5a0 msgr2=0x7f384805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384e3ec700 1 -- 172.21.15.47:0/3744968386 shutdown_connections 2020-07-18T03:33:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.617+0000 7f384e3ec700 1 -- 172.21.15.47:0/3744968386 wait complete. 2020-07-18T03:33:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.619+0000 7f384e3ec700 1 Processor -- start 2020-07-18T03:33:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.619+0000 7f384e3ec700 1 -- start start 2020-07-18T03:33:09.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f384812b150 0x7f384812a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f384812beb0 0x7f38480b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384812b530 con 0x7f3848129c00 2020-07-18T03:33:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38481245a0 con 0x7f384812beb0 2020-07-18T03:33:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38480b9960 con 0x7f384812b150 2020-07-18T03:33:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f384812beb0 0x7f38480b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f384812beb0 0x7f38480b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34670/0 (socket says 172.21.15.47:34670) 2020-07-18T03:33:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f3847fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f384812b150 0x7f384812a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 -- 172.21.15.47:0/2315839093 learned_addr learned my addr 172.21.15.47:0/2315839093 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 -- 172.21.15.47:0/2315839093 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f384812b150 msgr2=0x7f384812a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f384812b150 0x7f384812a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 -- 172.21.15.47:0/2315839093 >> v1:172.21.15.18:6789/0 conn(0x7f3848129c00 legacy=0x7f3848063fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 -- 172.21.15.47:0/2315839093 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480b9c00 con 0x7f384812beb0 2020-07-18T03:33:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384c989700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f384812beb0 0x7f38480b71f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f3840000b20 tx=0x7f3840007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384000fe20 con 0x7f384812beb0 2020-07-18T03:33:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3840012350 con 0x7f384812beb0 2020-07-18T03:33:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38400126b0 con 0x7f384812beb0 2020-07-18T03:33:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38480b9ea0 con 0x7f384812beb0 2020-07-18T03:33:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.620+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480646c0 con 0x7f384812beb0 2020-07-18T03:33:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.621+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38480ba030 con 0x7f384812beb0 2020-07-18T03:33:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.621+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3840012810 con 0x7f384812beb0 2020-07-18T03:33:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.622+0000 7f3827fff700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3820050040 0x7f3820052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.622+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f38400675d0 con 0x7f384812beb0 2020-07-18T03:33:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.622+0000 7f3847fff700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3820050040 0x7f3820052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.622+0000 7f3847fff700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3820050040 0x7f3820052500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f3838008000 tx=0x7f3838008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.625+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f384003fd60 con 0x7f384812beb0 2020-07-18T03:33:09.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.783+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f384805b120 con 0x7f384812beb0 2020-07-18T03:33:09.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.783+0000 7f3827fff700 1 -- 172.21.15.47:0/2315839093 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3840019020 con 0x7f384812beb0 2020-07-18T03:33:09.786 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:33:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.784+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3820050040 msgr2=0x7f3820052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.784+0000 7f384e3ec700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3820050040 0x7f3820052500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f3838008000 tx=0x7f3838008040).stop 2020-07-18T03:33:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.784+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f384812beb0 msgr2=0x7f38480b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.784+0000 7f384e3ec700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f384812beb0 0x7f38480b71f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f3840000b20 tx=0x7f3840007ad0).stop 2020-07-18T03:33:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 shutdown_connections 2020-07-18T03:33:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3820050040 0x7f3820052500 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f384812beb0 0x7f38480b71f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 --2- 172.21.15.47:0/2315839093 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f384812b150 0x7f384812a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 >> 172.21.15.47:0/2315839093 conn(0x7f384805f5a0 msgr2=0x7f38480b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 shutdown_connections 2020-07-18T03:33:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.785+0000 7f384e3ec700 1 -- 172.21.15.47:0/2315839093 wait complete. 2020-07-18T03:33:09.797 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:33:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.912+0000 7f592210d700 1 Processor -- start 2020-07-18T03:33:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.912+0000 7f592210d700 1 -- start start 2020-07-18T03:33:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f592210d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f591c0ad330 0x7f591c143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f592210d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f592210d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f591c0b6300 0x7f591c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f592210d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c147de0 con 0x7f591c0b59b0 2020-07-18T03:33:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f592210d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0ae000 con 0x7f591c0b6300 2020-07-18T03:33:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f592210d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c148010 con 0x7f591c0ad330 2020-07-18T03:33:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f591c0b6300 0x7f591c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f591c0b6300 0x7f591c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34678/0 (socket says 172.21.15.47:34678) 2020-07-18T03:33:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 -- 172.21.15.47:0/4105267582 learned_addr learned my addr 172.21.15.47:0/4105267582 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591b7fe700 1 --2- 172.21.15.47:0/4105267582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f591c0ad330 0x7f591c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591affd700 1 --2- 172.21.15.47:0/4105267582 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 -- 172.21.15.47:0/4105267582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f591c0ad330 msgr2=0x7f591c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 --2- 172.21.15.47:0/4105267582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f591c0ad330 0x7f591c143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 -- 172.21.15.47:0/4105267582 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 msgr2=0x7f591c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 --2- 172.21.15.47:0/4105267582 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.913+0000 7f591bfff700 1 -- 172.21.15.47:0/4105267582 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0ad710 con 0x7f591c0b6300 2020-07-18T03:33:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f591bfff700 1 --2- 172.21.15.47:0/4105267582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0b6300 0x7f591c0b3a10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f59140032b0 tx=0x7f5914009270).ready entity=mon.1 client_cookie=9bc47d78130359a8 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f591a7fc700 1 -- 172.21.15.47:0/4105267582 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591400bc00 con 0x7f591c0b6300 2020-07-18T03:33:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f591a7fc700 1 -- 172.21.15.47:0/4105267582 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591400bd60 con 0x7f591c0b6300 2020-07-18T03:33:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f591a7fc700 1 -- 172.21.15.47:0/4105267582 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5914011400 con 0x7f591c0b6300 2020-07-18T03:33:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 -- 172.21.15.47:0/4105267582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0b6300 msgr2=0x7f591c0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 --2- 172.21.15.47:0/4105267582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0b6300 0x7f591c0b3a10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f59140032b0 tx=0x7f5914009270).stop 2020-07-18T03:33:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 -- 172.21.15.47:0/4105267582 shutdown_connections 2020-07-18T03:33:09.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 --2- 172.21.15.47:0/4105267582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0b6300 0x7f591c0b3a10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 --2- 172.21.15.47:0/4105267582 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f591c0ad330 0x7f591c143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.928 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 --2- 172.21.15.47:0/4105267582 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 -- 172.21.15.47:0/4105267582 >> 172.21.15.47:0/4105267582 conn(0x7f591c05f5a0 msgr2=0x7f591c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 -- 172.21.15.47:0/4105267582 shutdown_connections 2020-07-18T03:33:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.914+0000 7f592210d700 1 -- 172.21.15.47:0/4105267582 wait complete. 2020-07-18T03:33:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.915+0000 7f592210d700 1 Processor -- start 2020-07-18T03:33:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f592210d700 1 -- start start 2020-07-18T03:33:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f592210d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f591c0ad330 0x7f591c13a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f592210d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f592210d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c0b3050 con 0x7f591c0b09d0 2020-07-18T03:33:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f592210d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c14b040 con 0x7f591c0b59b0 2020-07-18T03:33:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f592210d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c14b180 con 0x7f591c0ad330 2020-07-18T03:33:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f591b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f591c0ad330 0x7f591c13a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f591b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f591c0ad330 0x7f591c13a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34680/0 (socket says 172.21.15.47:34680) 2020-07-18T03:33:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.916+0000 7f591b7fe700 1 -- 172.21.15.47:0/1513326865 learned_addr learned my addr 172.21.15.47:0/1513326865 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f591b7fe700 1 -- 172.21.15.47:0/1513326865 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 msgr2=0x7f591c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f591affd700 1 --2- 172.21.15.47:0/1513326865 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f591b7fe700 1 --2- 172.21.15.47:0/1513326865 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f591b7fe700 1 -- 172.21.15.47:0/1513326865 >> v1:172.21.15.18:6789/0 conn(0x7f591c0b09d0 legacy=0x7f591c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f591b7fe700 1 -- 172.21.15.47:0/1513326865 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c14a310 con 0x7f591c0ad330 2020-07-18T03:33:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f591b7fe700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0ad330 0x7f591c13a180 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f591c0b3320 tx=0x7f590c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f590c00ce00 con 0x7f591c0ad330 2020-07-18T03:33:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c007710 con 0x7f591c0ad330 2020-07-18T03:33:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c14a5c0 con 0x7f591c0ad330 2020-07-18T03:33:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f590c007a70 con 0x7f591c0ad330 2020-07-18T03:33:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.917+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c0b2e20 con 0x7f591c0ad330 2020-07-18T03:33:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.918+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c0b34f0 con 0x7f591c0ad330 2020-07-18T03:33:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.918+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f590c007ca0 con 0x7f591c0ad330 2020-07-18T03:33:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.918+0000 7f58fb7fe700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f58f404fff0 0x7f58f40524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.919+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f590c067e40 con 0x7f591c0ad330 2020-07-18T03:33:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.919+0000 7f591affd700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f58f404fff0 0x7f58f40524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.919+0000 7f591affd700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f58f404fff0 0x7f58f40524b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f5910008000 tx=0x7f5910008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:09.922+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f590c040570 con 0x7f591c0ad330 2020-07-18T03:33:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.077+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f591c05b120 con 0x7f591c0ad330 2020-07-18T03:33:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.078+0000 7f58fb7fe700 1 -- 172.21.15.47:0/1513326865 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+10 (secure 0 0 0) 0x7f590c006c00 con 0x7f591c0ad330 2020-07-18T03:33:10.080 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:33:10.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f58f404fff0 msgr2=0x7f58f40524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f58f404fff0 0x7f58f40524b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f5910008000 tx=0x7f5910008040).stop 2020-07-18T03:33:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0ad330 msgr2=0x7f591c13a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0ad330 0x7f591c13a180 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f591c0b3320 tx=0x7f590c008040).stop 2020-07-18T03:33:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 shutdown_connections 2020-07-18T03:33:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f58f404fff0 0x7f58f40524b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 --2- 172.21.15.47:0/1513326865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c0ad330 0x7f591c13a180 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 --2- 172.21.15.47:0/1513326865 >> v2:172.21.15.18:3300/0 conn(0x7f591c0b59b0 0x7f591c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 >> 172.21.15.47:0/1513326865 conn(0x7f591c05f5a0 msgr2=0x7f591c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 shutdown_connections 2020-07-18T03:33:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.079+0000 7f592210d700 1 -- 172.21.15.47:0/1513326865 wait complete. 2020-07-18T03:33:10.093 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:33:10.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.208+0000 7f7b9ea0a700 1 Processor -- start 2020-07-18T03:33:10.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.209+0000 7f7b9ea0a700 1 -- start start 2020-07-18T03:33:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.209+0000 7f7b9ea0a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7b98143c50 0x7f7b98140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.209+0000 7f7b9ea0a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98141290 0x7f7b98144e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.209+0000 7f7b9ea0a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b981416e0 con 0x7f7b980ad0d0 2020-07-18T03:33:10.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.209+0000 7f7b9ea0a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980ad570 con 0x7f7b98143c50 2020-07-18T03:33:10.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.209+0000 7f7b9ea0a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980ad810 con 0x7f7b98141290 2020-07-18T03:33:10.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9da08700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7b980ad0d0 0x7f7b9805f760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44008/0 (socket says 172.21.15.47:44008) 2020-07-18T03:33:10.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9da08700 1 -- 172.21.15.47:0/357027621 learned_addr learned my addr 172.21.15.47:0/357027621 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:10.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9d207700 1 --2- 172.21.15.47:0/357027621 >> v2:172.21.15.18:3300/0 conn(0x7f7b98143c50 0x7f7b98140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9e209700 1 --2- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98141290 0x7f7b98144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9ca06700 1 -- 172.21.15.47:0/357027621 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062041387 0 0) 0x7f7b981416e0 con 0x7f7b980ad0d0 2020-07-18T03:33:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9ca06700 1 -- 172.21.15.47:0/357027621 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b80003240 con 0x7f7b980ad0d0 2020-07-18T03:33:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9d207700 1 -- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98141290 msgr2=0x7f7b98144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9d207700 1 --2- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98141290 0x7f7b98144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9ca06700 1 -- 172.21.15.47:0/357027621 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2894881113 0 0) 0x7f7b80003240 con 0x7f7b980ad0d0 2020-07-18T03:33:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9d207700 1 -- 172.21.15.47:0/357027621 >> v1:172.21.15.47:6789/0 conn(0x7f7b980ad0d0 legacy=0x7f7b9805f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9d207700 1 -- 172.21.15.47:0/357027621 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980adab0 con 0x7f7b98143c50 2020-07-18T03:33:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9d207700 1 --2- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b98143c50 0x7f7b98140da0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f7b88001870 tx=0x7f7b94006dd0).ready entity=mon.0 client_cookie=336ac42a7f3f2bd3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.210+0000 7f7b9ca06700 1 -- 172.21.15.47:0/357027621 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b880016d0 con 0x7f7b98143c50 2020-07-18T03:33:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ca06700 1 -- 172.21.15.47:0/357027621 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9400d570 con 0x7f7b98143c50 2020-07-18T03:33:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ca06700 1 -- 172.21.15.47:0/357027621 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9400d6e0 con 0x7f7b98143c50 2020-07-18T03:33:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b98143c50 msgr2=0x7f7b98140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b98143c50 0x7f7b98140da0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f7b88001870 tx=0x7f7b94006dd0).stop 2020-07-18T03:33:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/357027621 shutdown_connections 2020-07-18T03:33:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98141290 0x7f7b98144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/357027621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b98143c50 0x7f7b98140da0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/357027621 >> 172.21.15.47:0/357027621 conn(0x7f7b980b91d0 msgr2=0x7f7b980b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/357027621 shutdown_connections 2020-07-18T03:33:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.211+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/357027621 wait complete. 2020-07-18T03:33:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.212+0000 7f7b9ea0a700 1 Processor -- start 2020-07-18T03:33:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.212+0000 7f7b9ea0a700 1 -- start start 2020-07-18T03:33:10.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9ea0a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b98141290 0x7f7b980b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9ea0a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98151ae0 0x7f7b981522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9ea0a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9812c870 con 0x7f7b980ad0d0 2020-07-18T03:33:10.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9ea0a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b98151f20 con 0x7f7b98141290 2020-07-18T03:33:10.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9ea0a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980af630 con 0x7f7b98151ae0 2020-07-18T03:33:10.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b98141290 0x7f7b980b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b98141290 0x7f7b980b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34696/0 (socket says 172.21.15.47:34696) 2020-07-18T03:33:10.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 -- 172.21.15.47:0/3908859616 learned_addr learned my addr 172.21.15.47:0/3908859616 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:10.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9e209700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98151ae0 0x7f7b981522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 -- 172.21.15.47:0/3908859616 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98151ae0 msgr2=0x7f7b981522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98151ae0 0x7f7b981522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 -- 172.21.15.47:0/3908859616 >> v1:172.21.15.18:6789/0 conn(0x7f7b980ad0d0 legacy=0x7f7b980afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 -- 172.21.15.47:0/3908859616 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980b22a0 con 0x7f7b98141290 2020-07-18T03:33:10.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b9d207700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b98141290 0x7f7b980b1cb0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f7b88001870 tx=0x7f7b94015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.213+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b94011d00 con 0x7f7b98141290 2020-07-18T03:33:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.214+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94011e60 con 0x7f7b98141290 2020-07-18T03:33:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.214+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980b2540 con 0x7f7b98141290 2020-07-18T03:33:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.214+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9401e650 con 0x7f7b98141290 2020-07-18T03:33:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.214+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980b2d00 con 0x7f7b98141290 2020-07-18T03:33:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.215+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980690d0 con 0x7f7b98141290 2020-07-18T03:33:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.215+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7b94025030 con 0x7f7b98141290 2020-07-18T03:33:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.215+0000 7f7b85ffb700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b7004fff0 0x7f7b700524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.215+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7b94069350 con 0x7f7b98141290 2020-07-18T03:33:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.215+0000 7f7b9da08700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b7004fff0 0x7f7b700524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.215+0000 7f7b9da08700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b7004fff0 0x7f7b700524b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7b9812c610 tx=0x7f7b88001e00).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.219+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7b94041aa0 con 0x7f7b98141290 2020-07-18T03:33:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.374+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b98069f30 con 0x7f7b98141290 2020-07-18T03:33:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.375+0000 7f7b85ffb700 1 -- 172.21.15.47:0/3908859616 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7b94045080 con 0x7f7b98141290 2020-07-18T03:33:10.377 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:33:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b7004fff0 msgr2=0x7f7b700524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b7004fff0 0x7f7b700524b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7b9812c610 tx=0x7f7b88001e00).stop 2020-07-18T03:33:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b98141290 msgr2=0x7f7b980b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b98141290 0x7f7b980b1cb0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f7b88001870 tx=0x7f7b94015000).stop 2020-07-18T03:33:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 shutdown_connections 2020-07-18T03:33:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7b7004fff0 0x7f7b700524b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7b98151ae0 0x7f7b981522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 --2- 172.21.15.47:0/3908859616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b98141290 0x7f7b980b1cb0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 >> 172.21.15.47:0/3908859616 conn(0x7f7b980b91d0 msgr2=0x7f7b9805f760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 shutdown_connections 2020-07-18T03:33:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.376+0000 7f7b9ea0a700 1 -- 172.21.15.47:0/3908859616 wait complete. 2020-07-18T03:33:10.389 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:33:10.390 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.502+0000 7f83b043a700 1 Processor -- start 2020-07-18T03:33:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.503+0000 7f83b043a700 1 -- start start 2020-07-18T03:33:10.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.503+0000 7f83b043a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f83a80b41c0 0x7f83a80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.503+0000 7f83b043a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a80b3ac0 con 0x7f83a80ad7c0 2020-07-18T03:33:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.503+0000 7f83b043a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a80b3d20 con 0x7f83a80b5370 2020-07-18T03:33:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.503+0000 7f83b043a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a80b5a10 con 0x7f83a80b41c0 2020-07-18T03:33:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ae1d6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83a80b5370 0x7f83a805fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44022/0 (socket says 172.21.15.47:44022) 2020-07-18T03:33:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ae1d6700 1 -- 172.21.15.47:0/2336887220 learned_addr learned my addr 172.21.15.47:0/2336887220 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad9d5700 1 --2- 172.21.15.47:0/2336887220 >> v2:172.21.15.18:3300/0 conn(0x7f83a80b41c0 0x7f83a80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276879205 0 0) 0x7f83a80b3d20 con 0x7f83a80b5370 2020-07-18T03:33:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8390003240 con 0x7f83a80b5370 2020-07-18T03:33:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1645903194 0 0) 0x7f8390003240 con 0x7f83a80b5370 2020-07-18T03:33:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 >> v2:172.21.15.18:3300/0 conn(0x7f83a80b41c0 msgr2=0x7f83a80ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 --2- 172.21.15.47:0/2336887220 >> v2:172.21.15.18:3300/0 conn(0x7f83a80b41c0 0x7f83a80ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 >> v1:172.21.15.18:6789/0 conn(0x7f83a80ad7c0 legacy=0x7f83a81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80b5750 con 0x7f83a80b5370 2020-07-18T03:33:10.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f839c0016d0 con 0x7f83a80b5370 2020-07-18T03:33:10.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 >> v1:172.21.15.47:6789/0 conn(0x7f83a80b5370 legacy=0x7f83a805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390003730 0x7f8390003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83900044e0 0x7f8390004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8390004e50 0x7f839000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80b5a10 con 0x7f83900044e0 2020-07-18T03:33:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390004350 con 0x7f8390003730 2020-07-18T03:33:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839000bc50 con 0x7f8390004e50 2020-07-18T03:33:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ae1d6700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390003730 0x7f8390003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ad9d5700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83900044e0 0x7f8390004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.504+0000 7f83ae9d7700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8390004e50 0x7f839000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae1d6700 1 -- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8390004e50 msgr2=0x7f839000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae1d6700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8390004e50 0x7f839000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae1d6700 1 -- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83900044e0 msgr2=0x7f8390004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae1d6700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83900044e0 0x7f8390004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae1d6700 1 -- 172.21.15.47:0/2336887220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390003240 con 0x7f8390003730 2020-07-18T03:33:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae9d7700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8390004e50 0x7f839000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae9d7700 1 -- 172.21.15.47:0/2336887220 reap_dead start 2020-07-18T03:33:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ae1d6700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390003730 0x7f8390003b50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f839c0013e0 tx=0x7f83a80b3eb0).ready entity=mon.1 client_cookie=72cfdca50dfff4b5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c0072c0 con 0x7f8390003730 2020-07-18T03:33:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c007420 con 0x7f8390003730 2020-07-18T03:33:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83ad1d4700 1 -- 172.21.15.47:0/2336887220 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839c0057e0 con 0x7f8390003730 2020-07-18T03:33:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 -- 172.21.15.47:0/2336887220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390003730 msgr2=0x7f8390003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390003730 0x7f8390003b50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f839c0013e0 tx=0x7f83a80b3eb0).stop 2020-07-18T03:33:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 -- 172.21.15.47:0/2336887220 shutdown_connections 2020-07-18T03:33:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 --2- 172.21.15.47:0/2336887220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390003730 0x7f8390003b50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 -- 172.21.15.47:0/2336887220 >> 172.21.15.47:0/2336887220 conn(0x7f83a80b9340 msgr2=0x7f83a80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 -- 172.21.15.47:0/2336887220 shutdown_connections 2020-07-18T03:33:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.505+0000 7f83b043a700 1 -- 172.21.15.47:0/2336887220 wait complete. 2020-07-18T03:33:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.506+0000 7f83b043a700 1 Processor -- start 2020-07-18T03:33:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83b043a700 1 -- start start 2020-07-18T03:33:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83b043a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83a813f580 0x7f83a8066200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83b043a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a8065840 con 0x7f83a8061910 2020-07-18T03:33:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83b043a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a8065ae0 con 0x7f83a8064e50 2020-07-18T03:33:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83b043a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a8065e10 con 0x7f83a813f580 2020-07-18T03:33:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83ae1d6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83a8064e50 0x7f83a8145740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44034/0 (socket says 172.21.15.47:44034) 2020-07-18T03:33:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83ae1d6700 1 -- 172.21.15.47:0/314822610 learned_addr learned my addr 172.21.15.47:0/314822610 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.507+0000 7f83ae9d7700 1 --2- 172.21.15.47:0/314822610 >> v2:172.21.15.47:3300/0 conn(0x7f83a813f580 0x7f83a8066200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801430005 0 0) 0x7f83a8065ae0 con 0x7f83a8064e50 2020-07-18T03:33:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8380003240 con 0x7f83a8064e50 2020-07-18T03:33:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f83ae9d7700 1 -- 172.21.15.47:0/314822610 >> v1:172.21.15.47:6789/0 conn(0x7f83a8064e50 legacy=0x7f83a8145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459196509 0 0) 0x7f83a8065840 con 0x7f83a8061910 2020-07-18T03:33:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f83ae9d7700 1 -- 172.21.15.47:0/314822610 >> v1:172.21.15.18:6789/0 conn(0x7f83a8061910 legacy=0x7f83a813ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f83ae9d7700 1 -- 172.21.15.47:0/314822610 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a8065230 con 0x7f83a813f580 2020-07-18T03:33:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f83ae9d7700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a813f580 0x7f83a8066200 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f83a00011d0 tx=0x7f83a0009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83a0010680 con 0x7f83a813f580 2020-07-18T03:33:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a0010040 con 0x7f83a813f580 2020-07-18T03:33:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83a00109d0 con 0x7f83a813f580 2020-07-18T03:33:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a80654e0 con 0x7f83a813f580 2020-07-18T03:33:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.508+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a8143b60 con 0x7f83a813f580 2020-07-18T03:33:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.509+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a805f950 con 0x7f83a813f580 2020-07-18T03:33:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.509+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f83a001b020 con 0x7f83a813f580 2020-07-18T03:33:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.509+0000 7f839a7fc700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8380050340 0x7f8380052800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.509+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f83a0067790 con 0x7f83a813f580 2020-07-18T03:33:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.509+0000 7f83ae1d6700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8380050340 0x7f8380052800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.510+0000 7f83ae1d6700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8380050340 0x7f8380052800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f839c001a50 tx=0x7f839c017000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.513+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f83a003fe80 con 0x7f83a813f580 2020-07-18T03:33:10.724 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.722+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83a8057450 con 0x7f83a813f580 2020-07-18T03:33:10.725 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.722+0000 7f839a7fc700 1 -- 172.21.15.47:0/314822610 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f83a00434a0 con 0x7f83a813f580 2020-07-18T03:33:10.725 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:10.734 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/848402287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":848402287},{"type":"v1","addr":"172.21.15.18:6833","nonce":848402287}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:00.941352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6295":{"gid":6295,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/2273595707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2273595707},{"type":"v1","addr":"172.21.15.47:6835","nonce":2273595707}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8380050340 msgr2=0x7f8380052800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8380050340 0x7f8380052800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f839c001a50 tx=0x7f839c017000).stop 2020-07-18T03:33:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a813f580 msgr2=0x7f83a8066200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a813f580 0x7f83a8066200 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f83a00011d0 tx=0x7f83a0009040).stop 2020-07-18T03:33:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 shutdown_connections 2020-07-18T03:33:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a813f580 0x7f83a8066200 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 --2- 172.21.15.47:0/314822610 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8380050340 0x7f8380052800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 >> 172.21.15.47:0/314822610 conn(0x7f83a80b9340 msgr2=0x7f83a80b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 shutdown_connections 2020-07-18T03:33:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:10.724+0000 7f83b043a700 1 -- 172.21.15.47:0/314822610 wait complete. 2020-07-18T03:33:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2020-07-18T03:33:10.941 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:32:59.940158+0000', 'modified': '2020-07-18T03:33:00.941352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6295': {'gid': 6295, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/2273595707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 2273595707}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 2273595707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:33:10.943 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:33:10.943 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:33:11.081 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6295,"want_state":"up:active","state":"up:active","rank_uptime":11.130324992,"mdsmap_epoch":29,"osdmap_epoch":66,"osdmap_epoch_barrier":64,"uptime":16.519131570999999} 2020-07-18T03:33:11.082 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6295, "mdsmap_epoch": 29, "osdmap_epoch": 66, "osdmap_epoch_barrier": 64, "rank_uptime": 11.130324992, "state": "up:active", "uptime": 16.519131571, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:33:11.082 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:33:11.083 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:33:11.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.084 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:33:11.099 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:33:11.099 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:33:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:33:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.101 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:11.101 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:33:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:33:11.102 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42400sec preferred_lft 42400sec 2020-07-18T03:33:11.102 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:33:11.102 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.102 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:11.102 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:11.103 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:11.103 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:11.103 INFO:teuthology.orchestra.run.smithi047.stdout:19: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:33:11.103 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:4c:91:0b:22:52 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:11.103 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:33:11.104 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.104 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7436:62ff:fe20:c122/64 scope link 2020-07-18T03:33:11.104 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.105 INFO:teuthology.orchestra.run.smithi047.stdout:20: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:33:11.105 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:4c:91:0b:22:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:33:11.105 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::484c:91ff:fe0b:2252/64 scope link 2020-07-18T03:33:11.106 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.107 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:33:11.172 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:33:11.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.174 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:33:11.228 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:33:11.228 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:33:11.229 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:33:11.229 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:33:11.229 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:33:11.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.231 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:33:11.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.302 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:33:11.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.374 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:33:11.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.448 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:33:11.500 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 1) 2020-07-18T03:33:11.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.501 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:33:11.555 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:33:11.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.557 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:33:11.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.628 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0' 2020-07-18T03:33:11.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.701 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:33:11.786 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:33:11.787 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:11.787 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:33:11.787 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.788 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:33:11.788 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.788 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if20: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:33:11.788 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1a:7e:19:c6:c8:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:33:11.788 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:33:11.789 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.789 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::187e:19ff:fec6:c8e3/64 scope link 2020-07-18T03:33:11.789 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.797 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:33:11.875 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:33:11.876 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:11.876 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:33:11.876 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.877 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:33:11.877 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.877 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if20: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:33:11.877 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 1a:7e:19:c6:c8:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:33:11.877 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:33:11.878 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.878 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::187e:19ff:fec6:c8e3/64 scope link 2020-07-18T03:33:11.878 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:11.884 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-07-18T03:33:11.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.885 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0' 2020-07-18T03:33:11.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:11.957 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:33:12.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:12.048 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:33:12.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:12.104 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:33:12.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:12.170 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:33:12.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:12.225 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:33:12.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:12.267 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:33:12.367 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:33:12.367 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:33:12.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:12.368 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:33:12.383 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:33:12.456 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:33:12.456 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:33:12.526 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:33:12.528 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:33:12.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:12.530 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:33:12.599 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:33:12.600 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:33:12.601 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:33:12.642 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:33:12.674 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:33:12.674 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:33:12.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:33:12.736+0000 7f5b3dd011c0 -1 init, newargv = 0x557471052010 newargc=15 2020-07-18T03:33:12.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[46917]: starting ceph client 2020-07-18T03:33:12.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[46917]: starting fuse 2020-07-18T03:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:33:12.748 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:33:12.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:12.749 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:33:12.802 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:33:12.802 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:33:12.803 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:33:12.803 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:33:12.804 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:33:12.804 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:33:12.804 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:33:12.805 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:33:12.806 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:12.806 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:33:12.807 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:33:12.807 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:33:12.808 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:33:12.808 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:12.809 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:33:12.809 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:33:12.809 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:33:12.810 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:33:12.810 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:12.811 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:33:12.811 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:33:12.812 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:33:12.812 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:33:12.812 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:33:12.813 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:33:12.813 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:33:12.814 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:33:12.819 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:12.819 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:33:12.820 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:33:12.933 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.46917.asok 2020-07-18T03:33:13.114 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.46917.asok 2020-07-18T03:33:13.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:13.116 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.46917.asok status 2020-07-18T03:33:13.268 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:33:13.268 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:33:13.268 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:33:13.269 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:33:13.269 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:33:13.269 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:33:13.269 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:33:13.270 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "46917", 2020-07-18T03:33:13.270 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:33:13.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:13.270 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:33:13.271 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:33:13.271 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6535, 2020-07-18T03:33:13.271 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:33:13.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:33:13.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:33:13.272 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6535 2020-07-18T03:33:13.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:13.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:33:13.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:33:13.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:33:13.273 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 840777243 2020-07-18T03:33:13.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:33:13.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:13.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:33:13.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:33:13.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:33:13.274 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 840777243 2020-07-18T03:33:13.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:33:13.274 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6535 192.168.0.2:0/840777243", 2020-07-18T03:33:13.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/840777243", 2020-07-18T03:33:13.275 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:33:13.275 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 29, 2020-07-18T03:33:13.275 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 66, 2020-07-18T03:33:13.275 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:33:13.276 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:33:13.276 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:33:13.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:13.277 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:33:13.278 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:33:13.294 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:33:13.295 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:33:13.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:33:13.296 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:33:13.367 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2020-07-18T03:33:13.535 INFO:teuthology.orchestra.run.smithi047.stdout:{"mds_reconnect_timeout":"45.000000"} 2020-07-18T03:33:13.536 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2020-07-18T03:33:13.537 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2020-07-18T03:33:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:{"ms_max_backoff":"15.000000"} 2020-07-18T03:33:13.668 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2020-07-18T03:33:13.668 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:33:13.669 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:33:13.669 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:33:13.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[46917]: fuse finished with error 0 and tester_r 0 2020-07-18T03:33:13.716 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:33:13.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:13.732 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:33:13.796 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:33:13.797 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:33:13.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:13.799 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:33:13.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:13.875 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:33:13.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:13.973 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:33:14.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:14.021 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:33:14.042 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:33:14.042 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:14.043 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:33:14.043 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:14.043 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:33:14.043 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:14.044 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:14.044 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.044 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:33:14.044 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.044 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:33:14.045 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42397sec preferred_lft 42397sec 2020-07-18T03:33:14.046 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:33:14.046 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:14.046 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:14.046 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.047 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:14.047 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.047 INFO:teuthology.orchestra.run.smithi047.stdout:19: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:33:14.047 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:4c:91:0b:22:52 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.048 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:33:14.048 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:14.048 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7436:62ff:fe20:c122/64 scope link 2020-07-18T03:33:14.049 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:14.049 INFO:teuthology.orchestra.run.smithi047.stdout:20: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:33:14.049 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:4c:91:0b:22:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:33:14.049 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::484c:91ff:fe0b:2252/64 scope link 2020-07-18T03:33:14.049 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:14.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:14.050 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:33:14.114 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.115 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:14.115 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.115 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.115 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.116 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.116 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.116 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.116 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.117 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.117 INFO:teuthology.orchestra.run.smithi047.stdout:19: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.118 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:4c:91:0b:22:52 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:14.118 INFO:teuthology.orchestra.run.smithi047.stdout:20: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:33:14.118 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:4c:91:0b:22:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:33:14.120 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:33:14.121 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:33:14.121 INFO:teuthology.orchestra.run.smithi047:> import time 2020-07-18T03:33:14.122 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:14.122 INFO:teuthology.orchestra.run.smithi047:> path = "/home/ubuntu/cephtest/mnt.0/subdir" 2020-07-18T03:33:14.123 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:14.124 INFO:teuthology.orchestra.run.smithi047:> print("Starting creation...") 2020-07-18T03:33:14.124 INFO:teuthology.orchestra.run.smithi047:> start = time.time() 2020-07-18T03:33:14.125 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:14.125 INFO:teuthology.orchestra.run.smithi047:> os.mkdir(path) 2020-07-18T03:33:14.125 INFO:teuthology.orchestra.run.smithi047:> dfd = os.open(path, os.O_DIRECTORY) 2020-07-18T03:33:14.126 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:14.127 INFO:teuthology.orchestra.run.smithi047:> fd = open(os.path.join(path, "childfile"), "w") 2020-07-18T03:33:14.127 INFO:teuthology.orchestra.run.smithi047:> print("Finished creation in {0}s".format(time.time() - start)) 2020-07-18T03:33:14.128 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:33:14.128 INFO:teuthology.orchestra.run.smithi047:> print("Starting fsync...") 2020-07-18T03:33:14.128 INFO:teuthology.orchestra.run.smithi047:> start = time.time() 2020-07-18T03:33:14.129 INFO:teuthology.orchestra.run.smithi047:> if False: 2020-07-18T03:33:14.129 INFO:teuthology.orchestra.run.smithi047:> os.fsync(dfd) 2020-07-18T03:33:14.131 INFO:teuthology.orchestra.run.smithi047:> else: 2020-07-18T03:33:14.131 INFO:teuthology.orchestra.run.smithi047:> os.fsync(fd) 2020-07-18T03:33:14.132 INFO:teuthology.orchestra.run.smithi047:> print("Finished fsync in {0}s".format(time.time() - start)) 2020-07-18T03:33:14.132 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:33:14.251 INFO:teuthology.orchestra.run.smithi047.stdout:Starting creation... 2020-07-18T03:33:14.252 INFO:teuthology.orchestra.run.smithi047.stdout:Finished creation in 0.006026744842529297s 2020-07-18T03:33:14.252 INFO:teuthology.orchestra.run.smithi047.stdout:Starting fsync... 2020-07-18T03:33:14.252 INFO:teuthology.orchestra.run.smithi047.stdout:Finished fsync in 0.0020051002502441406s 2020-07-18T03:33:14.433 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:33:14.434 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:33:14.435 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:33:14.435 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:33:14.445 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:33:14.446 DEBUG:teuthology.parallel:result is None 2020-07-18T03:33:14.482 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:33:14.482 DEBUG:teuthology.parallel:result is None 2020-07-18T03:33:14.483 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail a 2020-07-18T03:33:14.484 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail b 2020-07-18T03:33:14.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 Processor -- start 2020-07-18T03:33:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 -- start start 2020-07-18T03:33:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7f461486c700 1 Processor -- start 2020-07-18T03:33:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7f461486c700 1 -- start start 2020-07-18T03:33:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 0x7fb69c146960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb69c146e50 0x7fb69c05fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c0b3a60 con 0x7fb69c062040 2020-07-18T03:33:14.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c062c60 con 0x7fb69c146e50 2020-07-18T03:33:14.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.639+0000 7fb6a2688700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0afc30 con 0x7fb69c061c60 2020-07-18T03:33:14.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 0x7fb69c146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb6a0c25700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb69c146e50 0x7fb69c05fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 0x7fb69c146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55568/0 (socket says 172.21.15.47:55568) 2020-07-18T03:33:14.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 -- 172.21.15.47:0/2097204056 learned_addr learned my addr 172.21.15.47:0/2097204056 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:14.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69b7fe700 1 --1- 172.21.15.47:0/2097204056 >> v1:172.21.15.18:6789/0 conn(0x7fb69c062040 0x7fb69c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34594/0 (socket says 172.21.15.47:34594) 2020-07-18T03:33:14.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69affd700 1 -- 172.21.15.47:0/2097204056 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768391042 0 0) 0x7fb69c0b3a60 con 0x7fb69c062040 2020-07-18T03:33:14.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69affd700 1 -- 172.21.15.47:0/2097204056 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb688003240 con 0x7fb69c062040 2020-07-18T03:33:14.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 -- 172.21.15.47:0/2097204056 >> v2:172.21.15.18:3300/0 conn(0x7fb69c146e50 msgr2=0x7fb69c05fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:14.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 --2- 172.21.15.47:0/2097204056 >> v2:172.21.15.18:3300/0 conn(0x7fb69c146e50 0x7fb69c05fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 -- 172.21.15.47:0/2097204056 >> v1:172.21.15.18:6789/0 conn(0x7fb69c062040 legacy=0x7fb69c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7fb69bfff700 1 -- 172.21.15.47:0/2097204056 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0afed0 con 0x7fb69c061c60 2020-07-18T03:33:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb69bfff700 1 --2- 172.21.15.47:0/2097204056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 0x7fb69c146960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fb68c002f30 tx=0x7fb68c006e90).ready entity=mon.2 client_cookie=79b7bce234c64612 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb69affd700 1 -- 172.21.15.47:0/2097204056 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb68c009780 con 0x7fb69c061c60 2020-07-18T03:33:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb69affd700 1 -- 172.21.15.47:0/2097204056 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c0098e0 con 0x7fb69c061c60 2020-07-18T03:33:14.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb69affd700 1 -- 172.21.15.47:0/2097204056 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb68c009c40 con 0x7fb69c061c60 2020-07-18T03:33:14.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 -- 172.21.15.47:0/2097204056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 msgr2=0x7fb69c146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 --2- 172.21.15.47:0/2097204056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 0x7fb69c146960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fb68c002f30 tx=0x7fb68c006e90).stop 2020-07-18T03:33:14.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 -- 172.21.15.47:0/2097204056 shutdown_connections 2020-07-18T03:33:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 --2- 172.21.15.47:0/2097204056 >> v2:172.21.15.18:3300/0 conn(0x7fb69c146e50 0x7fb69c05fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 --2- 172.21.15.47:0/2097204056 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c061c60 0x7fb69c146960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 -- 172.21.15.47:0/2097204056 >> 172.21.15.47:0/2097204056 conn(0x7fb69c0b93c0 msgr2=0x7fb69c0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 -- 172.21.15.47:0/2097204056 shutdown_connections 2020-07-18T03:33:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7fb6a2688700 1 -- 172.21.15.47:0/2097204056 wait complete. 2020-07-18T03:33:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 Processor -- start 2020-07-18T03:33:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 -- start start 2020-07-18T03:33:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c146e50 0x7fb69c066360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb69c158be0 0x7fb69c159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c159670 con 0x7fb69c062040 2020-07-18T03:33:14.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c066850 con 0x7fb69c158be0 2020-07-18T03:33:14.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7fb6a2688700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c159910 con 0x7fb69c146e50 2020-07-18T03:33:14.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb69bfff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fb69c062040 0x7fb69c147330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34610/0 (socket says 172.21.15.47:34610) 2020-07-18T03:33:14.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb69bfff700 1 -- 172.21.15.47:0/4099500278 learned_addr learned my addr 172.21.15.47:0/4099500278 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:14.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb6a0c25700 1 --2- 172.21.15.47:0/4099500278 >> v2:172.21.15.47:3300/0 conn(0x7fb69c158be0 0x7fb69c159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb69b7fe700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c146e50 0x7fb69c066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845519878 0 0) 0x7fb69c159670 con 0x7fb69c062040 2020-07-18T03:33:14.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb674003240 con 0x7fb69c062040 2020-07-18T03:33:14.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb6a0c25700 1 -- 172.21.15.47:0/4099500278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c146e50 msgr2=0x7fb69c066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb6a0c25700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c146e50 0x7fb69c066360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:14.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb6a0c25700 1 -- 172.21.15.47:0/4099500278 >> v1:172.21.15.18:6789/0 conn(0x7fb69c062040 legacy=0x7fb69c147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb6a0c25700 1 -- 172.21.15.47:0/4099500278 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c14a330 con 0x7fb69c158be0 2020-07-18T03:33:14.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb6a0c25700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb69c158be0 0x7fb69c159070 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fb69400c000 tx=0x7fb6940091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:14.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb69400ca10 con 0x7fb69c158be0 2020-07-18T03:33:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.645+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69400cb70 con 0x7fb69c158be0 2020-07-18T03:33:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.645+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c147a40 con 0x7fb69c158be0 2020-07-18T03:33:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.645+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6940123f0 con 0x7fb69c158be0 2020-07-18T03:33:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.645+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c14a7d0 con 0x7fb69c158be0 2020-07-18T03:33:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c057450 con 0x7fb69c158be0 2020-07-18T03:33:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb694022070 con 0x7fb69c158be0 2020-07-18T03:33:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7fb67bfff700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb674050380 0x7fb674052840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.647+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb6940681f0 con 0x7fb69c158be0 2020-07-18T03:33:14.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.647+0000 7fb69bfff700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb674050380 0x7fb674052840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.647+0000 7fb69bfff700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb674050380 0x7fb674052840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb68c003070 tx=0x7fb68c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:14.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f461486c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46100b8d20 0x7f4610148490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f461486c700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100b52c0 con 0x7f46100b8940 2020-07-18T03:33:14.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f461486c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100b5560 con 0x7f4610148980 2020-07-18T03:33:14.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f461486c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46100b5800 con 0x7f46100b8d20 2020-07-18T03:33:14.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46100b8d20 0x7f4610148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4610148980 0x7f4610148dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44068/0 (socket says 172.21.15.47:44068) 2020-07-18T03:33:14.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460ed9d700 1 -- 172.21.15.47:0/3409308425 learned_addr learned my addr 172.21.15.47:0/3409308425 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:14.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460e59c700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f46100b8940 0x7f4610144440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34598/0 (socket says 172.21.15.47:34598) 2020-07-18T03:33:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460dd9b700 1 -- 172.21.15.47:0/3409308425 >> v1:172.21.15.47:6789/0 conn(0x7f4610148980 legacy=0x7f4610148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460dd9b700 1 -- 172.21.15.47:0/3409308425 >> v1:172.21.15.18:6789/0 conn(0x7f46100b8940 legacy=0x7f4610144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.640+0000 7f460d59a700 1 -- 172.21.15.47:0/3409308425 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057778188 0 0) 0x7f46100b5560 con 0x7f4610148980 2020-07-18T03:33:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f460dd9b700 1 -- 172.21.15.47:0/3409308425 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100b4f40 con 0x7f46100b8d20 2020-07-18T03:33:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f460dd9b700 1 --2- 172.21.15.47:0/3409308425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46100b8d20 0x7f4610148490 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f4604002f30 tx=0x7f4604006dd0).ready entity=mon.1 client_cookie=ebfa0174a506cd5d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f460d59a700 1 -- 172.21.15.47:0/3409308425 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46040097f0 con 0x7f46100b8d20 2020-07-18T03:33:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f460d59a700 1 -- 172.21.15.47:0/3409308425 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4604009950 con 0x7f46100b8d20 2020-07-18T03:33:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f460d59a700 1 -- 172.21.15.47:0/3409308425 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4604009cb0 con 0x7f46100b8d20 2020-07-18T03:33:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 -- 172.21.15.47:0/3409308425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46100b8d20 msgr2=0x7f4610148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 --2- 172.21.15.47:0/3409308425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46100b8d20 0x7f4610148490 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f4604002f30 tx=0x7f4604006dd0).stop 2020-07-18T03:33:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 -- 172.21.15.47:0/3409308425 shutdown_connections 2020-07-18T03:33:14.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 --2- 172.21.15.47:0/3409308425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46100b8d20 0x7f4610148490 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:14.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 -- 172.21.15.47:0/3409308425 >> 172.21.15.47:0/3409308425 conn(0x7f46100bc0a0 msgr2=0x7f46100b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 -- 172.21.15.47:0/3409308425 shutdown_connections 2020-07-18T03:33:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.641+0000 7f461486c700 1 -- 172.21.15.47:0/3409308425 wait complete. 2020-07-18T03:33:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 Processor -- start 2020-07-18T03:33:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 -- start start 2020-07-18T03:33:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46100b8940 0x7f46100af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4610148980 0x7f46100ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100af870 con 0x7f46100ad720 2020-07-18T03:33:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46100af650 con 0x7f4610148980 2020-07-18T03:33:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f461486c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46100afb10 con 0x7f46100b8940 2020-07-18T03:33:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f460dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4610148980 0x7f46100ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f460ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f46100ad720 0x7f461013b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44074/0 (socket says 172.21.15.47:44074) 2020-07-18T03:33:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.643+0000 7f460ed9d700 1 -- 172.21.15.47:0/908719802 learned_addr learned my addr 172.21.15.47:0/908719802 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f460e59c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46100b8940 0x7f46100af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046188128 0 0) 0x7f46100af870 con 0x7f46100ad720 2020-07-18T03:33:14.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e4003240 con 0x7f46100ad720 2020-07-18T03:33:14.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f460dd9b700 1 -- 172.21.15.47:0/908719802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46100b8940 msgr2=0x7f46100af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f460dd9b700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46100b8940 0x7f46100af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:14.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f460dd9b700 1 -- 172.21.15.47:0/908719802 >> v1:172.21.15.47:6789/0 conn(0x7f46100ad720 legacy=0x7f461013b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:14.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f460dd9b700 1 -- 172.21.15.47:0/908719802 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100afdb0 con 0x7f4610148980 2020-07-18T03:33:14.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f460dd9b700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4610148980 0x7f46100ad230 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f4604002f30 tx=0x7f4604015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:14.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4604011e20 con 0x7f4610148980 2020-07-18T03:33:14.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460401d5d0 con 0x7f4610148980 2020-07-18T03:33:14.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100b0250 con 0x7f4610148980 2020-07-18T03:33:14.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f460401d930 con 0x7f4610148980 2020-07-18T03:33:14.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.644+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461013c170 con 0x7f4610148980 2020-07-18T03:33:14.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46100b85c0 con 0x7f4610148980 2020-07-18T03:33:14.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f4604025030 con 0x7f4610148980 2020-07-18T03:33:14.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7f45fe7fc700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45e4050360 0x7f45e4052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:14.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4604069350 con 0x7f4610148980 2020-07-18T03:33:14.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7f460e59c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45e4050360 0x7f45e4052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:14.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.646+0000 7f460e59c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45e4050360 0x7f45e4052820 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f460000d000 tx=0x7f4600007b20).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:14.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.650+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f4604041ae0 con 0x7f4610148980 2020-07-18T03:33:14.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.653+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb694040920 con 0x7fb69c158be0 2020-07-18T03:33:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.872+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f46100aff40 con 0x7f4610148980 2020-07-18T03:33:14.875 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:14.873+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fb69c0681b0 con 0x7fb69c158be0 2020-07-18T03:33:15.437 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:33:15.435+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:33:15.439 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.437+0000 7f45fe7fc700 1 -- 172.21.15.47:0/908719802 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6295 v30) v1 ==== 95+0+0 (secure 0 0 0) 0x7f4604045100 con 0x7f4610148980 2020-07-18T03:33:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.437+0000 7fb67bfff700 1 -- 172.21.15.47:0/4099500278 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6003 v30) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb694016020 con 0x7fb69c158be0 2020-07-18T03:33:15.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.439+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45e4050360 msgr2=0x7f45e4052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:15.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.439+0000 7f461486c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45e4050360 0x7f45e4052820 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f460000d000 tx=0x7f4600007b20).stop 2020-07-18T03:33:15.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4610148980 msgr2=0x7f46100ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:15.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7f461486c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4610148980 0x7f46100ad230 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f4604002f30 tx=0x7f4604015000).stop 2020-07-18T03:33:15.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 shutdown_connections 2020-07-18T03:33:15.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f45e4050360 0x7f45e4052820 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f46100b8940 0x7f46100af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 --2- 172.21.15.47:0/908719802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4610148980 0x7f46100ad230 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 >> 172.21.15.47:0/908719802 conn(0x7f46100bc0a0 msgr2=0x7f461005f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 shutdown_connections 2020-07-18T03:33:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.441+0000 7f461486c700 1 -- 172.21.15.47:0/908719802 wait complete. 2020-07-18T03:33:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.439+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb674050380 msgr2=0x7fb674052840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.439+0000 7fb6a2688700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb674050380 0x7fb674052840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb68c003070 tx=0x7fb68c015000).stop 2020-07-18T03:33:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.439+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb69c158be0 msgr2=0x7fb69c159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.439+0000 7fb6a2688700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb69c158be0 0x7fb69c159070 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fb69400c000 tx=0x7fb6940091d0).stop 2020-07-18T03:33:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 shutdown_connections 2020-07-18T03:33:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb674050380 0x7fb674052840 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb69c158be0 0x7fb69c159070 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 --2- 172.21.15.47:0/4099500278 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb69c146e50 0x7fb69c066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 >> 172.21.15.47:0/4099500278 conn(0x7fb69c0b93c0 msgr2=0x7fb69c05f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 shutdown_connections 2020-07-18T03:33:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:15.440+0000 7fb6a2688700 1 -- 172.21.15.47:0/4099500278 wait complete. 2020-07-18T03:33:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 6003 2020-07-18T03:33:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 6295 2020-07-18T03:33:15.456 DEBUG:teuthology.parallel:result is '' 2020-07-18T03:33:15.458 DEBUG:teuthology.parallel:result is '' 2020-07-18T03:33:15.459 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:33:15.460 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:33:15.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.461 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:33:15.549 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:33:15.552 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:33:15.552 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:33:15.553 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:33:15.635 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:33:15.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.637 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:33:15.705 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:33:15.706 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:33:15.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.707 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.1 down' 2020-07-18T03:33:15.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.781 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.1' 2020-07-18T03:33:15.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.882 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:33:15.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.939 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:33:15.971 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:33:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:33:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:33:15.973 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:15.973 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:15.973 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:15.973 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:33:15.979 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:15.979 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:33:15.979 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42395sec preferred_lft 42395sec 2020-07-18T03:33:15.980 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:33:15.980 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:15.981 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:15.982 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:15.986 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:33:15.986 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:15.986 INFO:teuthology.orchestra.run.smithi047.stdout:19: ceph-brx: mtu 1500 qdisc noqueue state DOWN group default qlen 1000 2020-07-18T03:33:15.987 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:15.987 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:33:15.987 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:15.987 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7436:62ff:fe20:c122/64 scope link 2020-07-18T03:33:15.988 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:33:15.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:15.997 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:33:16.042 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:16.042 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:33:16.043 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:16.043 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:16.043 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:33:16.044 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:16.044 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:16.044 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:16.044 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:16.045 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:16.045 INFO:teuthology.orchestra.run.smithi047.stdout:19: ceph-brx: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:33:16.045 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:33:16.046 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:33:16.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:16.048 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:33:16.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:16.121 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:33:16.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:16.223 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:33:16.259 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:33:16.259 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:33:16.260 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:33:16.260 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:33:16.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:16.261 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:33:16.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:16.341 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:33:16.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:33:16.420 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:33:16.499 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:33:16.500 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:33:16.501 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:33:16.502 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:33:16.505 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:33:16.505 DEBUG:teuthology.parallel:result is None 2020-07-18T03:33:16.542 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:33:16.543 DEBUG:teuthology.parallel:result is None 2020-07-18T03:33:16.544 INFO:tasks.cephfs.test_client_recovery:Waiting for reconnect... 2020-07-18T03:33:16.544 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:16.596 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:33:16.622 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:33:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.660+0000 7f67197ef700 1 Processor -- start 2020-07-18T03:33:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.661+0000 7f67197ef700 1 -- start start 2020-07-18T03:33:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.661+0000 7f67197ef700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.661+0000 7f67197ef700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f67141433e0 0x7f6714143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.661+0000 7f67197ef700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6714147330 con 0x7f67140ad100 2020-07-18T03:33:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.661+0000 7f67197ef700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140ae530 con 0x7f6714147a00 2020-07-18T03:33:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.661+0000 7f67197ef700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140b57e0 con 0x7f67141433e0 2020-07-18T03:33:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67127fc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67137fe700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f67141433e0 0x7f6714143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f6712ffd700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f67140ad100 0x7f67140b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34640/0 (socket says 172.21.15.47:34640) 2020-07-18T03:33:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f6712ffd700 1 -- 172.21.15.47:0/664630894 learned_addr learned my addr 172.21.15.47:0/664630894 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67137fe700 1 -- 172.21.15.47:0/664630894 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 msgr2=0x7f6714147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67137fe700 1 --2- 172.21.15.47:0/664630894 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67137fe700 1 -- 172.21.15.47:0/664630894 >> v1:172.21.15.18:6789/0 conn(0x7f67140ad100 legacy=0x7f67140b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f6711ffb700 1 -- 172.21.15.47:0/664630894 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487891672 0 0) 0x7f6714147330 con 0x7f67140ad100 2020-07-18T03:33:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67137fe700 1 -- 172.21.15.47:0/664630894 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140b5a80 con 0x7f67141433e0 2020-07-18T03:33:16.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67127fc700 1 --2- 172.21.15.47:0/664630894 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:16.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67137fe700 1 --2- 172.21.15.47:0/664630894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f67141433e0 0x7f6714143820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6704002f30 tx=0x7f6704006dd0).ready entity=mon.2 client_cookie=283a9c5a9ef42e39 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:16.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f6711ffb700 1 -- 172.21.15.47:0/664630894 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6704009760 con 0x7f67141433e0 2020-07-18T03:33:16.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f6711ffb700 1 -- 172.21.15.47:0/664630894 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040098c0 con 0x7f67141433e0 2020-07-18T03:33:16.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f6711ffb700 1 -- 172.21.15.47:0/664630894 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6704009c20 con 0x7f67141433e0 2020-07-18T03:33:16.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 -- 172.21.15.47:0/664630894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f67141433e0 msgr2=0x7f6714143820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 --2- 172.21.15.47:0/664630894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f67141433e0 0x7f6714143820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6704002f30 tx=0x7f6704006dd0).stop 2020-07-18T03:33:16.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 -- 172.21.15.47:0/664630894 shutdown_connections 2020-07-18T03:33:16.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 --2- 172.21.15.47:0/664630894 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f67141433e0 0x7f6714143820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 --2- 172.21.15.47:0/664630894 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 -- 172.21.15.47:0/664630894 >> 172.21.15.47:0/664630894 conn(0x7f67140b9340 msgr2=0x7f67140b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:16.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.662+0000 7f67197ef700 1 -- 172.21.15.47:0/664630894 shutdown_connections 2020-07-18T03:33:16.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.663+0000 7f67197ef700 1 -- 172.21.15.47:0/664630894 wait complete. 2020-07-18T03:33:16.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.664+0000 7f67197ef700 1 Processor -- start 2020-07-18T03:33:16.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.664+0000 7f67197ef700 1 -- start start 2020-07-18T03:33:16.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f67197ef700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f67197ef700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6714155f90 con 0x7f67140b2310 2020-07-18T03:33:16.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f67197ef700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6714156230 con 0x7f67140ad100 2020-07-18T03:33:16.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f67197ef700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6714064600 con 0x7f6714147a00 2020-07-18T03:33:16.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f6712ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67140ad100 0x7f6714061ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44112/0 (socket says 172.21.15.47:44112) 2020-07-18T03:33:16.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f6712ffd700 1 -- 172.21.15.47:0/1851291221 learned_addr learned my addr 172.21.15.47:0/1851291221 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:16.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f67127fc700 1 --2- 172.21.15.47:0/1851291221 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139338046 0 0) 0x7f6714156230 con 0x7f67140ad100 2020-07-18T03:33:16.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f0003240 con 0x7f67140ad100 2020-07-18T03:33:16.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3353150244 0 0) 0x7f66f0003240 con 0x7f67140ad100 2020-07-18T03:33:16.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6714156230 con 0x7f67140ad100 2020-07-18T03:33:16.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f670c001be0 con 0x7f67140ad100 2020-07-18T03:33:16.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 msgr2=0x7f6714148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 --2- 172.21.15.47:0/1851291221 >> v2:172.21.15.18:3300/0 conn(0x7f6714147a00 0x7f6714148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 >> v1:172.21.15.47:6789/0 conn(0x7f67140ad100 legacy=0x7f6714061ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 >> v1:172.21.15.18:6789/0 conn(0x7f67140b2310 legacy=0x7f67140624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f0003800 0x7f66f0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.665+0000 7f66faffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f00044e0 0x7f66f0004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f0004e10 0x7f66f000b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6714064600 con 0x7f66f0003800 2020-07-18T03:33:16.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00053b0 con 0x7f66f0004e10 2020-07-18T03:33:16.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f000bca0 con 0x7f66f00044e0 2020-07-18T03:33:16.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f0004e10 0x7f66f000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f6712ffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f0003800 0x7f66f0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67127fc700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f00044e0 0x7f66f0004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 -- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f00044e0 msgr2=0x7f66f0004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f66f00044e0 0x7f66f0004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 -- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f0003800 msgr2=0x7f66f0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f0003800 0x7f66f0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6714148f00 con 0x7f66f0004e10 2020-07-18T03:33:16.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6714062bc0 con 0x7f66f0004e10 2020-07-18T03:33:16.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 -- 172.21.15.47:0/1851291221 reap_dead start 2020-07-18T03:33:16.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67140b3380 con 0x7f66f0004e10 2020-07-18T03:33:16.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f67137fe700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f0004e10 0x7f66f000b730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f6714064390 tx=0x7f6704015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:16.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6704021070 con 0x7f66f0004e10 2020-07-18T03:33:16.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670400b160 con 0x7f66f0004e10 2020-07-18T03:33:16.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.666+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6704006920 con 0x7f66f0004e10 2020-07-18T03:33:16.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.667+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714057450 con 0x7f66f0004e10 2020-07-18T03:33:16.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.667+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f670400f070 con 0x7f66f0004e10 2020-07-18T03:33:16.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.667+0000 7f66faffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66f0055f50 0x7f66f0058410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.668+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6704069050 con 0x7f66f0004e10 2020-07-18T03:33:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.668+0000 7f6712ffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66f0055f50 0x7f66f0058410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.668+0000 7f6712ffd700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66f0055f50 0x7f66f0058410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f670c00d000 tx=0x7f670c009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.671+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f67040417c0 con 0x7f66f0004e10 2020-07-18T03:33:16.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.879+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671406a100 con 0x7f66f0004e10 2020-07-18T03:33:16.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.879+0000 7f66faffd700 1 -- 172.21.15.47:0/1851291221 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+1423 (secure 0 0 0) 0x7f6704044de0 con 0x7f66f0004e10 2020-07-18T03:33:16.882 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:16.883 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:15.436183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:16.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66f0055f50 msgr2=0x7f66f0058410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66f0055f50 0x7f66f0058410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f670c00d000 tx=0x7f670c009040).stop 2020-07-18T03:33:16.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f0004e10 msgr2=0x7f66f000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:16.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f0004e10 0x7f66f000b730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f6714064390 tx=0x7f6704015000).stop 2020-07-18T03:33:16.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 shutdown_connections 2020-07-18T03:33:16.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f66f0055f50 0x7f66f0058410 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 --2- 172.21.15.47:0/1851291221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f0004e10 0x7f66f000b730 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:16.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 >> 172.21.15.47:0/1851291221 conn(0x7f67140b9340 msgr2=0x7f67141469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:16.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 shutdown_connections 2020-07-18T03:33:16.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:16.881+0000 7f67197ef700 1 -- 172.21.15.47:0/1851291221 wait complete. 2020-07-18T03:33:16.896 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2020-07-18T03:33:16.897 INFO:tasks.cephfs.filesystem:mapped states [] to None 2020-07-18T03:33:17.907 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:18.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.032+0000 7f10e8ad8700 1 Processor -- start 2020-07-18T03:33:18.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.032+0000 7f10e8ad8700 1 -- start start 2020-07-18T03:33:18.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e8ad8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e40b5370 0x7f10e405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:18.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e8ad8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10e40b41c0 0x7f10e40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:18.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e8ad8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10e40b4620 0x7f10e40ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:18.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e8ad8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40b5a10 con 0x7f10e40b4620 2020-07-18T03:33:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e8ad8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40b0040 con 0x7f10e40b41c0 2020-07-18T03:33:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e8ad8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40b3af0 con 0x7f10e40b5370 2020-07-18T03:33:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10e40b41c0 0x7f10e40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10e40b41c0 0x7f10e40ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34816/0 (socket says 172.21.15.47:34816) 2020-07-18T03:33:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.033+0000 7f10e1d9b700 1 -- 172.21.15.47:0/474037479 learned_addr learned my addr 172.21.15.47:0/474037479 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:18.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e259c700 1 --2- 172.21.15.47:0/474037479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e40b5370 0x7f10e405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:18.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e2d9d700 1 --2- 172.21.15.47:0/474037479 >> v2:172.21.15.18:3300/0 conn(0x7f10e40b4620 0x7f10e40ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:18.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e1d9b700 1 -- 172.21.15.47:0/474037479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e40b5370 msgr2=0x7f10e405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:18.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e1d9b700 1 --2- 172.21.15.47:0/474037479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e40b5370 0x7f10e405fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e1d9b700 1 -- 172.21.15.47:0/474037479 >> v2:172.21.15.18:3300/0 conn(0x7f10e40b4620 msgr2=0x7f10e40ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:18.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e1d9b700 1 --2- 172.21.15.47:0/474037479 >> v2:172.21.15.18:3300/0 conn(0x7f10e40b4620 0x7f10e40ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e1d9b700 1 -- 172.21.15.47:0/474037479 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40adcb0 con 0x7f10e40b41c0 2020-07-18T03:33:18.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e1d9b700 1 --2- 172.21.15.47:0/474037479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e40b41c0 0x7f10e40ad2d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f10d40032b0 tx=0x7f10d4009270).ready entity=mon.1 client_cookie=642e61bab16c5e6a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:18.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e159a700 1 -- 172.21.15.47:0/474037479 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d4009e60 con 0x7f10e40b41c0 2020-07-18T03:33:18.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e159a700 1 -- 172.21.15.47:0/474037479 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d400bb50 con 0x7f10e40b41c0 2020-07-18T03:33:18.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e159a700 1 -- 172.21.15.47:0/474037479 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d400bcc0 con 0x7f10e40b41c0 2020-07-18T03:33:18.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 -- 172.21.15.47:0/474037479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e40b41c0 msgr2=0x7f10e40ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/474037479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e40b41c0 0x7f10e40ad2d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f10d40032b0 tx=0x7f10d4009270).stop 2020-07-18T03:33:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 -- 172.21.15.47:0/474037479 shutdown_connections 2020-07-18T03:33:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/474037479 >> v2:172.21.15.18:3300/0 conn(0x7f10e40b4620 0x7f10e40ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/474037479 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f10e40b5370 0x7f10e405fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/474037479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e40b41c0 0x7f10e40ad2d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 -- 172.21.15.47:0/474037479 >> 172.21.15.47:0/474037479 conn(0x7f10e40b9340 msgr2=0x7f10e40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 -- 172.21.15.47:0/474037479 shutdown_connections 2020-07-18T03:33:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.034+0000 7f10e8ad8700 1 -- 172.21.15.47:0/474037479 wait complete. 2020-07-18T03:33:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.036+0000 7f10e8ad8700 1 Processor -- start 2020-07-18T03:33:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.036+0000 7f10e8ad8700 1 -- start start 2020-07-18T03:33:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e8ad8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10e4138980 0x7f10e40628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e8ad8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e413fa80 con 0x7f10e4141820 2020-07-18T03:33:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e8ad8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e4127a50 con 0x7f10e40b5370 2020-07-18T03:33:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e8ad8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e4127cd0 con 0x7f10e4138980 2020-07-18T03:33:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10e40b5370 0x7f10e413bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44142/0 (socket says 172.21.15.47:44142) 2020-07-18T03:33:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e259c700 1 -- 172.21.15.47:0/1162015543 learned_addr learned my addr 172.21.15.47:0/1162015543 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e2d9d700 1 --2- 172.21.15.47:0/1162015543 >> v2:172.21.15.47:3300/0 conn(0x7f10e4138980 0x7f10e40628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187460363 0 0) 0x7f10e4127a50 con 0x7f10e40b5370 2020-07-18T03:33:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c0003240 con 0x7f10e40b5370 2020-07-18T03:33:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e2d9d700 1 -- 172.21.15.47:0/1162015543 >> v1:172.21.15.47:6789/0 conn(0x7f10e40b5370 legacy=0x7f10e413bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e2d9d700 1 -- 172.21.15.47:0/1162015543 >> v1:172.21.15.18:6789/0 conn(0x7f10e4141820 legacy=0x7f10e413f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807691335 0 0) 0x7f10e413fa80 con 0x7f10e4141820 2020-07-18T03:33:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e2d9d700 1 -- 172.21.15.47:0/1162015543 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4062430 con 0x7f10e4138980 2020-07-18T03:33:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e2d9d700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4138980 0x7f10e40628d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f10dc00c000 tx=0x7f10dc006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10dc00c860 con 0x7f10e4138980 2020-07-18T03:33:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc00c9c0 con 0x7f10e4138980 2020-07-18T03:33:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e4062160 con 0x7f10e4138980 2020-07-18T03:33:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.037+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10dc00cd20 con 0x7f10e4138980 2020-07-18T03:33:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.038+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e4063000 con 0x7f10e4138980 2020-07-18T03:33:18.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.038+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e40632b0 con 0x7f10e4138980 2020-07-18T03:33:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.039+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f10dc01c460 con 0x7f10e4138980 2020-07-18T03:33:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.039+0000 7f10da7fc700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10c00502e0 0x7f10c00527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.039+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f10dc068070 con 0x7f10e4138980 2020-07-18T03:33:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.039+0000 7f10e259c700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10c00502e0 0x7f10c00527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:18.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.039+0000 7f10e259c700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10c00502e0 0x7f10c00527a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f10cc00a000 tx=0x7f10cc006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:18.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.042+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f10dc040750 con 0x7f10e4138980 2020-07-18T03:33:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.249+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10e40625c0 con 0x7f10e4138980 2020-07-18T03:33:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.250+0000 7f10da7fc700 1 -- 172.21.15.47:0/1162015543 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1785 (secure 0 0 0) 0x7f10dc043d70 con 0x7f10e4138980 2020-07-18T03:33:18.253 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:18.253 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:17.447174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10c00502e0 msgr2=0x7f10c00527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10c00502e0 0x7f10c00527a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f10cc00a000 tx=0x7f10cc006dd0).stop 2020-07-18T03:33:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4138980 msgr2=0x7f10e40628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4138980 0x7f10e40628d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f10dc00c000 tx=0x7f10dc006dd0).stop 2020-07-18T03:33:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 shutdown_connections 2020-07-18T03:33:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10e4138980 0x7f10e40628d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 --2- 172.21.15.47:0/1162015543 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f10c00502e0 0x7f10c00527a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 >> 172.21.15.47:0/1162015543 conn(0x7f10e40b9340 msgr2=0x7f10e40b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 shutdown_connections 2020-07-18T03:33:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:18.251+0000 7f10e8ad8700 1 -- 172.21.15.47:0/1162015543 wait complete. 2020-07-18T03:33:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2020-07-18T03:33:18.265 INFO:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2020-07-18T03:33:19.267 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.393+0000 7f976cffc700 1 Processor -- start 2020-07-18T03:33:19.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.393+0000 7f976cffc700 1 -- start start 2020-07-18T03:33:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f976cffc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f97680b31f0 0x7f97680b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f976cffc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2620 0x7f97681454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f976cffc700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97680afbd0 con 0x7f97680b2e10 2020-07-18T03:33:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f976cffc700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9768144e30 con 0x7f97680b31f0 2020-07-18T03:33:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f976cffc700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97681491c0 con 0x7f97680b2620 2020-07-18T03:33:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f9765d9b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f97680b31f0 0x7f97680b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f9765d9b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f97680b31f0 0x7f97680b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:48742/0 (socket says 172.21.15.47:48742) 2020-07-18T03:33:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.394+0000 7f9765d9b700 1 -- 172.21.15.47:0/3765158281 learned_addr learned my addr 172.21.15.47:0/3765158281 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f9766d9d700 1 --2- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2620 0x7f97681454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f9765d9b700 1 -- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2620 msgr2=0x7f97681454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f9765d9b700 1 --2- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2620 0x7f97681454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f9765d9b700 1 -- 172.21.15.47:0/3765158281 >> v1:172.21.15.18:6789/0 conn(0x7f97680b2e10 legacy=0x7f97680b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f9765d9b700 1 -- 172.21.15.47:0/3765158281 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768145050 con 0x7f97680b31f0 2020-07-18T03:33:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f9765d9b700 1 --2- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b31f0 0x7f97680b2130 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f97680ba340 tx=0x7f9758006e90).ready entity=mon.0 client_cookie=889deb924d3425e1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976559a700 1 -- 172.21.15.47:0/3765158281 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9758009930 con 0x7f97680b31f0 2020-07-18T03:33:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976559a700 1 -- 172.21.15.47:0/3765158281 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9758009a90 con 0x7f97680b31f0 2020-07-18T03:33:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976559a700 1 -- 172.21.15.47:0/3765158281 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9758009df0 con 0x7f97680b31f0 2020-07-18T03:33:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 -- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b31f0 msgr2=0x7f97680b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 --2- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b31f0 0x7f97680b2130 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f97680ba340 tx=0x7f9758006e90).stop 2020-07-18T03:33:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 -- 172.21.15.47:0/3765158281 shutdown_connections 2020-07-18T03:33:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 --2- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2620 0x7f97681454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 --2- 172.21.15.47:0/3765158281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b31f0 0x7f97680b2130 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 -- 172.21.15.47:0/3765158281 >> 172.21.15.47:0/3765158281 conn(0x7f97680b9340 msgr2=0x7f97680b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 -- 172.21.15.47:0/3765158281 shutdown_connections 2020-07-18T03:33:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.395+0000 7f976cffc700 1 -- 172.21.15.47:0/3765158281 wait complete. 2020-07-18T03:33:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.397+0000 7f976cffc700 1 Processor -- start 2020-07-18T03:33:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.397+0000 7f976cffc700 1 -- start start 2020-07-18T03:33:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976cffc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f97680b2620 0x7f9768147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976cffc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2e10 0x7f97680adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976cffc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9768158f60 con 0x7f9768112710 2020-07-18T03:33:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976cffc700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97680ae2d0 con 0x7f97680b2620 2020-07-18T03:33:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976cffc700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97680ae4f0 con 0x7f97680b2e10 2020-07-18T03:33:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f9766d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9768112710 0x7f9768065d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44158/0 (socket says 172.21.15.47:44158) 2020-07-18T03:33:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f9766d9d700 1 -- 172.21.15.47:0/3690759691 learned_addr learned my addr 172.21.15.47:0/3690759691 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f9765d9b700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2e10 0x7f97680adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976659c700 1 --2- 172.21.15.47:0/3690759691 >> v2:172.21.15.18:3300/0 conn(0x7f97680b2620 0x7f9768147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211138189 0 0) 0x7f9768158f60 con 0x7f9768112710 2020-07-18T03:33:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f973c003240 con 0x7f9768112710 2020-07-18T03:33:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976659c700 1 -- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2e10 msgr2=0x7f97680adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976659c700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2e10 0x7f97680adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976659c700 1 -- 172.21.15.47:0/3690759691 >> v1:172.21.15.47:6789/0 conn(0x7f9768112710 legacy=0x7f9768065d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 837838171 0 0) 0x7f973c003240 con 0x7f9768112710 2020-07-18T03:33:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f976659c700 1 -- 172.21.15.47:0/3690759691 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97681591c0 con 0x7f97680b2620 2020-07-18T03:33:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.398+0000 7f9765d9b700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2e10 0x7f97680adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.399+0000 7f976659c700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b2620 0x7f9768147be0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f97600019b0 tx=0x7f975c007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.399+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9760001810 con 0x7f97680b2620 2020-07-18T03:33:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.399+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c00ce00 con 0x7f97680b2620 2020-07-18T03:33:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.399+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9768066430 con 0x7f97680b2620 2020-07-18T03:33:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.399+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f975c007240 con 0x7f97680b2620 2020-07-18T03:33:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.399+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9768155850 con 0x7f97680b2620 2020-07-18T03:33:19.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.400+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9768057450 con 0x7f97680b2620 2020-07-18T03:33:19.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.400+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f975c010070 con 0x7f97680b2620 2020-07-18T03:33:19.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.400+0000 7f97567fc700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f973c050360 0x7f973c052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.400+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f975c067050 con 0x7f97680b2620 2020-07-18T03:33:19.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.400+0000 7f9765d9b700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f973c050360 0x7f973c052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.401+0000 7f9765d9b700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f973c050360 0x7f973c052820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f975801c000 tx=0x7f9758015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:19.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.405+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f975c03f790 con 0x7f97680b2620 2020-07-18T03:33:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.615+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97680afbd0 con 0x7f97680b2620 2020-07-18T03:33:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.616+0000 7f97567fc700 1 -- 172.21.15.47:0/3690759691 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1788 (secure 0 0 0) 0x7f97680afbd0 con 0x7f97680b2620 2020-07-18T03:33:19.618 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:19.618 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:19.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f973c050360 msgr2=0x7f973c052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f973c050360 0x7f973c052820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f975801c000 tx=0x7f9758015000).stop 2020-07-18T03:33:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b2620 msgr2=0x7f9768147be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b2620 0x7f9768147be0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f97600019b0 tx=0x7f975c007b70).stop 2020-07-18T03:33:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 shutdown_connections 2020-07-18T03:33:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f973c050360 0x7f973c052820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97680b2620 0x7f9768147be0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 --2- 172.21.15.47:0/3690759691 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f97680b2e10 0x7f97680adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 >> 172.21.15.47:0/3690759691 conn(0x7f97680b9340 msgr2=0x7f97680b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 shutdown_connections 2020-07-18T03:33:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.617+0000 7f976cffc700 1 -- 172.21.15.47:0/3690759691 wait complete. 2020-07-18T03:33:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 33 2020-07-18T03:33:19.630 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to up:reconnect 2020-07-18T03:33:19.630 INFO:tasks.cephfs.filesystem:reached state 'up:reconnect' in 3.0858724117279053s 2020-07-18T03:33:19.631 INFO:tasks.cephfs.test_client_recovery:Waiting for active... 2020-07-18T03:33:19.631 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.744+0000 7f5200aae700 1 Processor -- start 2020-07-18T03:33:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.744+0000 7f5200aae700 1 -- start start 2020-07-18T03:33:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51fc0ad100 0x7f51fc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f51fc147a00 0x7f51fc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f51fc05f5a0 0x7f51fc05f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc0ae530 con 0x7f51fc147a00 2020-07-18T03:33:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc147330 con 0x7f51fc0ad100 2020-07-18T03:33:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0b57e0 con 0x7f51fc05f5a0 2020-07-18T03:33:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51fc0ad100 0x7f51fc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51fc0ad100 0x7f51fc0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34842/0 (socket says 172.21.15.47:34842) 2020-07-18T03:33:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 -- 172.21.15.47:0/1512393219 learned_addr learned my addr 172.21.15.47:0/1512393219 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51f9d9b700 1 --2- 172.21.15.47:0/1512393219 >> v2:172.21.15.18:3300/0 conn(0x7f51fc147a00 0x7f51fc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 -- 172.21.15.47:0/1512393219 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f51fc05f5a0 msgr2=0x7f51fc05f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:19.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fad9d700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f51fc05f5a0 0x7f51fc05f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f51fc05f5a0 0x7f51fc05f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 -- 172.21.15.47:0/1512393219 >> v2:172.21.15.18:3300/0 conn(0x7f51fc147a00 msgr2=0x7f51fc147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:19.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 --2- 172.21.15.47:0/1512393219 >> v2:172.21.15.18:3300/0 conn(0x7f51fc147a00 0x7f51fc147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 -- 172.21.15.47:0/1512393219 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0b5a80 con 0x7f51fc0ad100 2020-07-18T03:33:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fad9d700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f51fc05f5a0 0x7f51fc05f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:33:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51f9d9b700 1 --2- 172.21.15.47:0/1512393219 >> v2:172.21.15.18:3300/0 conn(0x7f51fc147a00 0x7f51fc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:33:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51fa59c700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc0ad100 0x7f51fc0b4c10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f51f40032b0 tx=0x7f51f4009270).ready entity=mon.1 client_cookie=59a5bff1bb23b482 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51f959a700 1 -- 172.21.15.47:0/1512393219 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51f400bc00 con 0x7f51fc0ad100 2020-07-18T03:33:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51f959a700 1 -- 172.21.15.47:0/1512393219 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f400bd60 con 0x7f51fc0ad100 2020-07-18T03:33:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f51f959a700 1 -- 172.21.15.47:0/1512393219 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51f4011400 con 0x7f51fc0ad100 2020-07-18T03:33:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 -- 172.21.15.47:0/1512393219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc0ad100 msgr2=0x7f51fc0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.745+0000 7f5200aae700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc0ad100 0x7f51fc0b4c10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f51f40032b0 tx=0x7f51f4009270).stop 2020-07-18T03:33:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 -- 172.21.15.47:0/1512393219 shutdown_connections 2020-07-18T03:33:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f51fc05f5a0 0x7f51fc05f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 --2- 172.21.15.47:0/1512393219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc0ad100 0x7f51fc0b4c10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 --2- 172.21.15.47:0/1512393219 >> v2:172.21.15.18:3300/0 conn(0x7f51fc147a00 0x7f51fc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 -- 172.21.15.47:0/1512393219 >> 172.21.15.47:0/1512393219 conn(0x7f51fc0b9340 msgr2=0x7f51fc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 -- 172.21.15.47:0/1512393219 shutdown_connections 2020-07-18T03:33:19.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.746+0000 7f5200aae700 1 -- 172.21.15.47:0/1512393219 wait complete. 2020-07-18T03:33:19.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.747+0000 7f5200aae700 1 Processor -- start 2020-07-18T03:33:19.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.747+0000 7f5200aae700 1 -- start start 2020-07-18T03:33:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f5200aae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51fc13c3b0 0x7f51fc1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f5200aae700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc061a90 con 0x7f51fc05f5a0 2020-07-18T03:33:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f5200aae700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc061d30 con 0x7f51fc147a00 2020-07-18T03:33:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f5200aae700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc0669c0 con 0x7f51fc13c3b0 2020-07-18T03:33:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51fad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51fc13c3b0 0x7f51fc1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51f9d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51fc147a00 0x7f51fc0b23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44172/0 (socket says 172.21.15.47:44172) 2020-07-18T03:33:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51f9d9b700 1 -- 172.21.15.47:0/2806950601 learned_addr learned my addr 172.21.15.47:0/2806950601 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763949306 0 0) 0x7f51fc061d30 con 0x7f51fc147a00 2020-07-18T03:33:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d8003240 con 0x7f51fc147a00 2020-07-18T03:33:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51fad9d700 1 -- 172.21.15.47:0/2806950601 >> v1:172.21.15.47:6789/0 conn(0x7f51fc147a00 legacy=0x7f51fc0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51fad9d700 1 -- 172.21.15.47:0/2806950601 >> v1:172.21.15.18:6789/0 conn(0x7f51fc05f5a0 legacy=0x7f51fc13bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51fad9d700 1 -- 172.21.15.47:0/2806950601 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc061fd0 con 0x7f51fc13c3b0 2020-07-18T03:33:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.748+0000 7f51fad9d700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc13c3b0 0x7f51fc1466e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f51ec007000 tx=0x7f51ec008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.749+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ec00e680 con 0x7f51fc13c3b0 2020-07-18T03:33:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.749+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00c030 con 0x7f51fc13c3b0 2020-07-18T03:33:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.749+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc064540 con 0x7f51fc13c3b0 2020-07-18T03:33:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.749+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ec00e9d0 con 0x7f51fc13c3b0 2020-07-18T03:33:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.749+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc146e10 con 0x7f51fc13c3b0 2020-07-18T03:33:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.749+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc0621d0 con 0x7f51fc13c3b0 2020-07-18T03:33:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.750+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f51ec00ebd0 con 0x7f51fc13c3b0 2020-07-18T03:33:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.750+0000 7f51f27fc700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f51d80502e0 0x7f51d80527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.750+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f51ec067840 con 0x7f51fc13c3b0 2020-07-18T03:33:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.750+0000 7f51fa59c700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f51d80502e0 0x7f51d80527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.750+0000 7f51fa59c700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f51d80502e0 0x7f51d80527a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f51fc0b1cc0 tx=0x7f51f4016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:19.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.753+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f51ec03ff00 con 0x7f51fc13c3b0 2020-07-18T03:33:19.970 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.968+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51fc057450 con 0x7f51fc13c3b0 2020-07-18T03:33:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.969+0000 7f51f27fc700 1 -- 172.21.15.47:0/2806950601 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1788 (secure 0 0 0) 0x7f51ec065e60 con 0x7f51fc13c3b0 2020-07-18T03:33:19.971 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:19.972 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:19.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f51d80502e0 msgr2=0x7f51d80527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.973 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f51d80502e0 0x7f51d80527a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f51fc0b1cc0 tx=0x7f51f4016000).stop 2020-07-18T03:33:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc13c3b0 msgr2=0x7f51fc1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc13c3b0 0x7f51fc1466e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f51ec007000 tx=0x7f51ec008040).stop 2020-07-18T03:33:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 shutdown_connections 2020-07-18T03:33:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51fc13c3b0 0x7f51fc1466e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 --2- 172.21.15.47:0/2806950601 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f51d80502e0 0x7f51d80527a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 >> 172.21.15.47:0/2806950601 conn(0x7f51fc0b9340 msgr2=0x7f51fc0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 shutdown_connections 2020-07-18T03:33:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:19.970+0000 7f5200aae700 1 -- 172.21.15.47:0/2806950601 wait complete. 2020-07-18T03:33:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 33 2020-07-18T03:33:19.983 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:20.418 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:33:20.421 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:33:20.984 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:21.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.110+0000 7f41c3eab700 1 Processor -- start 2020-07-18T03:33:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.111+0000 7f41c3eab700 1 -- start start 2020-07-18T03:33:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.111+0000 7f41c3eab700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41bc061b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.111+0000 7f41c3eab700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc1441f0 con 0x7f41bc062000 2020-07-18T03:33:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.111+0000 7f41c3eab700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc144490 con 0x7f41bc0628a0 2020-07-18T03:33:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.111+0000 7f41c3eab700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc143fe0 con 0x7f41bc0ad990 2020-07-18T03:33:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c1c47700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f41bc0628a0 0x7f41bc13db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44176/0 (socket says 172.21.15.47:44176) 2020-07-18T03:33:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c1c47700 1 -- 172.21.15.47:0/738914481 learned_addr learned my addr 172.21.15.47:0/738914481 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c1446700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41bc061b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059563276 0 0) 0x7f41bc144490 con 0x7f41bc0628a0 2020-07-18T03:33:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a4003240 con 0x7f41bc0628a0 2020-07-18T03:33:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498146112 0 0) 0x7f41bc1441f0 con 0x7f41bc062000 2020-07-18T03:33:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41bc144490 con 0x7f41bc062000 2020-07-18T03:33:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4029000511 0 0) 0x7f41a4003240 con 0x7f41bc0628a0 2020-07-18T03:33:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 msgr2=0x7f41bc061b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41bc061b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 >> v1:172.21.15.18:6789/0 conn(0x7f41bc062000 legacy=0x7f41bc0af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc143c80 con 0x7f41bc0628a0 2020-07-18T03:33:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f41b00016d0 con 0x7f41bc0628a0 2020-07-18T03:33:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c1446700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41bc061b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 >> v1:172.21.15.47:6789/0 conn(0x7f41bc0628a0 legacy=0x7f41bc13db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40038c0 0x7f41a4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41a40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41a40049d0 0x7f41a4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc143fe0 con 0x7f41a40049d0 2020-07-18T03:33:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4005340 con 0x7f41a40038c0 2020-07-18T03:33:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400b730 con 0x7f41bc0ad990 2020-07-18T03:33:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c1c47700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40038c0 0x7f41a4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.112+0000 7f41c1446700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41a40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c2448700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41a40049d0 0x7f41a4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c1c47700 1 -- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 msgr2=0x7f41a40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c1c47700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f41bc0ad990 0x7f41a40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c1c47700 1 -- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41a40049d0 msgr2=0x7f41a4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c1c47700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41a40049d0 0x7f41a4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c1c47700 1 -- 172.21.15.47:0/738914481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4003240 con 0x7f41a40038c0 2020-07-18T03:33:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c1c47700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40038c0 0x7f41a4003cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f41b00014a0 tx=0x7f41b0007750).ready entity=mon.1 client_cookie=57c332f241803238 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41b000cdb0 con 0x7f41a40038c0 2020-07-18T03:33:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b00070d0 con 0x7f41a40038c0 2020-07-18T03:33:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c0c45700 1 -- 172.21.15.47:0/738914481 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41b0007430 con 0x7f41a40038c0 2020-07-18T03:33:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c3eab700 1 -- 172.21.15.47:0/738914481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40038c0 msgr2=0x7f41a4003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c3eab700 1 --2- 172.21.15.47:0/738914481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40038c0 0x7f41a4003cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f41b00014a0 tx=0x7f41b0007750).stop 2020-07-18T03:33:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c2448700 1 -- 172.21.15.47:0/738914481 reap_dead start 2020-07-18T03:33:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c3eab700 1 -- 172.21.15.47:0/738914481 shutdown_connections 2020-07-18T03:33:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c3eab700 1 -- 172.21.15.47:0/738914481 >> 172.21.15.47:0/738914481 conn(0x7f41bc0b92e0 msgr2=0x7f41bc0b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c3eab700 1 -- 172.21.15.47:0/738914481 shutdown_connections 2020-07-18T03:33:21.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.113+0000 7f41c3eab700 1 -- 172.21.15.47:0/738914481 wait complete. 2020-07-18T03:33:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.115+0000 7f41c3eab700 1 Processor -- start 2020-07-18T03:33:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.115+0000 7f41c3eab700 1 -- start start 2020-07-18T03:33:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.115+0000 7f41c3eab700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f41bc0b4830 0x7f41bc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c3eab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41bc065df0 0x7f41bc066240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c3eab700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc158670 con 0x7f41bc1558c0 2020-07-18T03:33:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c3eab700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc064a60 con 0x7f41bc0b4830 2020-07-18T03:33:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c3eab700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc155d50 con 0x7f41bc065df0 2020-07-18T03:33:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c2448700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41bc065df0 0x7f41bc066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c1446700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f41bc1558c0 0x7f41bc13a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44190/0 (socket says 172.21.15.47:44190) 2020-07-18T03:33:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c1446700 1 -- 172.21.15.47:0/1530198391 learned_addr learned my addr 172.21.15.47:0/1530198391 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c1c47700 1 --2- 172.21.15.47:0/1530198391 >> v2:172.21.15.18:3300/0 conn(0x7f41bc0b4830 0x7f41bc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c2448700 1 -- 172.21.15.47:0/1530198391 >> v2:172.21.15.18:3300/0 conn(0x7f41bc0b4830 msgr2=0x7f41bc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c2448700 1 --2- 172.21.15.47:0/1530198391 >> v2:172.21.15.18:3300/0 conn(0x7f41bc0b4830 0x7f41bc0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c2448700 1 -- 172.21.15.47:0/1530198391 >> v1:172.21.15.47:6789/0 conn(0x7f41bc1558c0 legacy=0x7f41bc13a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c2448700 1 -- 172.21.15.47:0/1530198391 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc158910 con 0x7f41bc065df0 2020-07-18T03:33:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324624206 0 0) 0x7f41bc158670 con 0x7f41bc1558c0 2020-07-18T03:33:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c2448700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41bc065df0 0x7f41bc066240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f41b4000b20 tx=0x7f41b4007e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41b4006c70 con 0x7f41bc065df0 2020-07-18T03:33:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b40011d0 con 0x7f41bc065df0 2020-07-18T03:33:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc158bb0 con 0x7f41bc065df0 2020-07-18T03:33:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.116+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41b4007070 con 0x7f41bc065df0 2020-07-18T03:33:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.117+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc13b9d0 con 0x7f41bc065df0 2020-07-18T03:33:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.118+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f41b40071d0 con 0x7f41bc065df0 2020-07-18T03:33:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.118+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc158d40 con 0x7f41bc065df0 2020-07-18T03:33:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.118+0000 7f41adffb700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f419804fe00 0x7f41980522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.118+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f41b4067900 con 0x7f41bc065df0 2020-07-18T03:33:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.118+0000 7f41c1c47700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f419804fe00 0x7f41980522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.118+0000 7f41c1c47700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f419804fe00 0x7f41980522c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f41b0019000 tx=0x7f41b0015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:21.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.121+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f41b4040040 con 0x7f41bc065df0 2020-07-18T03:33:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.332+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41bc0690d0 con 0x7f41bc065df0 2020-07-18T03:33:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.332+0000 7f41adffb700 1 -- 172.21.15.47:0/1530198391 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1788 (secure 0 0 0) 0x7f41b4043660 con 0x7f41bc065df0 2020-07-18T03:33:21.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:21.335 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f419804fe00 msgr2=0x7f41980522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f419804fe00 0x7f41980522c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f41b0019000 tx=0x7f41b0015000).stop 2020-07-18T03:33:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41bc065df0 msgr2=0x7f41bc066240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41bc065df0 0x7f41bc066240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f41b4000b20 tx=0x7f41b4007e60).stop 2020-07-18T03:33:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 shutdown_connections 2020-07-18T03:33:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41bc065df0 0x7f41bc066240 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 --2- 172.21.15.47:0/1530198391 >> v2:172.21.15.18:3300/0 conn(0x7f41bc0b4830 0x7f41bc0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 --2- 172.21.15.47:0/1530198391 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f419804fe00 0x7f41980522c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 >> 172.21.15.47:0/1530198391 conn(0x7f41bc0b92e0 msgr2=0x7f41bc0b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 shutdown_connections 2020-07-18T03:33:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:21.334+0000 7f41c3eab700 1 -- 172.21.15.47:0/1530198391 wait complete. 2020-07-18T03:33:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 33 2020-07-18T03:33:21.346 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:22.350 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.479+0000 7ff79b148700 1 Processor -- start 2020-07-18T03:33:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.479+0000 7ff79b148700 1 -- start start 2020-07-18T03:33:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff79b148700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7940ad100 0x7ff7940b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff79b148700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff794147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff79b148700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff794147330 con 0x7ff794147a00 2020-07-18T03:33:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff79b148700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7940ae530 con 0x7ff7940ad100 2020-07-18T03:33:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff79b148700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b57e0 con 0x7ff7941443f0 2020-07-18T03:33:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff798ee4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7940ad100 0x7ff7940b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff793fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff794147a00 0x7ff79405f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44198/0 (socket says 172.21.15.47:44198) 2020-07-18T03:33:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff793fff700 1 -- 172.21.15.47:0/2767594386 learned_addr learned my addr 172.21.15.47:0/2767594386 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.480+0000 7ff7996e5700 1 --2- 172.21.15.47:0/2767594386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff794147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:22.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff798ee4700 1 -- 172.21.15.47:0/2767594386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 msgr2=0x7ff794147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:22.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff798ee4700 1 --2- 172.21.15.47:0/2767594386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff794147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff798ee4700 1 -- 172.21.15.47:0/2767594386 >> v1:172.21.15.47:6789/0 conn(0x7ff794147a00 legacy=0x7ff79405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:22.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff7937fe700 1 -- 172.21.15.47:0/2767594386 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614256055 0 0) 0x7ff794147330 con 0x7ff794147a00 2020-07-18T03:33:22.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff798ee4700 1 -- 172.21.15.47:0/2767594386 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940b5a80 con 0x7ff7940ad100 2020-07-18T03:33:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff798ee4700 1 --2- 172.21.15.47:0/2767594386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ad100 0x7ff7940b4c10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7ff784002f30 tx=0x7ff784006dd0).ready entity=mon.1 client_cookie=b9eed9e5f88e7a65 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff7937fe700 1 -- 172.21.15.47:0/2767594386 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff784009760 con 0x7ff7940ad100 2020-07-18T03:33:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff7937fe700 1 -- 172.21.15.47:0/2767594386 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7840098c0 con 0x7ff7940ad100 2020-07-18T03:33:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff7937fe700 1 -- 172.21.15.47:0/2767594386 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff784009c20 con 0x7ff7940ad100 2020-07-18T03:33:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 -- 172.21.15.47:0/2767594386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ad100 msgr2=0x7ff7940b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 --2- 172.21.15.47:0/2767594386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ad100 0x7ff7940b4c10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7ff784002f30 tx=0x7ff784006dd0).stop 2020-07-18T03:33:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 -- 172.21.15.47:0/2767594386 shutdown_connections 2020-07-18T03:33:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 --2- 172.21.15.47:0/2767594386 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff794147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 --2- 172.21.15.47:0/2767594386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ad100 0x7ff7940b4c10 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 -- 172.21.15.47:0/2767594386 >> 172.21.15.47:0/2767594386 conn(0x7ff7940b9340 msgr2=0x7ff7940b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 -- 172.21.15.47:0/2767594386 shutdown_connections 2020-07-18T03:33:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.481+0000 7ff79b148700 1 -- 172.21.15.47:0/2767594386 wait complete. 2020-07-18T03:33:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.483+0000 7ff79b148700 1 Processor -- start 2020-07-18T03:33:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.483+0000 7ff79b148700 1 -- start start 2020-07-18T03:33:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.483+0000 7ff79b148700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff7941417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff79b148700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff794141cb0 0x7ff7940b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff79b148700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7940b3200 con 0x7ff794147a00 2020-07-18T03:33:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff79b148700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff794141320 con 0x7ff794141cb0 2020-07-18T03:33:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff79b148700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b27a0 con 0x7ff7941443f0 2020-07-18T03:33:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff793fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff794147a00 0x7ff7940b2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44204/0 (socket says 172.21.15.47:44204) 2020-07-18T03:33:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff793fff700 1 -- 172.21.15.47:0/2539704229 learned_addr learned my addr 172.21.15.47:0/2539704229 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff7996e5700 1 --2- 172.21.15.47:0/2539704229 >> v2:172.21.15.47:3300/0 conn(0x7ff794141cb0 0x7ff7940b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff798ee4700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff7941417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349787628 0 0) 0x7ff7940b3200 con 0x7ff794147a00 2020-07-18T03:33:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff76c003240 con 0x7ff794147a00 2020-07-18T03:33:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff7996e5700 1 -- 172.21.15.47:0/2539704229 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 msgr2=0x7ff7941417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff7996e5700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff7941417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff7996e5700 1 -- 172.21.15.47:0/2539704229 >> v1:172.21.15.47:6789/0 conn(0x7ff794147a00 legacy=0x7ff7940b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff7996e5700 1 -- 172.21.15.47:0/2539704229 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940b2dd0 con 0x7ff794141cb0 2020-07-18T03:33:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff7996e5700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff794141cb0 0x7ff7940b0f30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7ff78c00c000 tx=0x7ff78c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff78c00c950 con 0x7ff794141cb0 2020-07-18T03:33:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00cab0 con 0x7ff794141cb0 2020-07-18T03:33:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff78c00ce10 con 0x7ff794141cb0 2020-07-18T03:33:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7940b2fc0 con 0x7ff794141cb0 2020-07-18T03:33:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.484+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940b1420 con 0x7ff794141cb0 2020-07-18T03:33:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.485+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940aec80 con 0x7ff794141cb0 2020-07-18T03:33:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.485+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff78c022070 con 0x7ff794141cb0 2020-07-18T03:33:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.486+0000 7ff790ff9700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff76c0503b0 0x7ff76c052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.486+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff78c0680a0 con 0x7ff794141cb0 2020-07-18T03:33:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.486+0000 7ff798ee4700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff76c0503b0 0x7ff76c052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.486+0000 7ff798ee4700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff76c0503b0 0x7ff76c052870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7ff784019000 tx=0x7ff784015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.489+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff78c040890 con 0x7ff794141cb0 2020-07-18T03:33:22.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.700+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7940b3390 con 0x7ff794141cb0 2020-07-18T03:33:22.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.700+0000 7ff790ff9700 1 -- 172.21.15.47:0/2539704229 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7ff78c016090 con 0x7ff794141cb0 2020-07-18T03:33:22.702 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:22.703 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff76c0503b0 msgr2=0x7ff76c052870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff76c0503b0 0x7ff76c052870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7ff784019000 tx=0x7ff784015000).stop 2020-07-18T03:33:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff794141cb0 msgr2=0x7ff7940b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff794141cb0 0x7ff7940b0f30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7ff78c00c000 tx=0x7ff78c0091d0).stop 2020-07-18T03:33:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 shutdown_connections 2020-07-18T03:33:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff794141cb0 0x7ff7940b0f30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff7941443f0 0x7ff7941417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.701+0000 7ff79b148700 1 --2- 172.21.15.47:0/2539704229 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff76c0503b0 0x7ff76c052870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.702+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 >> 172.21.15.47:0/2539704229 conn(0x7ff7940b9340 msgr2=0x7ff794143820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.702+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 shutdown_connections 2020-07-18T03:33:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:22.702+0000 7ff79b148700 1 -- 172.21.15.47:0/2539704229 wait complete. 2020-07-18T03:33:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:22.714 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:23.715 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:23.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 Processor -- start 2020-07-18T03:33:23.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 -- start start 2020-07-18T03:33:23.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47f40b5d30 0x7f47f40ba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:23.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f47f40637d0 0x7f47f4063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:23.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47f41464c0 0x7f47f4146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:23.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40b6470 con 0x7f47f40637d0 2020-07-18T03:33:23.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40b61d0 con 0x7f47f40b5d30 2020-07-18T03:33:23.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.842+0000 7f47f9b92700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f414a5b0 con 0x7f47f41464c0 2020-07-18T03:33:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47f40b5d30 0x7f47f40ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47f40b5d30 0x7f47f40ba460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34890/0 (socket says 172.21.15.47:34890) 2020-07-18T03:33:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 -- 172.21.15.47:0/2435849774 learned_addr learned my addr 172.21.15.47:0/2435849774 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f3fff700 1 --2- 172.21.15.47:0/2435849774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47f41464c0 0x7f47f4146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f2ffd700 1 --2- 172.21.15.47:0/2435849774 >> v2:172.21.15.18:3300/0 conn(0x7f47f40637d0 0x7f47f4063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 -- 172.21.15.47:0/2435849774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47f41464c0 msgr2=0x7f47f4146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 --2- 172.21.15.47:0/2435849774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47f41464c0 0x7f47f4146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 -- 172.21.15.47:0/2435849774 >> v2:172.21.15.18:3300/0 conn(0x7f47f40637d0 msgr2=0x7f47f4063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 --2- 172.21.15.47:0/2435849774 >> v2:172.21.15.18:3300/0 conn(0x7f47f40637d0 0x7f47f4063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 -- 172.21.15.47:0/2435849774 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f4066ac0 con 0x7f47f40b5d30 2020-07-18T03:33:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f37fe700 1 --2- 172.21.15.47:0/2435849774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40b5d30 0x7f47f40ba460 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f47e4002f30 tx=0x7f47e4006e90).ready entity=mon.1 client_cookie=fe3167d7c26f6350 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f27fc700 1 -- 172.21.15.47:0/2435849774 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47e4009660 con 0x7f47f40b5d30 2020-07-18T03:33:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f27fc700 1 -- 172.21.15.47:0/2435849774 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40097c0 con 0x7f47f40b5d30 2020-07-18T03:33:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f27fc700 1 -- 172.21.15.47:0/2435849774 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47e4009b20 con 0x7f47f40b5d30 2020-07-18T03:33:23.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 -- 172.21.15.47:0/2435849774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40b5d30 msgr2=0x7f47f40ba460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 --2- 172.21.15.47:0/2435849774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40b5d30 0x7f47f40ba460 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f47e4002f30 tx=0x7f47e4006e90).stop 2020-07-18T03:33:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 -- 172.21.15.47:0/2435849774 shutdown_connections 2020-07-18T03:33:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 --2- 172.21.15.47:0/2435849774 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47f41464c0 0x7f47f4146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 --2- 172.21.15.47:0/2435849774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40b5d30 0x7f47f40ba460 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 --2- 172.21.15.47:0/2435849774 >> v2:172.21.15.18:3300/0 conn(0x7f47f40637d0 0x7f47f4063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:23.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 -- 172.21.15.47:0/2435849774 >> 172.21.15.47:0/2435849774 conn(0x7f47f40b7100 msgr2=0x7f47f40b9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:23.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 -- 172.21.15.47:0/2435849774 shutdown_connections 2020-07-18T03:33:23.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.843+0000 7f47f9b92700 1 -- 172.21.15.47:0/2435849774 wait complete. 2020-07-18T03:33:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.845+0000 7f47f9b92700 1 Processor -- start 2020-07-18T03:33:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.845+0000 7f47f9b92700 1 -- start start 2020-07-18T03:33:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f9b92700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f47f41464c0 0x7f47f40b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:23.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f9b92700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f41481d0 con 0x7f47f41135d0 2020-07-18T03:33:23.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f9b92700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f4148470 con 0x7f47f40637d0 2020-07-18T03:33:23.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f9b92700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f4148710 con 0x7f47f41464c0 2020-07-18T03:33:23.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47f40637d0 0x7f47f40b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44216/0 (socket says 172.21.15.47:44216) 2020-07-18T03:33:23.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f37fe700 1 -- 172.21.15.47:0/3749325315 learned_addr learned my addr 172.21.15.47:0/3749325315 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:23.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f2ffd700 1 --2- 172.21.15.47:0/3749325315 >> v2:172.21.15.18:3300/0 conn(0x7f47f41464c0 0x7f47f40b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:23.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811305893 0 0) 0x7f47f4148470 con 0x7f47f40637d0 2020-07-18T03:33:23.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47cc003240 con 0x7f47f40637d0 2020-07-18T03:33:23.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 874343641 0 0) 0x7f47cc003240 con 0x7f47f40637d0 2020-07-18T03:33:23.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.846+0000 7f47f2ffd700 1 -- 172.21.15.47:0/3749325315 >> v1:172.21.15.47:6789/0 conn(0x7f47f40637d0 legacy=0x7f47f40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:23.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47f2ffd700 1 -- 172.21.15.47:0/3749325315 >> v1:172.21.15.18:6789/0 conn(0x7f47f41135d0 legacy=0x7f47f4112b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:23.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47f2ffd700 1 -- 172.21.15.47:0/3749325315 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40b3970 con 0x7f47f41464c0 2020-07-18T03:33:23.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47f2ffd700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f41464c0 0x7f47f40b1a00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f47e40090c0 tx=0x7f47e8006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:23.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47e4009660 con 0x7f47f41464c0 2020-07-18T03:33:23.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f4113280 con 0x7f47f41464c0 2020-07-18T03:33:23.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800c7a0 con 0x7f47f41464c0 2020-07-18T03:33:23.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47e800cb00 con 0x7f47f41464c0 2020-07-18T03:33:23.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.847+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f4159230 con 0x7f47f41464c0 2020-07-18T03:33:23.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.848+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4057450 con 0x7f47f41464c0 2020-07-18T03:33:23.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.848+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f47e800ccb0 con 0x7f47f41464c0 2020-07-18T03:33:23.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.848+0000 7f47d37fe700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47cc0502c0 0x7f47cc052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:23.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.849+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f47e8067cd0 con 0x7f47f41464c0 2020-07-18T03:33:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.849+0000 7f47f37fe700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47cc0502c0 0x7f47cc052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.849+0000 7f47f37fe700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47cc0502c0 0x7f47cc052780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f47e400b870 tx=0x7f47e4015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:23.853+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f47e8040390 con 0x7f47f41464c0 2020-07-18T03:33:24.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.062+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47f406a690 con 0x7f47f41464c0 2020-07-18T03:33:24.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.063+0000 7f47d37fe700 1 -- 172.21.15.47:0/3749325315 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f47e80439b0 con 0x7f47f41464c0 2020-07-18T03:33:24.064 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:24.064 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:24.065 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47cc0502c0 msgr2=0x7f47cc052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:24.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47cc0502c0 0x7f47cc052780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f47e400b870 tx=0x7f47e4015000).stop 2020-07-18T03:33:24.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f41464c0 msgr2=0x7f47f40b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:24.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f41464c0 0x7f47f40b1a00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f47e40090c0 tx=0x7f47e8006dd0).stop 2020-07-18T03:33:24.066 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 shutdown_connections 2020-07-18T03:33:24.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47cc0502c0 0x7f47cc052780 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:24.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 --2- 172.21.15.47:0/3749325315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f41464c0 0x7f47f40b1a00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:24.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 >> 172.21.15.47:0/3749325315 conn(0x7f47f40b7100 msgr2=0x7f47f40b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:24.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 shutdown_connections 2020-07-18T03:33:24.067 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:24.064+0000 7f47f9b92700 1 -- 172.21.15.47:0/3749325315 wait complete. 2020-07-18T03:33:24.068 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:24.076 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:25.078 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:25.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.204+0000 7f47d64eb700 1 Processor -- start 2020-07-18T03:33:25.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.204+0000 7f47d64eb700 1 -- start start 2020-07-18T03:33:25.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d64eb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47d0147a00 0x7f47d0147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:25.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d64eb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d01433e0 0x7f47d0143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:25.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d64eb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d0147330 con 0x7f47d00ad100 2020-07-18T03:33:25.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d64eb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d00ae530 con 0x7f47d01433e0 2020-07-18T03:33:25.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d64eb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00b57e0 con 0x7f47d0147a00 2020-07-18T03:33:25.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d4a88700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d01433e0 0x7f47d0143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:25.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d4a88700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d01433e0 0x7f47d0143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34906/0 (socket says 172.21.15.47:34906) 2020-07-18T03:33:25.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47cffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47d00ad100 0x7f47d00b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44224/0 (socket says 172.21.15.47:44224) 2020-07-18T03:33:25.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47d4a88700 1 -- 172.21.15.47:0/874896444 learned_addr learned my addr 172.21.15.47:0/874896444 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:25.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.205+0000 7f47cf7fe700 1 --2- 172.21.15.47:0/874896444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47d0147a00 0x7f47d0147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:25.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d4a88700 1 -- 172.21.15.47:0/874896444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47d0147a00 msgr2=0x7f47d0147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:25.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d4a88700 1 --2- 172.21.15.47:0/874896444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47d0147a00 0x7f47d0147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:25.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d4a88700 1 -- 172.21.15.47:0/874896444 >> v1:172.21.15.47:6789/0 conn(0x7f47d00ad100 legacy=0x7f47d00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d4a88700 1 -- 172.21.15.47:0/874896444 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00b5a80 con 0x7f47d01433e0 2020-07-18T03:33:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d4a88700 1 --2- 172.21.15.47:0/874896444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d01433e0 0x7f47d0143850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f47c00010d0 tx=0x7f47c8009270).ready entity=mon.1 client_cookie=582059e8875d57b3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47ceffd700 1 -- 172.21.15.47:0/874896444 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800bac0 con 0x7f47d01433e0 2020-07-18T03:33:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47ceffd700 1 -- 172.21.15.47:0/874896444 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800bc20 con 0x7f47d01433e0 2020-07-18T03:33:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 -- 172.21.15.47:0/874896444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d01433e0 msgr2=0x7f47d0143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47ceffd700 1 -- 172.21.15.47:0/874896444 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800bd90 con 0x7f47d01433e0 2020-07-18T03:33:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 --2- 172.21.15.47:0/874896444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d01433e0 0x7f47d0143850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f47c00010d0 tx=0x7f47c8009270).stop 2020-07-18T03:33:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 -- 172.21.15.47:0/874896444 shutdown_connections 2020-07-18T03:33:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 --2- 172.21.15.47:0/874896444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d01433e0 0x7f47d0143850 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 --2- 172.21.15.47:0/874896444 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f47d0147a00 0x7f47d0147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 -- 172.21.15.47:0/874896444 >> 172.21.15.47:0/874896444 conn(0x7f47d00b9340 msgr2=0x7f47d00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:25.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 -- 172.21.15.47:0/874896444 shutdown_connections 2020-07-18T03:33:25.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.206+0000 7f47d64eb700 1 -- 172.21.15.47:0/874896444 wait complete. 2020-07-18T03:33:25.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.207+0000 7f47d64eb700 1 Processor -- start 2020-07-18T03:33:25.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47d64eb700 1 -- start start 2020-07-18T03:33:25.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47d64eb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d00ad100 0x7f47d01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:25.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47d64eb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d013ccf0 con 0x7f47d0066170 2020-07-18T03:33:25.222 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47d64eb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d013cf20 con 0x7f47d0147a00 2020-07-18T03:33:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47d64eb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d0064170 con 0x7f47d00ad100 2020-07-18T03:33:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47cffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d00ad100 0x7f47d01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47cf7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47d0147a00 0x7f47d0155eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44232/0 (socket says 172.21.15.47:44232) 2020-07-18T03:33:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.208+0000 7f47cf7fe700 1 -- 172.21.15.47:0/2028723951 learned_addr learned my addr 172.21.15.47:0/2028723951 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47cffff700 1 -- 172.21.15.47:0/2028723951 >> v1:172.21.15.47:6789/0 conn(0x7f47d0147a00 legacy=0x7f47d0155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:25.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47cffff700 1 -- 172.21.15.47:0/2028723951 >> v1:172.21.15.18:6789/0 conn(0x7f47d0066170 legacy=0x7f47d01517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:25.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47cffff700 1 -- 172.21.15.47:0/2028723951 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d013d1c0 con 0x7f47d00ad100 2020-07-18T03:33:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822477061 0 0) 0x7f47d013cf20 con 0x7f47d0147a00 2020-07-18T03:33:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47cffff700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d00ad100 0x7f47d01559c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f47c0001190 tx=0x7f47c0007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c0007320 con 0x7f47d00ad100 2020-07-18T03:33:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0005530 con 0x7f47d00ad100 2020-07-18T03:33:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00656d0 con 0x7f47d00ad100 2020-07-18T03:33:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c0005890 con 0x7f47d00ad100 2020-07-18T03:33:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.209+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d015c990 con 0x7f47d00ad100 2020-07-18T03:33:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.210+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f47c0010030 con 0x7f47d00ad100 2020-07-18T03:33:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.210+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d0057450 con 0x7f47d00ad100 2020-07-18T03:33:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.210+0000 7f47affff700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47a804fdc0 0x7f47a8052280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.211+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f47c0067b70 con 0x7f47d00ad100 2020-07-18T03:33:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.211+0000 7f47cf7fe700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47a804fdc0 0x7f47a8052280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.211+0000 7f47cf7fe700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47a804fdc0 0x7f47a8052280 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f47c4002f30 tx=0x7f47c4008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.214+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f47c0040230 con 0x7f47d00ad100 2020-07-18T03:33:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.431+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d00b53b0 con 0x7f47d00ad100 2020-07-18T03:33:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.431+0000 7f47affff700 1 -- 172.21.15.47:0/2028723951 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f47c0043850 con 0x7f47d00ad100 2020-07-18T03:33:25.433 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:25.433 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:25.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47a804fdc0 msgr2=0x7f47a8052280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47a804fdc0 0x7f47a8052280 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f47c4002f30 tx=0x7f47c4008040).stop 2020-07-18T03:33:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d00ad100 msgr2=0x7f47d01559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d00ad100 0x7f47d01559c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f47c0001190 tx=0x7f47c0007a90).stop 2020-07-18T03:33:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 shutdown_connections 2020-07-18T03:33:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d00ad100 0x7f47d01559c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 --2- 172.21.15.47:0/2028723951 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f47a804fdc0 0x7f47a8052280 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 >> 172.21.15.47:0/2028723951 conn(0x7f47d00b9340 msgr2=0x7f47d01469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 shutdown_connections 2020-07-18T03:33:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:25.433+0000 7f47d64eb700 1 -- 172.21.15.47:0/2028723951 wait complete. 2020-07-18T03:33:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:25.444 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:26.445 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:26.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.573+0000 7f29d18d1700 1 Processor -- start 2020-07-18T03:33:26.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.573+0000 7f29d18d1700 1 -- start start 2020-07-18T03:33:26.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29d18d1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f29cc0b2e10 0x7f29cc0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29d18d1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29cc0b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29d18d1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc0afbd0 con 0x7f29cc0b2130 2020-07-18T03:33:26.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29d18d1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc144e30 con 0x7f29cc0b2e10 2020-07-18T03:33:26.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29d18d1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1491c0 con 0x7f29cc0b1560 2020-07-18T03:33:26.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29cb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f29cc0b2130 0x7f29cc1454f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44242/0 (socket says 172.21.15.47:44242) 2020-07-18T03:33:26.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29cb7fe700 1 -- 172.21.15.47:0/1259124111 learned_addr learned my addr 172.21.15.47:0/1259124111 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:26.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29cc0b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29caffd700 1 --2- 172.21.15.47:0/1259124111 >> v2:172.21.15.18:3300/0 conn(0x7f29cc0b2e10 0x7f29cc0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357525623 0 0) 0x7f29c4001450 con 0x7f29cc0b2130 2020-07-18T03:33:26.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b8003240 con 0x7f29cc0b2130 2020-07-18T03:33:26.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 138603074 0 0) 0x7f29b8003240 con 0x7f29cc0b2130 2020-07-18T03:33:26.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 msgr2=0x7f29cc0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:26.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29cc0b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 >> v2:172.21.15.18:3300/0 conn(0x7f29cc0b2e10 msgr2=0x7f29cc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:26.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 --2- 172.21.15.47:0/1259124111 >> v2:172.21.15.18:3300/0 conn(0x7f29cc0b2e10 0x7f29cc0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.584 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc145050 con 0x7f29cc0b2130 2020-07-18T03:33:26.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f29c4001be0 con 0x7f29cc0b2130 2020-07-18T03:33:26.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.574+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29cc0b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:26.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 >> v1:172.21.15.47:6789/0 conn(0x7f29cc0b2130 legacy=0x7f29cc1454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:26.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b8003730 0x7f29b8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29b80044e0 0x7f29b8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29b8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc144e30 con 0x7f29b80044e0 2020-07-18T03:33:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1491c0 con 0x7f29b8003730 2020-07-18T03:33:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8004250 con 0x7f29cc0b1560 2020-07-18T03:33:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b8003730 0x7f29b8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29b80044e0 0x7f29b8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29cb7fe700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29b8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 -- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 msgr2=0x7f29b8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:26.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc0b1560 0x7f29b8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 -- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29b80044e0 msgr2=0x7f29b8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:26.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29b80044e0 0x7f29b8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 -- 172.21.15.47:0/1259124111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8003240 con 0x7f29b8003730 2020-07-18T03:33:26.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29b80044e0 0x7f29b8004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:26.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29caffd700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b8003730 0x7f29b8003b50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f29c000c000 tx=0x7f29c0008040).ready entity=mon.1 client_cookie=b81a81cec8193cdf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:26.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c000ebc0 con 0x7f29b8003730 2020-07-18T03:33:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000ed20 con 0x7f29b8003730 2020-07-18T03:33:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29c9ffb700 1 -- 172.21.15.47:0/1259124111 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c00125e0 con 0x7f29b8003730 2020-07-18T03:33:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29d18d1700 1 -- 172.21.15.47:0/1259124111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b8003730 msgr2=0x7f29b8003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29d18d1700 1 --2- 172.21.15.47:0/1259124111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b8003730 0x7f29b8003b50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f29c000c000 tx=0x7f29c0008040).stop 2020-07-18T03:33:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29cb7fe700 1 -- 172.21.15.47:0/1259124111 reap_dead start 2020-07-18T03:33:26.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29d18d1700 1 -- 172.21.15.47:0/1259124111 shutdown_connections 2020-07-18T03:33:26.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29d18d1700 1 -- 172.21.15.47:0/1259124111 >> 172.21.15.47:0/1259124111 conn(0x7f29cc0b9340 msgr2=0x7f29cc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:26.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29d18d1700 1 -- 172.21.15.47:0/1259124111 shutdown_connections 2020-07-18T03:33:26.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.575+0000 7f29d18d1700 1 -- 172.21.15.47:0/1259124111 wait complete. 2020-07-18T03:33:26.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.577+0000 7f29d18d1700 1 Processor -- start 2020-07-18T03:33:26.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.577+0000 7f29d18d1700 1 -- start start 2020-07-18T03:33:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29d18d1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29cc141760 0x7f29cc064cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29d18d1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f29cc141b40 0x7f29cc1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29d18d1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc061950 0x7f29cc0655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29d18d1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc066360 con 0x7f29cc141b40 2020-07-18T03:33:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29d18d1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc066600 con 0x7f29cc141760 2020-07-18T03:33:26.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29d18d1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1575c0 con 0x7f29cc061950 2020-07-18T03:33:26.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29cc141760 0x7f29cc064cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29cc141760 0x7f29cc064cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34930/0 (socket says 172.21.15.47:34930) 2020-07-18T03:33:26.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 -- 172.21.15.47:0/42169195 learned_addr learned my addr 172.21.15.47:0/42169195 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:26.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/42169195 >> v2:172.21.15.18:3300/0 conn(0x7f29cc141b40 0x7f29cc1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29cb7fe700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc061950 0x7f29cc0655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 -- 172.21.15.47:0/42169195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc061950 msgr2=0x7f29cc0655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc061950 0x7f29cc0655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 -- 172.21.15.47:0/42169195 >> v2:172.21.15.18:3300/0 conn(0x7f29cc141b40 msgr2=0x7f29cc1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 --2- 172.21.15.47:0/42169195 >> v2:172.21.15.18:3300/0 conn(0x7f29cc141b40 0x7f29cc1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 -- 172.21.15.47:0/42169195 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0668a0 con 0x7f29cc141760 2020-07-18T03:33:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29caffd700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29cc141760 0x7f29cc064cf0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f29c000c000 tx=0x7f29c0019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.578+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c00128c0 con 0x7f29cc141760 2020-07-18T03:33:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.579+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c0012a20 con 0x7f29cc141760 2020-07-18T03:33:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.579+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc065ab0 con 0x7f29cc141760 2020-07-18T03:33:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.579+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c0012d80 con 0x7f29cc141760 2020-07-18T03:33:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.579+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc153480 con 0x7f29cc141760 2020-07-18T03:33:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.579+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc057450 con 0x7f29cc141760 2020-07-18T03:33:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.580+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f29c002d3f0 con 0x7f29cc141760 2020-07-18T03:33:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.580+0000 7f29b2ffd700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f29a804ff20 0x7f29a80523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.580+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f29c0069e60 con 0x7f29cc141760 2020-07-18T03:33:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.580+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f29a804ff20 0x7f29a80523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.580+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f29a804ff20 0x7f29a80523e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f29bc008000 tx=0x7f29bc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.583+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f29c0042570 con 0x7f29cc141760 2020-07-18T03:33:26.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.796+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29cc066b60 con 0x7f29cc141760 2020-07-18T03:33:26.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.796+0000 7f29b2ffd700 1 -- 172.21.15.47:0/42169195 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f29c0045b90 con 0x7f29cc141760 2020-07-18T03:33:26.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:26.798 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:26.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f29a804ff20 msgr2=0x7f29a80523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:26.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f29a804ff20 0x7f29a80523e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f29bc008000 tx=0x7f29bc008040).stop 2020-07-18T03:33:26.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29cc141760 msgr2=0x7f29cc064cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:26.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29cc141760 0x7f29cc064cf0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f29c000c000 tx=0x7f29c0019000).stop 2020-07-18T03:33:26.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 shutdown_connections 2020-07-18T03:33:26.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f29a804ff20 0x7f29a80523e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f29cc061950 0x7f29cc0655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 --2- 172.21.15.47:0/42169195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29cc141760 0x7f29cc064cf0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 --2- 172.21.15.47:0/42169195 >> v2:172.21.15.18:3300/0 conn(0x7f29cc141b40 0x7f29cc1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:26.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 >> 172.21.15.47:0/42169195 conn(0x7f29cc0b9340 msgr2=0x7f29cc05d340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:26.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 shutdown_connections 2020-07-18T03:33:26.802 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:26.798+0000 7f29d18d1700 1 -- 172.21.15.47:0/42169195 wait complete. 2020-07-18T03:33:26.802 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:26.809 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:27.811 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.938+0000 7f6a459e6700 1 Processor -- start 2020-07-18T03:33:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.938+0000 7f6a459e6700 1 -- start start 2020-07-18T03:33:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a459e6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 0x7f6a4005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a459e6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a400b3ac0 con 0x7f6a400b4620 2020-07-18T03:33:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a459e6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a400b3d20 con 0x7f6a400b41c0 2020-07-18T03:33:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a459e6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400b5a10 con 0x7f6a400b5370 2020-07-18T03:33:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3e7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6a400b41c0 0x7f6a400ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44260/0 (socket says 172.21.15.47:44260) 2020-07-18T03:33:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3e7fc700 1 -- 172.21.15.47:0/2164867269 learned_addr learned my addr 172.21.15.47:0/2164867269 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3effd700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 0x7f6a4005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592447763 0 0) 0x7f6a400b3d20 con 0x7f6a400b41c0 2020-07-18T03:33:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a20003240 con 0x7f6a400b41c0 2020-07-18T03:33:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2151950994 0 0) 0x7f6a20003240 con 0x7f6a400b41c0 2020-07-18T03:33:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 msgr2=0x7f6a4005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 0x7f6a4005fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 >> v1:172.21.15.18:6789/0 conn(0x7f6a400b4620 legacy=0x7f6a401428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400b5750 con 0x7f6a400b41c0 2020-07-18T03:33:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f6a280016d0 con 0x7f6a400b41c0 2020-07-18T03:33:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 >> v1:172.21.15.47:6789/0 conn(0x7f6a400b41c0 legacy=0x7f6a400ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 0x7f6a20003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a20003c40 0x7f6a200044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a200049d0 0x7f6a20004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400b5a10 con 0x7f6a20003c40 2020-07-18T03:33:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a200041d0 con 0x7f6a200049d0 2020-07-18T03:33:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20005700 con 0x7f6a400b5370 2020-07-18T03:33:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.939+0000 7f6a3f7fe700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a200049d0 0x7f6a20004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3effd700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 0x7f6a20003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3e7fc700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a20003c40 0x7f6a200044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 -- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 msgr2=0x7f6a20003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a400b5370 0x7f6a20003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 -- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a20003c40 msgr2=0x7f6a200044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a20003c40 0x7f6a200044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 -- 172.21.15.47:0/2164867269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a20003240 con 0x7f6a200049d0 2020-07-18T03:33:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3e7fc700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a20003c40 0x7f6a200044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a200049d0 0x7f6a20004e50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f6a40067280 tx=0x7f6a30007e60).ready entity=mon.1 client_cookie=f45ad7611e7cb1d2 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3000ca10 con 0x7f6a200049d0 2020-07-18T03:33:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3000cb70 con 0x7f6a200049d0 2020-07-18T03:33:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3dffb700 1 -- 172.21.15.47:0/2164867269 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a30012400 con 0x7f6a200049d0 2020-07-18T03:33:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a459e6700 1 -- 172.21.15.47:0/2164867269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a200049d0 msgr2=0x7f6a20004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a459e6700 1 --2- 172.21.15.47:0/2164867269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a200049d0 0x7f6a20004e50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f6a40067280 tx=0x7f6a30007e60).stop 2020-07-18T03:33:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a3f7fe700 1 -- 172.21.15.47:0/2164867269 reap_dead start 2020-07-18T03:33:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a459e6700 1 -- 172.21.15.47:0/2164867269 shutdown_connections 2020-07-18T03:33:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a459e6700 1 -- 172.21.15.47:0/2164867269 >> 172.21.15.47:0/2164867269 conn(0x7f6a400b9340 msgr2=0x7f6a400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a459e6700 1 -- 172.21.15.47:0/2164867269 shutdown_connections 2020-07-18T03:33:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.940+0000 7f6a459e6700 1 -- 172.21.15.47:0/2164867269 wait complete. 2020-07-18T03:33:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.942+0000 7f6a459e6700 1 Processor -- start 2020-07-18T03:33:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.942+0000 7f6a459e6700 1 -- start start 2020-07-18T03:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a459e6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6a40065f20 0x7f6a40159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a459e6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 0x7f6a40145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a459e6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a401126b0 con 0x7f6a401153d0 2020-07-18T03:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a459e6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a401463f0 con 0x7f6a40065f20 2020-07-18T03:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a459e6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40146610 con 0x7f6a40066300 2020-07-18T03:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6a40065f20 0x7f6a40159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3e7fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 0x7f6a40145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6a40065f20 0x7f6a40159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:48862/0 (socket says 172.21.15.47:48862) 2020-07-18T03:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3e7fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 0x7f6a40145f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55782/0 (socket says 172.21.15.47:55782) 2020-07-18T03:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 -- 172.21.15.47:0/2404620034 learned_addr learned my addr 172.21.15.47:0/2404620034 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 -- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 msgr2=0x7f6a40145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 0x7f6a40145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 -- 172.21.15.47:0/2404620034 >> v1:172.21.15.18:6789/0 conn(0x7f6a401153d0 legacy=0x7f6a4013fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 -- 172.21.15.47:0/2404620034 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40140560 con 0x7f6a40065f20 2020-07-18T03:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3e7fc700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 0x7f6a40145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489698608 0 0) 0x7f6a401126b0 con 0x7f6a401153d0 2020-07-18T03:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a3effd700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a40065f20 0x7f6a40159550 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f6a38008000 tx=0x7f6a38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.943+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3800fa30 con 0x7f6a40065f20 2020-07-18T03:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.944+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3800fb90 con 0x7f6a40065f20 2020-07-18T03:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.944+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a40115810 con 0x7f6a40065f20 2020-07-18T03:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.944+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3800fd00 con 0x7f6a40065f20 2020-07-18T03:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.944+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a40112b10 con 0x7f6a40065f20 2020-07-18T03:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.944+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a400aec40 con 0x7f6a40065f20 2020-07-18T03:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.945+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6a38024070 con 0x7f6a40065f20 2020-07-18T03:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.945+0000 7f6a36ffd700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c04ff00 0x7f6a1c0523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.945+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6a380687f0 con 0x7f6a40065f20 2020-07-18T03:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.945+0000 7f6a3e7fc700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c04ff00 0x7f6a1c0523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.946+0000 7f6a3e7fc700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c04ff00 0x7f6a1c0523c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6a2800d000 tx=0x7f6a28007040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:27.949+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6a38040e90 con 0x7f6a40065f20 2020-07-18T03:33:28.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.156+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a40057450 con 0x7f6a40065f20 2020-07-18T03:33:28.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.157+0000 7f6a36ffd700 1 -- 172.21.15.47:0/2404620034 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f6a380444b0 con 0x7f6a40065f20 2020-07-18T03:33:28.159 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:28.159 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:28.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c04ff00 msgr2=0x7f6a1c0523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:28.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c04ff00 0x7f6a1c0523c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6a2800d000 tx=0x7f6a28007040).stop 2020-07-18T03:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a40065f20 msgr2=0x7f6a40159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a40065f20 0x7f6a40159550 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f6a38008000 tx=0x7f6a38008040).stop 2020-07-18T03:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 shutdown_connections 2020-07-18T03:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6a1c04ff00 0x7f6a1c0523c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a40065f20 0x7f6a40159550 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 --2- 172.21.15.47:0/2404620034 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6a40066300 0x7f6a40145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 >> 172.21.15.47:0/2404620034 conn(0x7f6a400b9340 msgr2=0x7f6a400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 shutdown_connections 2020-07-18T03:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:28.159+0000 7f6a459e6700 1 -- 172.21.15.47:0/2404620034 wait complete. 2020-07-18T03:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:28.171 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:29.172 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:29.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.298+0000 7fe585a94700 1 Processor -- start 2020-07-18T03:33:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.298+0000 7fe585a94700 1 -- start start 2020-07-18T03:33:29.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.299+0000 7fe585a94700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe5800ad100 0x7fe5800b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.299+0000 7fe585a94700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe580147330 con 0x7fe5801443f0 2020-07-18T03:33:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.299+0000 7fe585a94700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5800603c0 con 0x7fe580147a00 2020-07-18T03:33:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.299+0000 7fe585a94700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5800b57e0 con 0x7fe5800ad100 2020-07-18T03:33:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.299+0000 7fe57effd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe580147a00 0x7fe58005f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44278/0 (socket says 172.21.15.47:44278) 2020-07-18T03:33:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.299+0000 7fe57effd700 1 -- 172.21.15.47:0/2614838957 learned_addr learned my addr 172.21.15.47:0/2614838957 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57f7fe700 1 --2- 172.21.15.47:0/2614838957 >> v2:172.21.15.18:3300/0 conn(0x7fe5800ad100 0x7fe5800b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244791614 0 0) 0x7fe5800603c0 con 0x7fe580147a00 2020-07-18T03:33:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe56c003240 con 0x7fe580147a00 2020-07-18T03:33:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 155174433 0 0) 0x7fe56c003240 con 0x7fe580147a00 2020-07-18T03:33:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 >> v2:172.21.15.18:3300/0 conn(0x7fe5800ad100 msgr2=0x7fe5800b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:29.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 --2- 172.21.15.47:0/2614838957 >> v2:172.21.15.18:3300/0 conn(0x7fe5800ad100 0x7fe5800b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 >> v1:172.21.15.18:6789/0 conn(0x7fe5801443f0 legacy=0x7fe58005fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800b5a80 con 0x7fe580147a00 2020-07-18T03:33:29.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fe5740016d0 con 0x7fe580147a00 2020-07-18T03:33:29.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 >> v1:172.21.15.47:6789/0 conn(0x7fe580147a00 legacy=0x7fe58005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe56c003730 0x7fe56c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe56c0044e0 0x7fe56c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe56c004e50 0x7fe56c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800b57e0 con 0x7fe56c003730 2020-07-18T03:33:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c004350 con 0x7fe56c0044e0 2020-07-18T03:33:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c00de60 con 0x7fe56c004e50 2020-07-18T03:33:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57effd700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe56c0044e0 0x7fe56c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57f7fe700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe56c003730 0x7fe56c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57ffff700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe56c004e50 0x7fe56c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57effd700 1 -- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe56c004e50 msgr2=0x7fe56c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:29.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57effd700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe56c004e50 0x7fe56c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57effd700 1 -- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe56c003730 msgr2=0x7fe56c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57effd700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe56c003730 0x7fe56c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57effd700 1 -- 172.21.15.47:0/2614838957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c003240 con 0x7fe56c0044e0 2020-07-18T03:33:29.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.300+0000 7fe57ffff700 1 -- 172.21.15.47:0/2614838957 reap_dead start 2020-07-18T03:33:29.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe57effd700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe56c0044e0 0x7fe56c004960 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fe5740013e0 tx=0x7fe5800b03e0).ready entity=mon.1 client_cookie=e997f01845cb70ae server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:29.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5740070d0 con 0x7fe56c0044e0 2020-07-18T03:33:29.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574007230 con 0x7fe56c0044e0 2020-07-18T03:33:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe57e7fc700 1 -- 172.21.15.47:0/2614838957 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5740055f0 con 0x7fe56c0044e0 2020-07-18T03:33:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 -- 172.21.15.47:0/2614838957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe56c0044e0 msgr2=0x7fe56c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe56c0044e0 0x7fe56c004960 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fe5740013e0 tx=0x7fe5800b03e0).stop 2020-07-18T03:33:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 -- 172.21.15.47:0/2614838957 shutdown_connections 2020-07-18T03:33:29.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 --2- 172.21.15.47:0/2614838957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe56c0044e0 0x7fe56c004960 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 -- 172.21.15.47:0/2614838957 >> 172.21.15.47:0/2614838957 conn(0x7fe5800b9340 msgr2=0x7fe5800b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 -- 172.21.15.47:0/2614838957 shutdown_connections 2020-07-18T03:33:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.301+0000 7fe585a94700 1 -- 172.21.15.47:0/2614838957 wait complete. 2020-07-18T03:33:29.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 Processor -- start 2020-07-18T03:33:29.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 -- start start 2020-07-18T03:33:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe580154200 0x7fe58013ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.316 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5801545e0 0x7fe58013ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe5800b1f30 0x7fe580150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe580154e60 con 0x7fe5800b1f30 2020-07-18T03:33:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5800b23d0 con 0x7fe5801545e0 2020-07-18T03:33:29.317 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe585a94700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800b2670 con 0x7fe580154200 2020-07-18T03:33:29.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe57effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5801545e0 0x7fe58013ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe57effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe5801545e0 0x7fe58013ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34972/0 (socket says 172.21.15.47:34972) 2020-07-18T03:33:29.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe57effd700 1 -- 172.21.15.47:0/602264144 learned_addr learned my addr 172.21.15.47:0/602264144 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:29.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe57ffff700 1 --2- 172.21.15.47:0/602264144 >> v2:172.21.15.18:3300/0 conn(0x7fe5800b1f30 0x7fe580150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.318 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.303+0000 7fe57f7fe700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe580154200 0x7fe58013ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe57effd700 1 -- 172.21.15.47:0/602264144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe580154200 msgr2=0x7fe58013ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe57effd700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe580154200 0x7fe58013ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe57effd700 1 -- 172.21.15.47:0/602264144 >> v2:172.21.15.18:3300/0 conn(0x7fe5800b1f30 msgr2=0x7fe580150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe57effd700 1 --2- 172.21.15.47:0/602264144 >> v2:172.21.15.18:3300/0 conn(0x7fe5800b1f30 0x7fe580150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.319 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe57effd700 1 -- 172.21.15.47:0/602264144 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800b2910 con 0x7fe5801545e0 2020-07-18T03:33:29.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe57effd700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5801545e0 0x7fe58013ef10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fe5740013e0 tx=0x7fe574015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:29.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5740070d0 con 0x7fe5801545e0 2020-07-18T03:33:29.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574013850 con 0x7fe5801545e0 2020-07-18T03:33:29.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe580141d30 con 0x7fe5801545e0 2020-07-18T03:33:29.320 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe574013bb0 con 0x7fe5801545e0 2020-07-18T03:33:29.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.304+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe580150f60 con 0x7fe5801545e0 2020-07-18T03:33:29.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.305+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5800b2aa0 con 0x7fe5801545e0 2020-07-18T03:33:29.321 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.305+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe574010070 con 0x7fe5801545e0 2020-07-18T03:33:29.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.306+0000 7fe55f7fe700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe558050000 0x7fe5580524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:29.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.306+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe574019030 con 0x7fe5801545e0 2020-07-18T03:33:29.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.306+0000 7fe57f7fe700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe558050000 0x7fe5580524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:29.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.306+0000 7fe57f7fe700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe558050000 0x7fe5580524c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe570008000 tx=0x7fe570008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:29.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.309+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe574041740 con 0x7fe5801545e0 2020-07-18T03:33:29.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.526+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe580057450 con 0x7fe5801545e0 2020-07-18T03:33:29.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.527+0000 7fe55f7fe700 1 -- 172.21.15.47:0/602264144 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fe574044d60 con 0x7fe5801545e0 2020-07-18T03:33:29.528 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:29.529 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe558050000 msgr2=0x7fe5580524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe558050000 0x7fe5580524c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe570008000 tx=0x7fe570008040).stop 2020-07-18T03:33:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5801545e0 msgr2=0x7fe58013ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5801545e0 0x7fe58013ef10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fe5740013e0 tx=0x7fe574015000).stop 2020-07-18T03:33:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 shutdown_connections 2020-07-18T03:33:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe558050000 0x7fe5580524c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 --2- 172.21.15.47:0/602264144 >> v2:172.21.15.18:3300/0 conn(0x7fe5800b1f30 0x7fe580150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe580154200 0x7fe58013ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 --2- 172.21.15.47:0/602264144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5801545e0 0x7fe58013ef10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.528+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 >> 172.21.15.47:0/602264144 conn(0x7fe5800b9340 msgr2=0x7fe5800b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.529+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 shutdown_connections 2020-07-18T03:33:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:29.529+0000 7fe585a94700 1 -- 172.21.15.47:0/602264144 wait complete. 2020-07-18T03:33:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:29.540 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:30.541 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.669+0000 7f665236f700 1 Processor -- start 2020-07-18T03:33:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.669+0000 7f665236f700 1 -- start start 2020-07-18T03:33:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665236f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f664c147e40 0x7f664c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665236f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c147330 con 0x7f664c0ad100 2020-07-18T03:33:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665236f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0b57e0 con 0x7f664c147a00 2020-07-18T03:33:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665236f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0ae530 con 0x7f664c147e40 2020-07-18T03:33:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665090c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f664c147e40 0x7f664c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f664b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f664c147a00 0x7f664c1433e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44300/0 (socket says 172.21.15.47:44300) 2020-07-18T03:33:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f664b7fe700 1 -- 172.21.15.47:0/3065734541 learned_addr learned my addr 172.21.15.47:0/3065734541 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665090c700 1 -- 172.21.15.47:0/3065734541 >> v1:172.21.15.47:6789/0 conn(0x7f664c147a00 legacy=0x7f664c1433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665090c700 1 -- 172.21.15.47:0/3065734541 >> v1:172.21.15.18:6789/0 conn(0x7f664c0ad100 legacy=0x7f664c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f664affd700 1 -- 172.21.15.47:0/3065734541 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562571417 0 0) 0x7f664c0b57e0 con 0x7f664c147a00 2020-07-18T03:33:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665090c700 1 -- 172.21.15.47:0/3065734541 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0b5a80 con 0x7f664c147e40 2020-07-18T03:33:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f665090c700 1 --2- 172.21.15.47:0/3065734541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664c147e40 0x7f664c143af0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f66440032b0 tx=0x7f66440091d0).ready entity=mon.1 client_cookie=a598d8a5150dc5bf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.670+0000 7f664affd700 1 -- 172.21.15.47:0/3065734541 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400ba50 con 0x7f664c147e40 2020-07-18T03:33:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f664affd700 1 -- 172.21.15.47:0/3065734541 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400bbb0 con 0x7f664c147e40 2020-07-18T03:33:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f664affd700 1 -- 172.21.15.47:0/3065734541 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400bd20 con 0x7f664c147e40 2020-07-18T03:33:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 -- 172.21.15.47:0/3065734541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664c147e40 msgr2=0x7f664c143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 --2- 172.21.15.47:0/3065734541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664c147e40 0x7f664c143af0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f66440032b0 tx=0x7f66440091d0).stop 2020-07-18T03:33:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 -- 172.21.15.47:0/3065734541 shutdown_connections 2020-07-18T03:33:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 --2- 172.21.15.47:0/3065734541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664c147e40 0x7f664c143af0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 -- 172.21.15.47:0/3065734541 >> 172.21.15.47:0/3065734541 conn(0x7f664c0b9340 msgr2=0x7f664c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 -- 172.21.15.47:0/3065734541 shutdown_connections 2020-07-18T03:33:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.671+0000 7f665236f700 1 -- 172.21.15.47:0/3065734541 wait complete. 2020-07-18T03:33:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.672+0000 7f665236f700 1 Processor -- start 2020-07-18T03:33:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665236f700 1 -- start start 2020-07-18T03:33:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665236f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f664c142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665236f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0b1540 con 0x7f664c0ad100 2020-07-18T03:33:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665236f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0b2460 con 0x7f664c0644c0 2020-07-18T03:33:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665236f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b1770 con 0x7f664c147a00 2020-07-18T03:33:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665090c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f664c0644c0 0x7f664c065b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44308/0 (socket says 172.21.15.47:44308) 2020-07-18T03:33:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.673+0000 7f665090c700 1 -- 172.21.15.47:0/178488909 learned_addr learned my addr 172.21.15.47:0/178488909 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664b7fe700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f664c142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108735585 0 0) 0x7f664c0b2460 con 0x7f664c0644c0 2020-07-18T03:33:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6624003240 con 0x7f664c0644c0 2020-07-18T03:33:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 76836922 0 0) 0x7f6624003240 con 0x7f664c0644c0 2020-07-18T03:33:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f664c0b2460 con 0x7f664c0644c0 2020-07-18T03:33:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633819604 0 0) 0x7f664c0b1540 con 0x7f664c0ad100 2020-07-18T03:33:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6624003240 con 0x7f664c0ad100 2020-07-18T03:33:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f664400ba50 con 0x7f664c0644c0 2020-07-18T03:33:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 msgr2=0x7f664c142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f664c142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 >> v1:172.21.15.47:6789/0 conn(0x7f664c0644c0 legacy=0x7f664c065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 >> v1:172.21.15.18:6789/0 conn(0x7f664c0ad100 legacy=0x7f664c12b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624003990 0x7f6624003d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f66240044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66240049d0 0x7f6624004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b1770 con 0x7f66240049d0 2020-07-18T03:33:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624005340 con 0x7f6624003990 2020-07-18T03:33:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662400fb40 con 0x7f664c147a00 2020-07-18T03:33:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624003990 0x7f6624003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664b7fe700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f66240044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f665090c700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66240049d0 0x7f6624004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664bfff700 1 -- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 msgr2=0x7f66240044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f66240044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664bfff700 1 -- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66240049d0 msgr2=0x7f6624004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66240049d0 0x7f6624004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664bfff700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0b26c0 con 0x7f6624003990 2020-07-18T03:33:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c0b2b60 con 0x7f6624003990 2020-07-18T03:33:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.674+0000 7f664b7fe700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f664c147a00 0x7f66240044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.675+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c142cb0 con 0x7f6624003990 2020-07-18T03:33:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.675+0000 7f664bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624003990 0x7f6624003d90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f664c0b69e0 tx=0x7f663c006e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.675+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c00d570 con 0x7f6624003990 2020-07-18T03:33:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.675+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c00d6d0 con 0x7f6624003990 2020-07-18T03:33:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.675+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c00da30 con 0x7f6624003990 2020-07-18T03:33:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.675+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c057450 con 0x7f6624003990 2020-07-18T03:33:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.676+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f663c00f030 con 0x7f6624003990 2020-07-18T03:33:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.676+0000 7f662bfff700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f662405c890 0x7f662405ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.676+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f663c067900 con 0x7f6624003990 2020-07-18T03:33:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.676+0000 7f664b7fe700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f662405c890 0x7f662405ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.676+0000 7f664b7fe700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f662405c890 0x7f662405ed50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f6640007000 tx=0x7f6640009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:30.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.680+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f663c040040 con 0x7f6624003990 2020-07-18T03:33:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.889+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f664c0aef30 con 0x7f6624003990 2020-07-18T03:33:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.889+0000 7f662bfff700 1 -- 172.21.15.47:0/178488909 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f663c043660 con 0x7f6624003990 2020-07-18T03:33:30.891 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:30.892 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f662405c890 msgr2=0x7f662405ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f662405c890 0x7f662405ed50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f6640007000 tx=0x7f6640009040).stop 2020-07-18T03:33:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624003990 msgr2=0x7f6624003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 --2- 172.21.15.47:0/178488909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624003990 0x7f6624003d90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f664c0b69e0 tx=0x7f663c006e20).stop 2020-07-18T03:33:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665090c700 1 -- 172.21.15.47:0/178488909 reap_dead start 2020-07-18T03:33:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665090c700 1 -- 172.21.15.47:0/178488909 reap_dead start 2020-07-18T03:33:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 shutdown_connections 2020-07-18T03:33:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 >> 172.21.15.47:0/178488909 conn(0x7f664c0b9340 msgr2=0x7f664c1469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 shutdown_connections 2020-07-18T03:33:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:30.891+0000 7f665236f700 1 -- 172.21.15.47:0/178488909 wait complete. 2020-07-18T03:33:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:30.902 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:31.904 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:32.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.030+0000 7faa13296700 1 Processor -- start 2020-07-18T03:33:32.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 -- start start 2020-07-18T03:33:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faa0c147150 0x7faa0c1445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa0c063d20 0x7faa0c14a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faa0c0641d0 0x7faa0c14ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c147530 con 0x7faa0c147150 2020-07-18T03:33:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c060ff0 con 0x7faa0c063d20 2020-07-18T03:33:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.031+0000 7faa13296700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c147740 con 0x7faa0c0641d0 2020-07-18T03:33:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa0c063d20 0x7faa0c14a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa0c063d20 0x7faa0c14a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35002/0 (socket says 172.21.15.47:35002) 2020-07-18T03:33:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 -- 172.21.15.47:0/708396404 learned_addr learned my addr 172.21.15.47:0/708396404 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:32.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa11833700 1 --2- 172.21.15.47:0/708396404 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faa0c0641d0 0x7faa0c14ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:32.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa11032700 1 --2- 172.21.15.47:0/708396404 >> v2:172.21.15.18:3300/0 conn(0x7faa0c147150 0x7faa0c1445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:32.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 -- 172.21.15.47:0/708396404 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faa0c0641d0 msgr2=0x7faa0c14ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:32.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 --2- 172.21.15.47:0/708396404 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faa0c0641d0 0x7faa0c14ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 -- 172.21.15.47:0/708396404 >> v2:172.21.15.18:3300/0 conn(0x7faa0c147150 msgr2=0x7faa0c1445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:32.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 --2- 172.21.15.47:0/708396404 >> v2:172.21.15.18:3300/0 conn(0x7faa0c147150 0x7faa0c1445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 -- 172.21.15.47:0/708396404 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c1479e0 con 0x7faa0c063d20 2020-07-18T03:33:32.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa10831700 1 --2- 172.21.15.47:0/708396404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c063d20 0x7faa0c14a720 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7faa08002f30 tx=0x7faa08006e90).ready entity=mon.1 client_cookie=55eb01ff97f5b115 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:32.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7fa9fffff700 1 -- 172.21.15.47:0/708396404 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa08009920 con 0x7faa0c063d20 2020-07-18T03:33:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7fa9fffff700 1 -- 172.21.15.47:0/708396404 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa08009a80 con 0x7faa0c063d20 2020-07-18T03:33:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7fa9fffff700 1 -- 172.21.15.47:0/708396404 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa08009de0 con 0x7faa0c063d20 2020-07-18T03:33:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 -- 172.21.15.47:0/708396404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c063d20 msgr2=0x7faa0c14a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 --2- 172.21.15.47:0/708396404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c063d20 0x7faa0c14a720 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7faa08002f30 tx=0x7faa08006e90).stop 2020-07-18T03:33:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 -- 172.21.15.47:0/708396404 shutdown_connections 2020-07-18T03:33:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 --2- 172.21.15.47:0/708396404 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7faa0c0641d0 0x7faa0c14ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 --2- 172.21.15.47:0/708396404 >> v2:172.21.15.18:3300/0 conn(0x7faa0c147150 0x7faa0c1445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 --2- 172.21.15.47:0/708396404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c063d20 0x7faa0c14a720 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.032+0000 7faa13296700 1 -- 172.21.15.47:0/708396404 >> 172.21.15.47:0/708396404 conn(0x7faa0c05f5a0 msgr2=0x7faa0c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:32.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.033+0000 7faa13296700 1 -- 172.21.15.47:0/708396404 shutdown_connections 2020-07-18T03:33:32.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.033+0000 7faa13296700 1 -- 172.21.15.47:0/708396404 wait complete. 2020-07-18T03:33:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.034+0000 7faa13296700 1 Processor -- start 2020-07-18T03:33:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.034+0000 7faa13296700 1 -- start start 2020-07-18T03:33:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa13296700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa0c0641d0 0x7faa0c146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa13296700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faa0c0b2af0 0x7faa0c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa13296700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0c146530 con 0x7faa0c147150 2020-07-18T03:33:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa13296700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c1467c0 con 0x7faa0c0b2af0 2020-07-18T03:33:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa13296700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c1446d0 con 0x7faa0c0641d0 2020-07-18T03:33:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11032700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa0c0641d0 0x7faa0c146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa10831700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faa0c147150 0x7faa0c141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44324/0 (socket says 172.21.15.47:44324) 2020-07-18T03:33:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa10831700 1 -- 172.21.15.47:0/3907146697 learned_addr learned my addr 172.21.15.47:0/3907146697 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11833700 1 --2- 172.21.15.47:0/3907146697 >> v2:172.21.15.18:3300/0 conn(0x7faa0c0b2af0 0x7faa0c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11032700 1 -- 172.21.15.47:0/3907146697 >> v2:172.21.15.18:3300/0 conn(0x7faa0c0b2af0 msgr2=0x7faa0c0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11032700 1 --2- 172.21.15.47:0/3907146697 >> v2:172.21.15.18:3300/0 conn(0x7faa0c0b2af0 0x7faa0c0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11032700 1 -- 172.21.15.47:0/3907146697 >> v1:172.21.15.47:6789/0 conn(0x7faa0c147150 legacy=0x7faa0c141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11032700 1 -- 172.21.15.47:0/3907146697 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c0b4c30 con 0x7faa0c0641d0 2020-07-18T03:33:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80592538 0 0) 0x7faa0c146530 con 0x7faa0c147150 2020-07-18T03:33:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11833700 1 --2- 172.21.15.47:0/3907146697 >> v2:172.21.15.18:3300/0 conn(0x7faa0c0b2af0 0x7faa0c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:33:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.035+0000 7faa11032700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c0641d0 0x7faa0c146040 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7faa0000a000 tx=0x7faa00008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.036+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa00011780 con 0x7faa0c0641d0 2020-07-18T03:33:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.036+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa000118e0 con 0x7faa0c0641d0 2020-07-18T03:33:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.036+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c13fe40 con 0x7faa0c0641d0 2020-07-18T03:33:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.036+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa00011c40 con 0x7faa0c0641d0 2020-07-18T03:33:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.036+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c140390 con 0x7faa0c0641d0 2020-07-18T03:33:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.037+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c0bc3c0 con 0x7faa0c0641d0 2020-07-18T03:33:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.037+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7faa0001c460 con 0x7faa0c0641d0 2020-07-18T03:33:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.037+0000 7fa9fd7fa700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa9e404ff50 0x7fa9e4052410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.037+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7faa00068190 con 0x7faa0c0641d0 2020-07-18T03:33:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.037+0000 7faa10831700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa9e404ff50 0x7fa9e4052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.038+0000 7faa10831700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa9e404ff50 0x7fa9e4052410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7faa080094e0 tx=0x7faa08015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.041+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7faa000407f0 con 0x7faa0c0641d0 2020-07-18T03:33:32.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.250+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa0c057450 con 0x7faa0c0641d0 2020-07-18T03:33:32.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.250+0000 7fa9fd7fa700 1 -- 172.21.15.47:0/3907146697 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7faa00043e10 con 0x7faa0c0641d0 2020-07-18T03:33:32.252 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:32.252 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:32.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa9e404ff50 msgr2=0x7fa9e4052410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:32.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa9e404ff50 0x7fa9e4052410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7faa080094e0 tx=0x7faa08015000).stop 2020-07-18T03:33:32.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c0641d0 msgr2=0x7faa0c146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:32.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c0641d0 0x7faa0c146040 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7faa0000a000 tx=0x7faa00008040).stop 2020-07-18T03:33:32.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 shutdown_connections 2020-07-18T03:33:32.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 --2- 172.21.15.47:0/3907146697 >> v2:172.21.15.18:3300/0 conn(0x7faa0c0b2af0 0x7faa0c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa0c0641d0 0x7faa0c146040 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 --2- 172.21.15.47:0/3907146697 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa9e404ff50 0x7fa9e4052410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:32.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 >> 172.21.15.47:0/3907146697 conn(0x7faa0c05f5a0 msgr2=0x7faa0c0bb570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:32.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 shutdown_connections 2020-07-18T03:33:32.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:32.252+0000 7faa13296700 1 -- 172.21.15.47:0/3907146697 wait complete. 2020-07-18T03:33:32.256 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:32.263 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:33.265 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:33.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.393+0000 7f15de48e700 1 Processor -- start 2020-07-18T03:33:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.393+0000 7f15de48e700 1 -- start start 2020-07-18T03:33:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15de48e700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15d80b2e10 0x7f15d80b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15de48e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f15d80b19e0 0x7f15d81454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15de48e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d80afbd0 con 0x7f15d80b1560 2020-07-18T03:33:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15de48e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d8144e30 con 0x7f15d80b19e0 2020-07-18T03:33:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15de48e700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81491c0 con 0x7f15d80b2e10 2020-07-18T03:33:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15dca2b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f15d80b19e0 0x7f15d81454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:33.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15d80b2e10 0x7f15d80b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:33.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15d77fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f15d80b1560 0x7f15d80b2130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34870/0 (socket says 172.21.15.47:34870) 2020-07-18T03:33:33.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15d77fe700 1 -- 172.21.15.47:0/313584698 learned_addr learned my addr 172.21.15.47:0/313584698 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15dca2b700 1 -- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15d80b2e10 msgr2=0x7f15d80b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15dca2b700 1 --2- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15d80b2e10 0x7f15d80b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15dca2b700 1 -- 172.21.15.47:0/313584698 >> v1:172.21.15.18:6789/0 conn(0x7f15d80b1560 legacy=0x7f15d80b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15d6ffd700 1 -- 172.21.15.47:0/313584698 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852162464 0 0) 0x7f15cc0010d0 con 0x7f15d80b1560 2020-07-18T03:33:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.394+0000 7f15dca2b700 1 -- 172.21.15.47:0/313584698 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8145050 con 0x7f15d80b19e0 2020-07-18T03:33:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15dca2b700 1 --2- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d80b19e0 0x7f15d81454f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f15d00032b0 tx=0x7f15d00091d0).ready entity=mon.0 client_cookie=c6546508841be9e2 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15d6ffd700 1 -- 172.21.15.47:0/313584698 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15d000b9a0 con 0x7f15d80b19e0 2020-07-18T03:33:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15d6ffd700 1 -- 172.21.15.47:0/313584698 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000bb00 con 0x7f15d80b19e0 2020-07-18T03:33:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15d6ffd700 1 -- 172.21.15.47:0/313584698 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15d000bc70 con 0x7f15d80b19e0 2020-07-18T03:33:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 -- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d80b19e0 msgr2=0x7f15d81454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 --2- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d80b19e0 0x7f15d81454f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f15d00032b0 tx=0x7f15d00091d0).stop 2020-07-18T03:33:33.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 -- 172.21.15.47:0/313584698 shutdown_connections 2020-07-18T03:33:33.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 --2- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d80b19e0 0x7f15d81454f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:33.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 --2- 172.21.15.47:0/313584698 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f15d80b2e10 0x7f15d80b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:33.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 -- 172.21.15.47:0/313584698 >> 172.21.15.47:0/313584698 conn(0x7f15d80b9340 msgr2=0x7f15d80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:33.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 -- 172.21.15.47:0/313584698 shutdown_connections 2020-07-18T03:33:33.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.395+0000 7f15de48e700 1 -- 172.21.15.47:0/313584698 wait complete. 2020-07-18T03:33:33.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.396+0000 7f15de48e700 1 Processor -- start 2020-07-18T03:33:33.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15de48e700 1 -- start start 2020-07-18T03:33:33.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15de48e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15d80b2e10 0x7f15d8141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:33.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15de48e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d81419a0 con 0x7f15d813fb70 2020-07-18T03:33:33.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15de48e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d8141c40 con 0x7f15d80b1560 2020-07-18T03:33:33.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15de48e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d8142380 con 0x7f15d80b2e10 2020-07-18T03:33:33.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15d77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15d80b2e10 0x7f15d8141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:33.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15d7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f15d80b1560 0x7f15d812de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44338/0 (socket says 172.21.15.47:44338) 2020-07-18T03:33:33.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.397+0000 7f15d7fff700 1 -- 172.21.15.47:0/2602039313 learned_addr learned my addr 172.21.15.47:0/2602039313 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:33.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15d77fe700 1 -- 172.21.15.47:0/2602039313 >> v1:172.21.15.47:6789/0 conn(0x7f15d80b1560 legacy=0x7f15d812de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:33.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697327605 0 0) 0x7f15d8141c40 con 0x7f15d80b1560 2020-07-18T03:33:33.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15d77fe700 1 -- 172.21.15.47:0/2602039313 >> v1:172.21.15.18:6789/0 conn(0x7f15d813fb70 legacy=0x7f15d814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:33.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15d77fe700 1 -- 172.21.15.47:0/2602039313 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8141ee0 con 0x7f15d80b2e10 2020-07-18T03:33:33.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15d77fe700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d80b2e10 0x7f15d8141290 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f15cc000b80 tx=0x7f15cc007820).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:33.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15cc0072e0 con 0x7f15d80b2e10 2020-07-18T03:33:33.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc007440 con 0x7f15d80b2e10 2020-07-18T03:33:33.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d814e010 con 0x7f15d80b2e10 2020-07-18T03:33:33.414 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15cc0057e0 con 0x7f15d80b2e10 2020-07-18T03:33:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.398+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d815fa90 con 0x7f15d80b2e10 2020-07-18T03:33:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.399+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8142070 con 0x7f15d80b2e10 2020-07-18T03:33:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.399+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f15cc01a060 con 0x7f15d80b2e10 2020-07-18T03:33:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.400+0000 7f15b7fff700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f15b004fdf0 0x7f15b00522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.400+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f15cc0673c0 con 0x7f15d80b2e10 2020-07-18T03:33:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.400+0000 7f15d7fff700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f15b004fdf0 0x7f15b00522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.400+0000 7f15d7fff700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f15b004fdf0 0x7f15b00522b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f15c8002f30 tx=0x7f15c8006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.403+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f15cc03fa80 con 0x7f15d80b2e10 2020-07-18T03:33:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.613+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d80afbd0 con 0x7f15d80b2e10 2020-07-18T03:33:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.614+0000 7f15b7fff700 1 -- 172.21.15.47:0/2602039313 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f15d80afbd0 con 0x7f15d80b2e10 2020-07-18T03:33:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:33.616 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f15b004fdf0 msgr2=0x7f15b00522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f15b004fdf0 0x7f15b00522b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f15c8002f30 tx=0x7f15c8006dd0).stop 2020-07-18T03:33:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d80b2e10 msgr2=0x7f15d8141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d80b2e10 0x7f15d8141290 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f15cc000b80 tx=0x7f15cc007820).stop 2020-07-18T03:33:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 shutdown_connections 2020-07-18T03:33:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f15b004fdf0 0x7f15b00522b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 --2- 172.21.15.47:0/2602039313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d80b2e10 0x7f15d8141290 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 >> 172.21.15.47:0/2602039313 conn(0x7f15d80b9340 msgr2=0x7f15d80b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 shutdown_connections 2020-07-18T03:33:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:33.615+0000 7f15de48e700 1 -- 172.21.15.47:0/2602039313 wait complete. 2020-07-18T03:33:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:33.627 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:34.628 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:34.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.756+0000 7f9ca8bf9700 1 Processor -- start 2020-07-18T03:33:34.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.756+0000 7f9ca8bf9700 1 -- start start 2020-07-18T03:33:34.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca8bf9700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9ca40b5370 0x7f9ca405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:34.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca8bf9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca40ad9e0 0x7f9ca41428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:34.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca8bf9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca40b3ac0 con 0x7f9ca40b41c0 2020-07-18T03:33:34.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca8bf9700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40b5a10 con 0x7f9ca40b5370 2020-07-18T03:33:34.760 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca8bf9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40b0040 con 0x7f9ca40ad9e0 2020-07-18T03:33:34.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca40ad9e0 0x7f9ca41428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:34.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca1d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ca40b41c0 0x7f9ca40ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44348/0 (socket says 172.21.15.47:44348) 2020-07-18T03:33:34.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca1d9b700 1 -- 172.21.15.47:0/3751769382 learned_addr learned my addr 172.21.15.47:0/3751769382 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:34.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca259c700 1 --2- 172.21.15.47:0/3751769382 >> v2:172.21.15.18:3300/0 conn(0x7f9ca40b5370 0x7f9ca405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:34.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca2d9d700 1 -- 172.21.15.47:0/3751769382 >> v2:172.21.15.18:3300/0 conn(0x7f9ca40b5370 msgr2=0x7f9ca405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:34.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca2d9d700 1 --2- 172.21.15.47:0/3751769382 >> v2:172.21.15.18:3300/0 conn(0x7f9ca40b5370 0x7f9ca405fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca2d9d700 1 -- 172.21.15.47:0/3751769382 >> v1:172.21.15.47:6789/0 conn(0x7f9ca40b41c0 legacy=0x7f9ca40ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:34.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca159a700 1 -- 172.21.15.47:0/3751769382 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352369268 0 0) 0x7f9ca40b3ac0 con 0x7f9ca40b41c0 2020-07-18T03:33:34.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca2d9d700 1 -- 172.21.15.47:0/3751769382 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40b3d20 con 0x7f9ca40ad9e0 2020-07-18T03:33:34.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.757+0000 7f9ca2d9d700 1 --2- 172.21.15.47:0/3751769382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40ad9e0 0x7f9ca41428f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f9ca40afb30 tx=0x7f9c9c0091d0).ready entity=mon.1 client_cookie=2495a19324ef9651 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca159a700 1 -- 172.21.15.47:0/3751769382 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9c00b960 con 0x7f9ca40ad9e0 2020-07-18T03:33:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca159a700 1 -- 172.21.15.47:0/3751769382 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00bac0 con 0x7f9ca40ad9e0 2020-07-18T03:33:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca159a700 1 -- 172.21.15.47:0/3751769382 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9c00bc30 con 0x7f9ca40ad9e0 2020-07-18T03:33:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/3751769382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40ad9e0 msgr2=0x7f9ca41428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/3751769382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40ad9e0 0x7f9ca41428f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f9ca40afb30 tx=0x7f9c9c0091d0).stop 2020-07-18T03:33:34.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/3751769382 shutdown_connections 2020-07-18T03:33:34.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/3751769382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40ad9e0 0x7f9ca41428f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/3751769382 >> v2:172.21.15.18:3300/0 conn(0x7f9ca40b5370 0x7f9ca405fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/3751769382 >> 172.21.15.47:0/3751769382 conn(0x7f9ca40b9340 msgr2=0x7f9ca40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:34.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/3751769382 shutdown_connections 2020-07-18T03:33:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.758+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/3751769382 wait complete. 2020-07-18T03:33:34.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.759+0000 7f9ca8bf9700 1 Processor -- start 2020-07-18T03:33:34.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.759+0000 7f9ca8bf9700 1 -- start start 2020-07-18T03:33:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca8bf9700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9ca40b41c0 0x7f9ca4066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca8bf9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca40b5370 0x7f9ca4062350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca8bf9700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca413e900 con 0x7f9ca413e1c0 2020-07-18T03:33:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca8bf9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40659e0 con 0x7f9ca40b5370 2020-07-18T03:33:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca8bf9700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca413eba0 con 0x7f9ca40b41c0 2020-07-18T03:33:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca40b5370 0x7f9ca4062350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca40b5370 0x7f9ca4062350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35036/0 (socket says 172.21.15.47:35036) 2020-07-18T03:33:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca1d9b700 1 -- 172.21.15.47:0/378730329 learned_addr learned my addr 172.21.15.47:0/378730329 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca259c700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9ca40b41c0 0x7f9ca4066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca1d9b700 1 -- 172.21.15.47:0/378730329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9ca40b41c0 msgr2=0x7f9ca4066170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca1d9b700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9ca40b41c0 0x7f9ca4066170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.760+0000 7f9ca1d9b700 1 -- 172.21.15.47:0/378730329 >> v1:172.21.15.18:6789/0 conn(0x7f9ca413e1c0 legacy=0x7f9ca413c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9ca1d9b700 1 -- 172.21.15.47:0/378730329 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4066660 con 0x7f9ca40b5370 2020-07-18T03:33:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9ca1d9b700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40b5370 0x7f9ca4062350 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f9ca4064790 tx=0x7f9c98007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9800c770 con 0x7f9ca40b5370 2020-07-18T03:33:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9800c8d0 con 0x7f9ca40b5370 2020-07-18T03:33:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9800cc30 con 0x7f9ca40b5370 2020-07-18T03:33:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca4062840 con 0x7f9ca40b5370 2020-07-18T03:33:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.761+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca413cf20 con 0x7f9ca40b5370 2020-07-18T03:33:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.762+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca40aed70 con 0x7f9ca40b5370 2020-07-18T03:33:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.762+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9c980055f0 con 0x7f9ca40b5370 2020-07-18T03:33:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.762+0000 7f9c927fc700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9c78050000 0x7f9c780524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.762+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9c9806a420 con 0x7f9ca40b5370 2020-07-18T03:33:34.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.763+0000 7f9ca259c700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9c78050000 0x7f9c780524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:34.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.763+0000 7f9ca259c700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9c78050000 0x7f9c780524c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f9c9400b000 tx=0x7f9c94008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:34.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.766+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9c98042b00 con 0x7f9ca40b5370 2020-07-18T03:33:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.973+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ca413d1f0 con 0x7f9ca40b5370 2020-07-18T03:33:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.973+0000 7f9c927fc700 1 -- 172.21.15.47:0/378730329 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f9c98046120 con 0x7f9ca40b5370 2020-07-18T03:33:34.975 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:34.975 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:34.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9c78050000 msgr2=0x7f9c780524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:34.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9c78050000 0x7f9c780524c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f9c9400b000 tx=0x7f9c94008040).stop 2020-07-18T03:33:34.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40b5370 msgr2=0x7f9ca4062350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:34.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40b5370 0x7f9ca4062350 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f9ca4064790 tx=0x7f9c98007a90).stop 2020-07-18T03:33:34.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 shutdown_connections 2020-07-18T03:33:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9c78050000 0x7f9c780524c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9ca40b41c0 0x7f9ca4066170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 --2- 172.21.15.47:0/378730329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40b5370 0x7f9ca4062350 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 >> 172.21.15.47:0/378730329 conn(0x7f9ca40b9340 msgr2=0x7f9ca40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 shutdown_connections 2020-07-18T03:33:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:34.975+0000 7f9ca8bf9700 1 -- 172.21.15.47:0/378730329 wait complete. 2020-07-18T03:33:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:34.986 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:35.988 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:36.115 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.114+0000 7fe8ad647700 1 Processor -- start 2020-07-18T03:33:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.114+0000 7fe8ad647700 1 -- start start 2020-07-18T03:33:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8ad647700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8a80ad100 0x7fe8a80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8ad647700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe8a8147a00 0x7fe8a8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8ad647700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a8147330 con 0x7fe8a805f5a0 2020-07-18T03:33:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8ad647700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a80ae530 con 0x7fe8a8147a00 2020-07-18T03:33:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8ad647700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a80b57e0 con 0x7fe8a80ad100 2020-07-18T03:33:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8a6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8a80ad100 0x7fe8a80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8a77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe8a805f5a0 0x7fe8a805f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44362/0 (socket says 172.21.15.47:44362) 2020-07-18T03:33:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8a77fe700 1 -- 172.21.15.47:0/4251940335 learned_addr learned my addr 172.21.15.47:0/4251940335 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8a67fc700 1 --2- 172.21.15.47:0/4251940335 >> v2:172.21.15.18:3300/0 conn(0x7fe8a8147a00 0x7fe8a8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.115+0000 7fe8a6ffd700 1 -- 172.21.15.47:0/4251940335 >> v2:172.21.15.18:3300/0 conn(0x7fe8a8147a00 msgr2=0x7fe8a8147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a6ffd700 1 --2- 172.21.15.47:0/4251940335 >> v2:172.21.15.18:3300/0 conn(0x7fe8a8147a00 0x7fe8a8147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a6ffd700 1 -- 172.21.15.47:0/4251940335 >> v1:172.21.15.47:6789/0 conn(0x7fe8a805f5a0 legacy=0x7fe8a805f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a5ffb700 1 -- 172.21.15.47:0/4251940335 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883132105 0 0) 0x7fe8a8147330 con 0x7fe8a805f5a0 2020-07-18T03:33:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a6ffd700 1 -- 172.21.15.47:0/4251940335 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80b5a80 con 0x7fe8a80ad100 2020-07-18T03:33:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a6ffd700 1 --2- 172.21.15.47:0/4251940335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8a80ad100 0x7fe8a80b4c10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fe89c002f30 tx=0x7fe89c006dd0).ready entity=mon.1 client_cookie=66bb5d2792d8e96 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a5ffb700 1 -- 172.21.15.47:0/4251940335 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c0097b0 con 0x7fe8a80ad100 2020-07-18T03:33:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a5ffb700 1 -- 172.21.15.47:0/4251940335 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c009910 con 0x7fe8a80ad100 2020-07-18T03:33:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8a5ffb700 1 -- 172.21.15.47:0/4251940335 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c009c70 con 0x7fe8a80ad100 2020-07-18T03:33:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 -- 172.21.15.47:0/4251940335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8a80ad100 msgr2=0x7fe8a80b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 --2- 172.21.15.47:0/4251940335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8a80ad100 0x7fe8a80b4c10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fe89c002f30 tx=0x7fe89c006dd0).stop 2020-07-18T03:33:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 -- 172.21.15.47:0/4251940335 shutdown_connections 2020-07-18T03:33:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 --2- 172.21.15.47:0/4251940335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8a80ad100 0x7fe8a80b4c10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 --2- 172.21.15.47:0/4251940335 >> v2:172.21.15.18:3300/0 conn(0x7fe8a8147a00 0x7fe8a8147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 -- 172.21.15.47:0/4251940335 >> 172.21.15.47:0/4251940335 conn(0x7fe8a80b9340 msgr2=0x7fe8a80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 -- 172.21.15.47:0/4251940335 shutdown_connections 2020-07-18T03:33:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.116+0000 7fe8ad647700 1 -- 172.21.15.47:0/4251940335 wait complete. 2020-07-18T03:33:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.118+0000 7fe8ad647700 1 Processor -- start 2020-07-18T03:33:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.118+0000 7fe8ad647700 1 -- start start 2020-07-18T03:33:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.118+0000 7fe8ad647700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe8a805f5a0 0x7fe8a81423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.118+0000 7fe8ad647700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a8062d10 con 0x7fe8a8063cd0 2020-07-18T03:33:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.118+0000 7fe8ad647700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a8062fb0 con 0x7fe8a8147a00 2020-07-18T03:33:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.118+0000 7fe8ad647700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a8066a30 con 0x7fe8a805f5a0 2020-07-18T03:33:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a67fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe8a8147a00 0x7fe8a8065a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44368/0 (socket says 172.21.15.47:44368) 2020-07-18T03:33:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a67fc700 1 -- 172.21.15.47:0/3253258526 learned_addr learned my addr 172.21.15.47:0/3253258526 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a6ffd700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe8a805f5a0 0x7fe8a81423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351611269 0 0) 0x7fe8a8062fb0 con 0x7fe8a8147a00 2020-07-18T03:33:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe884003240 con 0x7fe8a8147a00 2020-07-18T03:33:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094070458 0 0) 0x7fe8a8062d10 con 0x7fe8a8063cd0 2020-07-18T03:33:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a8062fb0 con 0x7fe8a8063cd0 2020-07-18T03:33:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3165450164 0 0) 0x7fe884003240 con 0x7fe8a8147a00 2020-07-18T03:33:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a6ffd700 1 -- 172.21.15.47:0/3253258526 >> v1:172.21.15.47:6789/0 conn(0x7fe8a8147a00 legacy=0x7fe8a8065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a6ffd700 1 -- 172.21.15.47:0/3253258526 >> v1:172.21.15.18:6789/0 conn(0x7fe8a8063cd0 legacy=0x7fe8a8066320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a6ffd700 1 -- 172.21.15.47:0/3253258526 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8063250 con 0x7fe8a805f5a0 2020-07-18T03:33:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8a6ffd700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe8a805f5a0 0x7fe8a81423c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe898007000 tx=0x7fe89c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c021070 con 0x7fe8a805f5a0 2020-07-18T03:33:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c01c5c0 con 0x7fe8a805f5a0 2020-07-18T03:33:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.119+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a80643f0 con 0x7fe8a805f5a0 2020-07-18T03:33:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.120+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89800b7f0 con 0x7fe8a805f5a0 2020-07-18T03:33:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.120+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a80b2170 con 0x7fe8a805f5a0 2020-07-18T03:33:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.120+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a80633e0 con 0x7fe8a805f5a0 2020-07-18T03:33:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.121+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe89c00f070 con 0x7fe8a805f5a0 2020-07-18T03:33:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.121+0000 7fe88effd700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe884050410 0x7fe8840528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.121+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe89c068d30 con 0x7fe8a805f5a0 2020-07-18T03:33:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.121+0000 7fe8a67fc700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe884050410 0x7fe8840528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.122+0000 7fe8a67fc700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe884050410 0x7fe8840528d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fe898003070 tx=0x7fe898006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.125+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe89c06d070 con 0x7fe8a805f5a0 2020-07-18T03:33:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.332+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8a8057450 con 0x7fe8a805f5a0 2020-07-18T03:33:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.332+0000 7fe88effd700 1 -- 172.21.15.47:0/3253258526 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fe89c06d070 con 0x7fe8a805f5a0 2020-07-18T03:33:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe884050410 msgr2=0x7fe8840528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe884050410 0x7fe8840528d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fe898003070 tx=0x7fe898006dd0).stop 2020-07-18T03:33:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe8a805f5a0 msgr2=0x7fe8a81423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe8a805f5a0 0x7fe8a81423c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe898007000 tx=0x7fe89c015000).stop 2020-07-18T03:33:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 shutdown_connections 2020-07-18T03:33:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe8a805f5a0 0x7fe8a81423c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 --2- 172.21.15.47:0/3253258526 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe884050410 0x7fe8840528d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 >> 172.21.15.47:0/3253258526 conn(0x7fe8a80b9340 msgr2=0x7fe8a80601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 shutdown_connections 2020-07-18T03:33:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:36.334+0000 7fe8ad647700 1 -- 172.21.15.47:0/3253258526 wait complete. 2020-07-18T03:33:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:36.347 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:36.347 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:36.348 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:37.349 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.476+0000 7ff2140a3700 1 Processor -- start 2020-07-18T03:33:37.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.477+0000 7ff2140a3700 1 -- start start 2020-07-18T03:33:37.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.477+0000 7ff2140a3700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 0x7ff20c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.477+0000 7ff2140a3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0b3ac0 con 0x7ff20c0b5370 2020-07-18T03:33:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.477+0000 7ff2140a3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0b3d20 con 0x7ff20c0b41c0 2020-07-18T03:33:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.477+0000 7ff2140a3700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0b5a10 con 0x7ff20c0ad9e0 2020-07-18T03:33:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff21163e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff20c0b41c0 0x7ff20c0ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44374/0 (socket says 172.21.15.47:44374) 2020-07-18T03:33:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff21163e700 1 -- 172.21.15.47:0/2799946272 learned_addr learned my addr 172.21.15.47:0/2799946272 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff212640700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 0x7ff20c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545607111 0 0) 0x7ff20c0b3d20 con 0x7ff20c0b41c0 2020-07-18T03:33:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1f8003240 con 0x7ff20c0b41c0 2020-07-18T03:33:37.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547442487 0 0) 0x7ff20c0b3ac0 con 0x7ff20c0b5370 2020-07-18T03:33:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff20c0b3d20 con 0x7ff20c0b5370 2020-07-18T03:33:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2481959464 0 0) 0x7ff1f8003240 con 0x7ff20c0b41c0 2020-07-18T03:33:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 msgr2=0x7ff20c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 0x7ff20c1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:37.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 >> v1:172.21.15.18:6789/0 conn(0x7ff20c0b5370 legacy=0x7ff20c05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c0b5750 con 0x7ff20c0b41c0 2020-07-18T03:33:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7ff2000016d0 con 0x7ff20c0b41c0 2020-07-18T03:33:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 >> v1:172.21.15.47:6789/0 conn(0x7ff20c0b41c0 legacy=0x7ff20c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1f80038f0 0x7ff1f8003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 0x7ff1f80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f80049d0 0x7ff1f8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0b5a10 con 0x7ff1f80038f0 2020-07-18T03:33:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f8005370 con 0x7ff1f80049d0 2020-07-18T03:33:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.478+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f800b780 con 0x7ff20c0ad9e0 2020-07-18T03:33:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f80049d0 0x7ff1f8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff211e3f700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1f80038f0 0x7ff1f8003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff21163e700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 0x7ff1f80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 -- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 msgr2=0x7ff1f80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff20c0ad9e0 0x7ff1f80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 -- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1f80038f0 msgr2=0x7ff1f8003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1f80038f0 0x7ff1f8003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 -- 172.21.15.47:0/2799946272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f8003240 con 0x7ff1f80049d0 2020-07-18T03:33:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f80049d0 0x7ff1f8004e50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7ff1fc00c000 tx=0x7ff1fc008040).ready entity=mon.1 client_cookie=b988842b7d1b4293 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1fc00e970 con 0x7ff1f80049d0 2020-07-18T03:33:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc00ead0 con 0x7ff1f80049d0 2020-07-18T03:33:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff210e3d700 1 -- 172.21.15.47:0/2799946272 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1fc00ee30 con 0x7ff1f80049d0 2020-07-18T03:33:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff2140a3700 1 -- 172.21.15.47:0/2799946272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f80049d0 msgr2=0x7ff1f8004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff2140a3700 1 --2- 172.21.15.47:0/2799946272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f80049d0 0x7ff1f8004e50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7ff1fc00c000 tx=0x7ff1fc008040).stop 2020-07-18T03:33:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff212640700 1 -- 172.21.15.47:0/2799946272 reap_dead start 2020-07-18T03:33:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff2140a3700 1 -- 172.21.15.47:0/2799946272 shutdown_connections 2020-07-18T03:33:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff2140a3700 1 -- 172.21.15.47:0/2799946272 >> 172.21.15.47:0/2799946272 conn(0x7ff20c0b9340 msgr2=0x7ff20c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff2140a3700 1 -- 172.21.15.47:0/2799946272 shutdown_connections 2020-07-18T03:33:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.479+0000 7ff2140a3700 1 -- 172.21.15.47:0/2799946272 wait complete. 2020-07-18T03:33:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.481+0000 7ff2140a3700 1 Processor -- start 2020-07-18T03:33:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.481+0000 7ff2140a3700 1 -- start start 2020-07-18T03:33:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.481+0000 7ff2140a3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff20c12e720 0x7ff20c132f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.481+0000 7ff2140a3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c065140 con 0x7ff20c12eb00 2020-07-18T03:33:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.481+0000 7ff2140a3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c065370 con 0x7ff20c131880 2020-07-18T03:33:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.481+0000 7ff2140a3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c065610 con 0x7ff20c12e720 2020-07-18T03:33:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff212640700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff20c131880 0x7ff20c065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44390/0 (socket says 172.21.15.47:44390) 2020-07-18T03:33:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff212640700 1 -- 172.21.15.47:0/1538728245 learned_addr learned my addr 172.21.15.47:0/1538728245 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff211e3f700 1 --2- 172.21.15.47:0/1538728245 >> v2:172.21.15.18:3300/0 conn(0x7ff20c12e720 0x7ff20c132f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955399420 0 0) 0x7ff20c065370 con 0x7ff20c131880 2020-07-18T03:33:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e4003240 con 0x7ff20c131880 2020-07-18T03:33:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff211e3f700 1 -- 172.21.15.47:0/1538728245 >> v1:172.21.15.47:6789/0 conn(0x7ff20c131880 legacy=0x7ff20c065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff211e3f700 1 -- 172.21.15.47:0/1538728245 >> v1:172.21.15.18:6789/0 conn(0x7ff20c12eb00 legacy=0x7ff20c12ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135638621 0 0) 0x7ff20c065140 con 0x7ff20c12eb00 2020-07-18T03:33:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff211e3f700 1 -- 172.21.15.47:0/1538728245 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c12c150 con 0x7ff20c12e720 2020-07-18T03:33:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff211e3f700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff20c12e720 0x7ff20c132f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ff20400d000 tx=0x7ff204007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff20400cb10 con 0x7ff20c12e720 2020-07-18T03:33:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c12c400 con 0x7ff20c12e720 2020-07-18T03:33:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20400cc70 con 0x7ff20c12e720 2020-07-18T03:33:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.482+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff204014480 con 0x7ff20c12e720 2020-07-18T03:33:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.483+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c066620 con 0x7ff20c12e720 2020-07-18T03:33:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.484+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c057450 con 0x7ff20c12e720 2020-07-18T03:33:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.484+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff204014630 con 0x7ff20c12e720 2020-07-18T03:33:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.484+0000 7ff209ffb700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff1e40502c0 0x7ff1e4052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.484+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff204067ac0 con 0x7ff20c12e720 2020-07-18T03:33:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.484+0000 7ff21163e700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff1e40502c0 0x7ff1e4052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:37.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.484+0000 7ff21163e700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff1e40502c0 0x7ff1e4052780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff2000014a0 tx=0x7ff20c0ba090).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:37.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.488+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff2040400e0 con 0x7ff20c12e720 2020-07-18T03:33:37.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.696+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff20c0b01a0 con 0x7ff20c12e720 2020-07-18T03:33:37.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.697+0000 7ff209ffb700 1 -- 172.21.15.47:0/1538728245 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7ff204019020 con 0x7ff20c12e720 2020-07-18T03:33:37.698 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:37.699 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:37.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff1e40502c0 msgr2=0x7ff1e4052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff1e40502c0 0x7ff1e4052780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff2000014a0 tx=0x7ff20c0ba090).stop 2020-07-18T03:33:37.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff20c12e720 msgr2=0x7ff20c132f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:37.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff20c12e720 0x7ff20c132f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ff20400d000 tx=0x7ff204007ef0).stop 2020-07-18T03:33:37.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 shutdown_connections 2020-07-18T03:33:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff1e40502c0 0x7ff1e4052780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 --2- 172.21.15.47:0/1538728245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff20c12e720 0x7ff20c132f00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 >> 172.21.15.47:0/1538728245 conn(0x7ff20c0b9340 msgr2=0x7ff20c0ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 shutdown_connections 2020-07-18T03:33:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:37.698+0000 7ff2140a3700 1 -- 172.21.15.47:0/1538728245 wait complete. 2020-07-18T03:33:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:37.710 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:38.712 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.838+0000 7f55cb7f6700 1 Processor -- start 2020-07-18T03:33:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.838+0000 7f55cb7f6700 1 -- start start 2020-07-18T03:33:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55cb7f6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 0x7f55c40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55cb7f6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f55c40b4620 0x7f55c40ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55cb7f6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c40b3ac0 con 0x7f55c40b5370 2020-07-18T03:33:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55cb7f6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c40b5a10 con 0x7f55c40b4620 2020-07-18T03:33:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55cb7f6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b0040 con 0x7f55c40b41c0 2020-07-18T03:33:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55c9592700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55c40b5370 0x7f55c405fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44394/0 (socket says 172.21.15.47:44394) 2020-07-18T03:33:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55c9592700 1 -- 172.21.15.47:0/1214558744 learned_addr learned my addr 172.21.15.47:0/1214558744 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55c9d93700 1 --2- 172.21.15.47:0/1214558744 >> v2:172.21.15.18:3300/0 conn(0x7f55c40b4620 0x7f55c40ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831846125 0 0) 0x7f55c40b3ac0 con 0x7f55c40b5370 2020-07-18T03:33:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55c8d91700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 0x7f55c40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.839+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55ac003240 con 0x7f55c40b5370 2020-07-18T03:33:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 328517797 0 0) 0x7f55ac003240 con 0x7f55c40b5370 2020-07-18T03:33:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 msgr2=0x7f55c40ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 0x7f55c40ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 >> v2:172.21.15.18:3300/0 conn(0x7f55c40b4620 msgr2=0x7f55c40ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 --2- 172.21.15.47:0/1214558744 >> v2:172.21.15.18:3300/0 conn(0x7f55c40b4620 0x7f55c40ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40adcb0 con 0x7f55c40b5370 2020-07-18T03:33:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f55b80016d0 con 0x7f55c40b5370 2020-07-18T03:33:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 >> v1:172.21.15.47:6789/0 conn(0x7f55c40b5370 legacy=0x7f55c405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 0x7f55ac003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55ac003c40 0x7f55ac0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55ac0049d0 0x7f55ac004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b5a10 con 0x7f55ac0049d0 2020-07-18T03:33:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b0040 con 0x7f55ac003c40 2020-07-18T03:33:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac004200 con 0x7f55c40b41c0 2020-07-18T03:33:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55ac003c40 0x7f55ac0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c9592700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 0x7f55ac003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c9d93700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55ac0049d0 0x7f55ac004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 -- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 msgr2=0x7f55ac003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c40b41c0 0x7f55ac003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 -- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55ac0049d0 msgr2=0x7f55ac004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55ac0049d0 0x7f55ac004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 -- 172.21.15.47:0/1214558744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac003240 con 0x7f55ac003c40 2020-07-18T03:33:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c9d93700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55ac0049d0 0x7f55ac004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.840+0000 7f55c8d91700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55ac003c40 0x7f55ac0044e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f55c000c000 tx=0x7f55c0008040).ready entity=mon.1 client_cookie=f1a048afdd8adcdc server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c000e970 con 0x7f55ac003c40 2020-07-18T03:33:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000ead0 con 0x7f55ac003c40 2020-07-18T03:33:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55b7fff700 1 -- 172.21.15.47:0/1214558744 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c000ee30 con 0x7f55ac003c40 2020-07-18T03:33:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55cb7f6700 1 -- 172.21.15.47:0/1214558744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55ac003c40 msgr2=0x7f55ac0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55cb7f6700 1 --2- 172.21.15.47:0/1214558744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55ac003c40 0x7f55ac0044e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f55c000c000 tx=0x7f55c0008040).stop 2020-07-18T03:33:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55c9d93700 1 -- 172.21.15.47:0/1214558744 reap_dead start 2020-07-18T03:33:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55cb7f6700 1 -- 172.21.15.47:0/1214558744 shutdown_connections 2020-07-18T03:33:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55cb7f6700 1 -- 172.21.15.47:0/1214558744 >> 172.21.15.47:0/1214558744 conn(0x7f55c40b9340 msgr2=0x7f55c40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55cb7f6700 1 -- 172.21.15.47:0/1214558744 shutdown_connections 2020-07-18T03:33:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.841+0000 7f55cb7f6700 1 -- 172.21.15.47:0/1214558744 wait complete. 2020-07-18T03:33:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.842+0000 7f55cb7f6700 1 Processor -- start 2020-07-18T03:33:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.842+0000 7f55cb7f6700 1 -- start start 2020-07-18T03:33:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55cb7f6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f55c4145ae0 0x7f55c4142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55cb7f6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f55c413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55cb7f6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c4140bc0 con 0x7f55c4145700 2020-07-18T03:33:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55cb7f6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c41409e0 con 0x7f55c4145ae0 2020-07-18T03:33:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55cb7f6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4140e60 con 0x7f55c4155250 2020-07-18T03:33:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55c9592700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55c4145700 0x7f55c40624f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44404/0 (socket says 172.21.15.47:44404) 2020-07-18T03:33:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.843+0000 7f55c9592700 1 -- 172.21.15.47:0/659037552 learned_addr learned my addr 172.21.15.47:0/659037552 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9d93700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f55c413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c8d91700 1 --2- 172.21.15.47:0/659037552 >> v2:172.21.15.18:3300/0 conn(0x7f55c4145ae0 0x7f55c4142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763055196 0 0) 0x7f55c4140bc0 con 0x7f55c4145700 2020-07-18T03:33:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f559c003240 con 0x7f55c4145700 2020-07-18T03:33:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2791048395 0 0) 0x7f559c003240 con 0x7f55c4145700 2020-07-18T03:33:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f55c4140bc0 con 0x7f55c4145700 2020-07-18T03:33:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f55b8007b40 con 0x7f55c4145700 2020-07-18T03:33:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 msgr2=0x7f55c413cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f55c413cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 >> v2:172.21.15.18:3300/0 conn(0x7f55c4145ae0 msgr2=0x7f55c4142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 --2- 172.21.15.47:0/659037552 >> v2:172.21.15.18:3300/0 conn(0x7f55c4145ae0 0x7f55c4142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 >> v1:172.21.15.47:6789/0 conn(0x7f55c4145700 legacy=0x7f55c40624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c003800 0x7f559c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f559c0044e0 0x7f559c004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f559c004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41409e0 con 0x7f559c0044e0 2020-07-18T03:33:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4140e60 con 0x7f559c003800 2020-07-18T03:33:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c005300 con 0x7f55c4155250 2020-07-18T03:33:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9592700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c003800 0x7f559c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c8d91700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f559c0044e0 0x7f559c004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9d93700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f559c004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9592700 1 -- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 msgr2=0x7f559c004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9592700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f559c004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9592700 1 -- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f559c0044e0 msgr2=0x7f559c004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9592700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f559c0044e0 0x7f559c004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9592700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c413d270 con 0x7f559c003800 2020-07-18T03:33:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55c9d93700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f55c4155250 0x7f559c004e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.844+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c413d460 con 0x7f559c003800 2020-07-18T03:33:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.845+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c413cfc0 con 0x7f559c003800 2020-07-18T03:33:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.845+0000 7f55c9592700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c003800 0x7f559c003c20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f55b8008000 tx=0x7f55b80077d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.845+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55b8011590 con 0x7f559c003800 2020-07-18T03:33:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.845+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b80116f0 con 0x7f559c003800 2020-07-18T03:33:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.845+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55b8011a50 con 0x7f559c003800 2020-07-18T03:33:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.846+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c4057450 con 0x7f559c003800 2020-07-18T03:33:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.846+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f55b8011bb0 con 0x7f559c003800 2020-07-18T03:33:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.846+0000 7f55b57fa700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f559c058090 0x7f559c05a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.846+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f55b8068200 con 0x7f559c003800 2020-07-18T03:33:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.846+0000 7f55c8d91700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f559c058090 0x7f559c05a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:38.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.847+0000 7f55c8d91700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f559c058090 0x7f559c05a550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f55c001c000 tx=0x7f55c001a000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:38.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:38.850+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f55b80407c0 con 0x7f559c003800 2020-07-18T03:33:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.058+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55c413d770 con 0x7f559c003800 2020-07-18T03:33:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.058+0000 7f55b57fa700 1 -- 172.21.15.47:0/659037552 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f55b8019020 con 0x7f559c003800 2020-07-18T03:33:39.060 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:39.060 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f559c058090 msgr2=0x7f559c05a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f559c058090 0x7f559c05a550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f55c001c000 tx=0x7f55c001a000).stop 2020-07-18T03:33:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c003800 msgr2=0x7f559c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 --2- 172.21.15.47:0/659037552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f559c003800 0x7f559c003c20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f55b8008000 tx=0x7f55b80077d0).stop 2020-07-18T03:33:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55c9d93700 1 -- 172.21.15.47:0/659037552 reap_dead start 2020-07-18T03:33:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 shutdown_connections 2020-07-18T03:33:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 >> 172.21.15.47:0/659037552 conn(0x7f55c40b9340 msgr2=0x7f55c40b8510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 shutdown_connections 2020-07-18T03:33:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:39.060+0000 7f55cb7f6700 1 -- 172.21.15.47:0/659037552 wait complete. 2020-07-18T03:33:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:39.072 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:40.073 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.200+0000 7f40a29cd700 1 Processor -- start 2020-07-18T03:33:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.200+0000 7f40a29cd700 1 -- start start 2020-07-18T03:33:40.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.201+0000 7f40a29cd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c062d60 0x7f409c05f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:40.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.201+0000 7f40a29cd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f409c14b150 0x7f409c146520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:40.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.201+0000 7f40a29cd700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c0b7ae0 con 0x7f409c14acf0 2020-07-18T03:33:40.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.201+0000 7f40a29cd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c0b7d10 con 0x7f409c14b150 2020-07-18T03:33:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.201+0000 7f40a29cd700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c14a850 con 0x7f409c062d60 2020-07-18T03:33:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a0f6a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f409c14b150 0x7f409c146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409bfff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c062d60 0x7f409c05f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409b7fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f409c14acf0 0x7f409c0b15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34954/0 (socket says 172.21.15.47:34954) 2020-07-18T03:33:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409b7fe700 1 -- 172.21.15.47:0/2872371766 learned_addr learned my addr 172.21.15.47:0/2872371766 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409affd700 1 -- 172.21.15.47:0/2872371766 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514833376 0 0) 0x7f409c0b7ae0 con 0x7f409c14acf0 2020-07-18T03:33:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409affd700 1 -- 172.21.15.47:0/2872371766 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4088003240 con 0x7f409c14acf0 2020-07-18T03:33:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a0f6a700 1 -- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c062d60 msgr2=0x7f409c05f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a0f6a700 1 --2- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c062d60 0x7f409c05f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a0f6a700 1 -- 172.21.15.47:0/2872371766 >> v1:172.21.15.18:6789/0 conn(0x7f409c14acf0 legacy=0x7f409c0b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a0f6a700 1 -- 172.21.15.47:0/2872371766 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c14a620 con 0x7f409c14b150 2020-07-18T03:33:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409bfff700 1 --2- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c062d60 0x7f409c05f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a0f6a700 1 --2- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f409c14b150 0x7f409c146520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f40940032b0 tx=0x7f40940091d0).ready entity=mon.0 client_cookie=88fd2bf526af6be1 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:40.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409affd700 1 -- 172.21.15.47:0/2872371766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409400b940 con 0x7f409c14b150 2020-07-18T03:33:40.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409affd700 1 -- 172.21.15.47:0/2872371766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400baa0 con 0x7f409c14b150 2020-07-18T03:33:40.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f409affd700 1 -- 172.21.15.47:0/2872371766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409400bc10 con 0x7f409c14b150 2020-07-18T03:33:40.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.202+0000 7f40a29cd700 1 -- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f409c14b150 msgr2=0x7f409c146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:40.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 --2- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f409c14b150 0x7f409c146520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f40940032b0 tx=0x7f40940091d0).stop 2020-07-18T03:33:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 -- 172.21.15.47:0/2872371766 shutdown_connections 2020-07-18T03:33:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 --2- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f409c14b150 0x7f409c146520 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 --2- 172.21.15.47:0/2872371766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f409c062d60 0x7f409c05f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 -- 172.21.15.47:0/2872371766 >> 172.21.15.47:0/2872371766 conn(0x7f409c05cb10 msgr2=0x7f409c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 -- 172.21.15.47:0/2872371766 shutdown_connections 2020-07-18T03:33:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.203+0000 7f40a29cd700 1 -- 172.21.15.47:0/2872371766 wait complete. 2020-07-18T03:33:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.204+0000 7f40a29cd700 1 Processor -- start 2020-07-18T03:33:40.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.204+0000 7f40a29cd700 1 -- start start 2020-07-18T03:33:40.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a29cd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409c062d60 0x7f409c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:40.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a29cd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f409c14acf0 0x7f409c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:40.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a29cd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c1553b0 con 0x7f409c158960 2020-07-18T03:33:40.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a29cd700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c146e80 con 0x7f409c14acf0 2020-07-18T03:33:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a29cd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c147120 con 0x7f409c062d60 2020-07-18T03:33:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f409bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f409c062d60 0x7f409c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a0f6a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f409c158960 0x7f409c1475c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44428/0 (socket says 172.21.15.47:44428) 2020-07-18T03:33:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f40a0f6a700 1 -- 172.21.15.47:0/3002401097 learned_addr learned my addr 172.21.15.47:0/3002401097 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.205+0000 7f409b7fe700 1 --2- 172.21.15.47:0/3002401097 >> v2:172.21.15.18:3300/0 conn(0x7f409c14acf0 0x7f409c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f409bfff700 1 -- 172.21.15.47:0/3002401097 >> v2:172.21.15.18:3300/0 conn(0x7f409c14acf0 msgr2=0x7f409c158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f409bfff700 1 --2- 172.21.15.47:0/3002401097 >> v2:172.21.15.18:3300/0 conn(0x7f409c14acf0 0x7f409c158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f409bfff700 1 -- 172.21.15.47:0/3002401097 >> v1:172.21.15.47:6789/0 conn(0x7f409c158960 legacy=0x7f409c1475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f409bfff700 1 -- 172.21.15.47:0/3002401097 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c157f40 con 0x7f409c062d60 2020-07-18T03:33:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919308166 0 0) 0x7f409c1553b0 con 0x7f409c158960 2020-07-18T03:33:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f409bfff700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c062d60 0x7f409c154ec0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f408c00c000 tx=0x7f408c006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c00cb40 con 0x7f409c062d60 2020-07-18T03:33:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00cca0 con 0x7f409c062d60 2020-07-18T03:33:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c0123f0 con 0x7f409c062d60 2020-07-18T03:33:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c158130 con 0x7f409c062d60 2020-07-18T03:33:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.206+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c147f10 con 0x7f409c062d60 2020-07-18T03:33:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.207+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c146cd0 con 0x7f409c062d60 2020-07-18T03:33:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.207+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f408c0125e0 con 0x7f409c062d60 2020-07-18T03:33:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.207+0000 7f407bfff700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f407404ff80 0x7f4074052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.207+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f408c068340 con 0x7f409c062d60 2020-07-18T03:33:40.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.207+0000 7f409b7fe700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f407404ff80 0x7f4074052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:40.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.208+0000 7f409b7fe700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f407404ff80 0x7f4074052440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f409000d000 tx=0x7f4090009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:40.220 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.211+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f408c040a20 con 0x7f409c062d60 2020-07-18T03:33:40.420 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.419+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f409c0b3d70 con 0x7f409c062d60 2020-07-18T03:33:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.420+0000 7f407bfff700 1 -- 172.21.15.47:0/3002401097 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f408c044040 con 0x7f409c062d60 2020-07-18T03:33:40.421 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:40.422 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f407404ff80 msgr2=0x7f4074052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f407404ff80 0x7f4074052440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f409000d000 tx=0x7f4090009040).stop 2020-07-18T03:33:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c062d60 msgr2=0x7f409c154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c062d60 0x7f409c154ec0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f408c00c000 tx=0x7f408c006dd0).stop 2020-07-18T03:33:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 shutdown_connections 2020-07-18T03:33:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f407404ff80 0x7f4074052440 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 --2- 172.21.15.47:0/3002401097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409c062d60 0x7f409c154ec0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 --2- 172.21.15.47:0/3002401097 >> v2:172.21.15.18:3300/0 conn(0x7f409c14acf0 0x7f409c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 >> 172.21.15.47:0/3002401097 conn(0x7f409c05cb10 msgr2=0x7f409c05b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 shutdown_connections 2020-07-18T03:33:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:40.421+0000 7f40a29cd700 1 -- 172.21.15.47:0/3002401097 wait complete. 2020-07-18T03:33:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:40.433 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:41.434 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.561+0000 7fc883d94700 1 Processor -- start 2020-07-18T03:33:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.561+0000 7fc883d94700 1 -- start start 2020-07-18T03:33:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc883d94700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc87c0ad100 0x7fc87c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc883d94700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c1443f0 0x7fc87c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc883d94700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c147330 con 0x7fc87c147a00 2020-07-18T03:33:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc883d94700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c0ae530 con 0x7fc87c0ad100 2020-07-18T03:33:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc883d94700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b57e0 con 0x7fc87c1443f0 2020-07-18T03:33:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc88132f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc87c147a00 0x7fc87c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44432/0 (socket says 172.21.15.47:44432) 2020-07-18T03:33:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc881b30700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc87c0ad100 0x7fc87c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc88132f700 1 -- 172.21.15.47:0/3873087343 learned_addr learned my addr 172.21.15.47:0/3873087343 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc882331700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c1443f0 0x7fc87c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc881b30700 1 -- 172.21.15.47:0/3873087343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c1443f0 msgr2=0x7fc87c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc881b30700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c1443f0 0x7fc87c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc881b30700 1 -- 172.21.15.47:0/3873087343 >> v1:172.21.15.47:6789/0 conn(0x7fc87c147a00 legacy=0x7fc87c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc881b30700 1 -- 172.21.15.47:0/3873087343 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c0b5a80 con 0x7fc87c0ad100 2020-07-18T03:33:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc880b2e700 1 -- 172.21.15.47:0/3873087343 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690901540 0 0) 0x7fc87c147330 con 0x7fc87c147a00 2020-07-18T03:33:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc882331700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c1443f0 0x7fc87c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.562+0000 7fc881b30700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c0ad100 0x7fc87c0b4c10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fc86c002f30 tx=0x7fc86c006dd0).ready entity=mon.1 client_cookie=24c9552f28ca559e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc880b2e700 1 -- 172.21.15.47:0/3873087343 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86c0097f0 con 0x7fc87c0ad100 2020-07-18T03:33:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc880b2e700 1 -- 172.21.15.47:0/3873087343 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c009950 con 0x7fc87c0ad100 2020-07-18T03:33:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc880b2e700 1 -- 172.21.15.47:0/3873087343 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86c009cb0 con 0x7fc87c0ad100 2020-07-18T03:33:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 -- 172.21.15.47:0/3873087343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c0ad100 msgr2=0x7fc87c0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c0ad100 0x7fc87c0b4c10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fc86c002f30 tx=0x7fc86c006dd0).stop 2020-07-18T03:33:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 -- 172.21.15.47:0/3873087343 shutdown_connections 2020-07-18T03:33:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c1443f0 0x7fc87c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 --2- 172.21.15.47:0/3873087343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c0ad100 0x7fc87c0b4c10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 -- 172.21.15.47:0/3873087343 >> 172.21.15.47:0/3873087343 conn(0x7fc87c0b9340 msgr2=0x7fc87c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 -- 172.21.15.47:0/3873087343 shutdown_connections 2020-07-18T03:33:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.563+0000 7fc883d94700 1 -- 172.21.15.47:0/3873087343 wait complete. 2020-07-18T03:33:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.564+0000 7fc883d94700 1 Processor -- start 2020-07-18T03:33:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.564+0000 7fc883d94700 1 -- start start 2020-07-18T03:33:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc883d94700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c147a00 0x7fc87c0650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc883d94700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc87c1553f0 0x7fc87c0658f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc883d94700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c066ef0 con 0x7fc87c1443f0 2020-07-18T03:33:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc883d94700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c155d70 con 0x7fc87c1553f0 2020-07-18T03:33:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc883d94700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c156010 con 0x7fc87c147a00 2020-07-18T03:33:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc87c1553f0 0x7fc87c0658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc87c1553f0 0x7fc87c0658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35122/0 (socket says 172.21.15.47:35122) 2020-07-18T03:33:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 -- 172.21.15.47:0/2450207864 learned_addr learned my addr 172.21.15.47:0/2450207864 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc88132f700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c147a00 0x7fc87c0650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 -- 172.21.15.47:0/2450207864 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c147a00 msgr2=0x7fc87c0650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c147a00 0x7fc87c0650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 -- 172.21.15.47:0/2450207864 >> v1:172.21.15.18:6789/0 conn(0x7fc87c1443f0 legacy=0x7fc87c1460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc882331700 1 -- 172.21.15.47:0/2450207864 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c1562b0 con 0x7fc87c1553f0 2020-07-18T03:33:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.565+0000 7fc88132f700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c147a00 0x7fc87c0650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.566+0000 7fc882331700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c1553f0 0x7fc87c0658f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7fc874006de0 tx=0x7fc874008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.566+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc87400cb30 con 0x7fc87c1553f0 2020-07-18T03:33:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.566+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87400cc90 con 0x7fc87c1553f0 2020-07-18T03:33:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.566+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c061c90 con 0x7fc87c1553f0 2020-07-18T03:33:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.566+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc87401a470 con 0x7fc87c1553f0 2020-07-18T03:33:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.566+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c155a20 con 0x7fc87c1553f0 2020-07-18T03:33:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.567+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c060500 con 0x7fc87c1553f0 2020-07-18T03:33:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.567+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc8740233f0 con 0x7fc87c1553f0 2020-07-18T03:33:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.567+0000 7fc869ffb700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc85404fe60 0x7fc854052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.568+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc87406a1b0 con 0x7fc87c1553f0 2020-07-18T03:33:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.568+0000 7fc881b30700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc85404fe60 0x7fc854052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:41.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.568+0000 7fc881b30700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc85404fe60 0x7fc854052320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fc87c063130 tx=0x7fc86c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:41.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.571+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc874042890 con 0x7fc87c1553f0 2020-07-18T03:33:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.787+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc87c057450 con 0x7fc87c1553f0 2020-07-18T03:33:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.787+0000 7fc869ffb700 1 -- 172.21.15.47:0/2450207864 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fc87400f040 con 0x7fc87c1553f0 2020-07-18T03:33:41.789 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:41.789 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:41.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc85404fe60 msgr2=0x7fc854052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:41.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc85404fe60 0x7fc854052320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fc87c063130 tx=0x7fc86c015000).stop 2020-07-18T03:33:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c1553f0 msgr2=0x7fc87c0658f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c1553f0 0x7fc87c0658f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7fc874006de0 tx=0x7fc874008040).stop 2020-07-18T03:33:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 shutdown_connections 2020-07-18T03:33:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc85404fe60 0x7fc854052320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c1553f0 0x7fc87c0658f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 --2- 172.21.15.47:0/2450207864 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc87c147a00 0x7fc87c0650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 >> 172.21.15.47:0/2450207864 conn(0x7fc87c0b9340 msgr2=0x7fc87c1439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 shutdown_connections 2020-07-18T03:33:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:41.789+0000 7fc883d94700 1 -- 172.21.15.47:0/2450207864 wait complete. 2020-07-18T03:33:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:41.800 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:42.802 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 Processor -- start 2020-07-18T03:33:42.930 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 -- start start 2020-07-18T03:33:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7effb00ad100 0x7effb00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb01443f0 0x7effb0147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb0147330 con 0x7effb0147a00 2020-07-18T03:33:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00ae530 con 0x7effb00ad100 2020-07-18T03:33:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.929+0000 7effb5685700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00b57e0 con 0x7effb01443f0 2020-07-18T03:33:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb01443f0 0x7effb0147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb01443f0 0x7effb0147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35130/0 (socket says 172.21.15.47:35130) 2020-07-18T03:33:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 -- 172.21.15.47:0/3444074632 learned_addr learned my addr 172.21.15.47:0/3444074632 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaeffd700 1 --2- 172.21.15.47:0/3444074632 >> v2:172.21.15.18:3300/0 conn(0x7effb00ad100 0x7effb00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 -- 172.21.15.47:0/3444074632 >> v2:172.21.15.18:3300/0 conn(0x7effb00ad100 msgr2=0x7effb00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 --2- 172.21.15.47:0/3444074632 >> v2:172.21.15.18:3300/0 conn(0x7effb00ad100 0x7effb00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 -- 172.21.15.47:0/3444074632 >> v1:172.21.15.18:6789/0 conn(0x7effb0147a00 legacy=0x7effb005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 -- 172.21.15.47:0/3444074632 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00b5a80 con 0x7effb01443f0 2020-07-18T03:33:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaeffd700 1 --2- 172.21.15.47:0/3444074632 >> v2:172.21.15.18:3300/0 conn(0x7effb00ad100 0x7effb00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effaf7fe700 1 --2- 172.21.15.47:0/3444074632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb01443f0 0x7effb0147e40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7effa0002f30 tx=0x7effa0006e90).ready entity=mon.1 client_cookie=1c2f406c53a20d6 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effadffb700 1 -- 172.21.15.47:0/3444074632 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa00099c0 con 0x7effb01443f0 2020-07-18T03:33:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effadffb700 1 -- 172.21.15.47:0/3444074632 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0009b20 con 0x7effb01443f0 2020-07-18T03:33:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effadffb700 1 -- 172.21.15.47:0/3444074632 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa0009e80 con 0x7effb01443f0 2020-07-18T03:33:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 -- 172.21.15.47:0/3444074632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb01443f0 msgr2=0x7effb0147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 --2- 172.21.15.47:0/3444074632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb01443f0 0x7effb0147e40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7effa0002f30 tx=0x7effa0006e90).stop 2020-07-18T03:33:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 -- 172.21.15.47:0/3444074632 shutdown_connections 2020-07-18T03:33:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 --2- 172.21.15.47:0/3444074632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb01443f0 0x7effb0147e40 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 --2- 172.21.15.47:0/3444074632 >> v2:172.21.15.18:3300/0 conn(0x7effb00ad100 0x7effb00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 -- 172.21.15.47:0/3444074632 >> 172.21.15.47:0/3444074632 conn(0x7effb00b9340 msgr2=0x7effb00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 -- 172.21.15.47:0/3444074632 shutdown_connections 2020-07-18T03:33:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.930+0000 7effb5685700 1 -- 172.21.15.47:0/3444074632 wait complete. 2020-07-18T03:33:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.932+0000 7effb5685700 1 Processor -- start 2020-07-18T03:33:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.932+0000 7effb5685700 1 -- start start 2020-07-18T03:33:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effb5685700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7effb0147a00 0x7effb0152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effb5685700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb01431b0 0x7effb01435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:42.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effb5685700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb0066720 con 0x7effb00ad100 2020-07-18T03:33:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effb5685700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb013a760 con 0x7effb0147a00 2020-07-18T03:33:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effb5685700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0064bc0 con 0x7effb01431b0 2020-07-18T03:33:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effaeffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7effb00ad100 0x7effb013bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44454/0 (socket says 172.21.15.47:44454) 2020-07-18T03:33:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effaeffd700 1 -- 172.21.15.47:0/3574361872 learned_addr learned my addr 172.21.15.47:0/3574361872 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effaf7fe700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb01431b0 0x7effb01435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effae7fc700 1 --2- 172.21.15.47:0/3574361872 >> v2:172.21.15.18:3300/0 conn(0x7effb0147a00 0x7effb0152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834445590 0 0) 0x7effb0066720 con 0x7effb00ad100 2020-07-18T03:33:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.933+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff8c003240 con 0x7effb00ad100 2020-07-18T03:33:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 815693370 0 0) 0x7eff8c003240 con 0x7effb00ad100 2020-07-18T03:33:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effae7fc700 1 -- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb01431b0 msgr2=0x7effb01435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effae7fc700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb01431b0 0x7effb01435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effae7fc700 1 -- 172.21.15.47:0/3574361872 >> v1:172.21.15.47:6789/0 conn(0x7effb00ad100 legacy=0x7effb013bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effae7fc700 1 -- 172.21.15.47:0/3574361872 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0152350 con 0x7effb0147a00 2020-07-18T03:33:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effaf7fe700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb01431b0 0x7effb01435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effae7fc700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effb0147a00 0x7effb0152a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7effa8009890 tx=0x7eff98007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effa80096f0 con 0x7effb0147a00 2020-07-18T03:33:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb0152650 con 0x7effb0147a00 2020-07-18T03:33:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800ce00 con 0x7effb0147a00 2020-07-18T03:33:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff98007240 con 0x7effb0147a00 2020-07-18T03:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.934+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00b31d0 con 0x7effb0147a00 2020-07-18T03:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.935+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0057450 con 0x7effb0147a00 2020-07-18T03:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.935+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7eff98010070 con 0x7effb0147a00 2020-07-18T03:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.935+0000 7effa6ffd700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8c050330 0x7eff8c0527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.936+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7eff98066d80 con 0x7effb0147a00 2020-07-18T03:33:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.936+0000 7effaeffd700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8c050330 0x7eff8c0527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.936+0000 7effaeffd700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8c050330 0x7eff8c0527f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7effa8009850 tx=0x7effa8008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:42.940+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7eff9803f3e0 con 0x7effb0147a00 2020-07-18T03:33:43.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.151+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effb0066e60 con 0x7effb0147a00 2020-07-18T03:33:43.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.151+0000 7effa6ffd700 1 -- 172.21.15.47:0/3574361872 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7eff98068090 con 0x7effb0147a00 2020-07-18T03:33:43.153 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:43.153 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8c050330 msgr2=0x7eff8c0527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8c050330 0x7eff8c0527f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7effa8009850 tx=0x7effa8008040).stop 2020-07-18T03:33:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effb0147a00 msgr2=0x7effb0152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effb0147a00 0x7effb0152a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7effa8009890 tx=0x7eff98007b70).stop 2020-07-18T03:33:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 shutdown_connections 2020-07-18T03:33:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7effb01431b0 0x7effb01435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7eff8c050330 0x7eff8c0527f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 --2- 172.21.15.47:0/3574361872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effb0147a00 0x7effb0152a90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 >> 172.21.15.47:0/3574361872 conn(0x7effb00b9340 msgr2=0x7effb005f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 shutdown_connections 2020-07-18T03:33:43.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:43.153+0000 7effb5685700 1 -- 172.21.15.47:0/3574361872 wait complete. 2020-07-18T03:33:43.157 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:43.165 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:44.166 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.292+0000 7f1f2651a700 1 Processor -- start 2020-07-18T03:33:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.293+0000 7f1f2651a700 1 -- start start 2020-07-18T03:33:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.293+0000 7f1f2651a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.293+0000 7f1f2651a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 0x7f1f20143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.293+0000 7f1f2651a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f20147330 con 0x7f1f200ad100 2020-07-18T03:33:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.293+0000 7f1f2651a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200ae530 con 0x7f1f20147a00 2020-07-18T03:33:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.293+0000 7f1f2651a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200b57e0 con 0x7f1f201433e0 2020-07-18T03:33:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f1f7fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 0x7f1f20143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 0x7f1f20143820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:55974/0 (socket says 172.21.15.47:55974) 2020-07-18T03:33:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f1f7fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:49054/0 (socket says 172.21.15.47:49054) 2020-07-18T03:33:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 -- 172.21.15.47:0/2233048781 learned_addr learned my addr 172.21.15.47:0/2233048781 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f1ffff700 1 --1- 172.21.15.47:0/2233048781 >> v1:172.21.15.18:6789/0 conn(0x7f1f200ad100 0x7f1f200b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:34994/0 (socket says 172.21.15.47:34994) 2020-07-18T03:33:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 -- 172.21.15.47:0/2233048781 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 msgr2=0x7f1f20147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 --2- 172.21.15.47:0/2233048781 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 -- 172.21.15.47:0/2233048781 >> v1:172.21.15.18:6789/0 conn(0x7f1f200ad100 legacy=0x7f1f200b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 -- 172.21.15.47:0/2233048781 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200b5a80 con 0x7f1f201433e0 2020-07-18T03:33:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f1f7fe700 1 --2- 172.21.15.47:0/2233048781 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.294+0000 7f1f24ab7700 1 --2- 172.21.15.47:0/2233048781 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 0x7f1f20143820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f1f180032b0 tx=0x7f1f1800b510).ready entity=mon.2 client_cookie=a1548533e29a8e0f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f1effd700 1 -- 172.21.15.47:0/2233048781 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f18010660 con 0x7f1f201433e0 2020-07-18T03:33:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f1effd700 1 -- 172.21.15.47:0/2233048781 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f180107c0 con 0x7f1f201433e0 2020-07-18T03:33:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f1effd700 1 -- 172.21.15.47:0/2233048781 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f18010b20 con 0x7f1f201433e0 2020-07-18T03:33:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 -- 172.21.15.47:0/2233048781 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 msgr2=0x7f1f20143820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 --2- 172.21.15.47:0/2233048781 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 0x7f1f20143820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f1f180032b0 tx=0x7f1f1800b510).stop 2020-07-18T03:33:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 -- 172.21.15.47:0/2233048781 shutdown_connections 2020-07-18T03:33:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 --2- 172.21.15.47:0/2233048781 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f201433e0 0x7f1f20143820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 --2- 172.21.15.47:0/2233048781 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 -- 172.21.15.47:0/2233048781 >> 172.21.15.47:0/2233048781 conn(0x7f1f200b9340 msgr2=0x7f1f200b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 -- 172.21.15.47:0/2233048781 shutdown_connections 2020-07-18T03:33:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.295+0000 7f1f2651a700 1 -- 172.21.15.47:0/2233048781 wait complete. 2020-07-18T03:33:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.296+0000 7f1f2651a700 1 Processor -- start 2020-07-18T03:33:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.296+0000 7f1f2651a700 1 -- start start 2020-07-18T03:33:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f2651a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f2651a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f20066420 0x7f1f20138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f2651a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f20063500 con 0x7f1f200ad100 2020-07-18T03:33:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f2651a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200637a0 con 0x7f1f20147a00 2020-07-18T03:33:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f2651a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20063a40 con 0x7f1f20066420 2020-07-18T03:33:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f24ab7700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f20066420 0x7f1f20138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f1ffff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f1f200ad100 0x7f1f200b1f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35002/0 (socket says 172.21.15.47:35002) 2020-07-18T03:33:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f1f7fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.297+0000 7f1f1ffff700 1 -- 172.21.15.47:0/709982603 learned_addr learned my addr 172.21.15.47:0/709982603 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927811046 0 0) 0x7f1f20063500 con 0x7f1f200ad100 2020-07-18T03:33:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef8003240 con 0x7f1f200ad100 2020-07-18T03:33:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f24ab7700 1 -- 172.21.15.47:0/709982603 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 msgr2=0x7f1f20139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f24ab7700 1 --2- 172.21.15.47:0/709982603 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f24ab7700 1 -- 172.21.15.47:0/709982603 >> v1:172.21.15.18:6789/0 conn(0x7f1f200ad100 legacy=0x7f1f200b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f24ab7700 1 -- 172.21.15.47:0/709982603 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20066110 con 0x7f1f20066420 2020-07-18T03:33:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f24ab7700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f20066420 0x7f1f20138f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f1f180032b0 tx=0x7f1f200b03e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f18018a70 con 0x7f1f20066420 2020-07-18T03:33:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18018bd0 con 0x7f1f20066420 2020-07-18T03:33:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f20061b60 con 0x7f1f20066420 2020-07-18T03:33:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f18023400 con 0x7f1f20066420 2020-07-18T03:33:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.298+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f20139400 con 0x7f1f20066420 2020-07-18T03:33:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.299+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20057450 con 0x7f1f20066420 2020-07-18T03:33:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.299+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1f18021070 con 0x7f1f20066420 2020-07-18T03:33:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.300+0000 7f1efffff700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef8050370 0x7f1ef8052830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.300+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1f1806b080 con 0x7f1f20066420 2020-07-18T03:33:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.300+0000 7f1f1ffff700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef8050370 0x7f1ef8052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.300+0000 7f1f1ffff700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef8050370 0x7f1ef8052830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f1f10000ef0 tx=0x7f1f10007810).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.304+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1f18043760 con 0x7f1f20066420 2020-07-18T03:33:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.512+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f20063b80 con 0x7f1f20066420 2020-07-18T03:33:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.513+0000 7f1efffff700 1 -- 172.21.15.47:0/709982603 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f1f18046d80 con 0x7f1f20066420 2020-07-18T03:33:44.514 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:44.514 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef8050370 msgr2=0x7f1ef8052830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef8050370 0x7f1ef8052830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f1f10000ef0 tx=0x7f1f10007810).stop 2020-07-18T03:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f20066420 msgr2=0x7f1f20138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f20066420 0x7f1f20138f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f1f180032b0 tx=0x7f1f200b03e0).stop 2020-07-18T03:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 shutdown_connections 2020-07-18T03:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1ef8050370 0x7f1ef8052830 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 --2- 172.21.15.47:0/709982603 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1f20066420 0x7f1f20138f10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 --2- 172.21.15.47:0/709982603 >> v2:172.21.15.18:3300/0 conn(0x7f1f20147a00 0x7f1f20139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 >> 172.21.15.47:0/709982603 conn(0x7f1f200b9340 msgr2=0x7f1f20143820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 shutdown_connections 2020-07-18T03:33:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:44.514+0000 7f1f2651a700 1 -- 172.21.15.47:0/709982603 wait complete. 2020-07-18T03:33:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:44.526 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:45.527 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.653+0000 7f3934e56700 1 Processor -- start 2020-07-18T03:33:45.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.653+0000 7f3934e56700 1 -- start start 2020-07-18T03:33:45.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f3934e56700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3930147a00 0x7f3930147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:45.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f3934e56700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39301433e0 0x7f3930143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:45.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f3934e56700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3930147330 con 0x7f39300ad100 2020-07-18T03:33:45.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f3934e56700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300ae530 con 0x7f3930147a00 2020-07-18T03:33:45.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f3934e56700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300b57e0 con 0x7f39301433e0 2020-07-18T03:33:45.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f392dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3930147a00 0x7f3930147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:45.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f392dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3930147a00 0x7f3930147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35156/0 (socket says 172.21.15.47:35156) 2020-07-18T03:33:45.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.654+0000 7f392dd9b700 1 -- 172.21.15.47:0/1132179825 learned_addr learned my addr 172.21.15.47:0/1132179825 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392ed9d700 1 --2- 172.21.15.47:0/1132179825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39301433e0 0x7f3930143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392dd9b700 1 -- 172.21.15.47:0/1132179825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39301433e0 msgr2=0x7f3930143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392dd9b700 1 --2- 172.21.15.47:0/1132179825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39301433e0 0x7f3930143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392dd9b700 1 -- 172.21.15.47:0/1132179825 >> v1:172.21.15.18:6789/0 conn(0x7f39300ad100 legacy=0x7f39300b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392dd9b700 1 -- 172.21.15.47:0/1132179825 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300b5a80 con 0x7f3930147a00 2020-07-18T03:33:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392dd9b700 1 --2- 172.21.15.47:0/1132179825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3930147a00 0x7f3930147e40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f3920002f30 tx=0x7f3920006e90).ready entity=mon.1 client_cookie=48561368f4d1a826 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392d59a700 1 -- 172.21.15.47:0/1132179825 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3920009850 con 0x7f3930147a00 2020-07-18T03:33:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392d59a700 1 -- 172.21.15.47:0/1132179825 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39200099b0 con 0x7f3930147a00 2020-07-18T03:33:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f392d59a700 1 -- 172.21.15.47:0/1132179825 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3920009d10 con 0x7f3930147a00 2020-07-18T03:33:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 -- 172.21.15.47:0/1132179825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3930147a00 msgr2=0x7f3930147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 --2- 172.21.15.47:0/1132179825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3930147a00 0x7f3930147e40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f3920002f30 tx=0x7f3920006e90).stop 2020-07-18T03:33:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 -- 172.21.15.47:0/1132179825 shutdown_connections 2020-07-18T03:33:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 --2- 172.21.15.47:0/1132179825 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f39301433e0 0x7f3930143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 --2- 172.21.15.47:0/1132179825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3930147a00 0x7f3930147e40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 -- 172.21.15.47:0/1132179825 >> 172.21.15.47:0/1132179825 conn(0x7f39300b9340 msgr2=0x7f39300b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 -- 172.21.15.47:0/1132179825 shutdown_connections 2020-07-18T03:33:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.655+0000 7f3934e56700 1 -- 172.21.15.47:0/1132179825 wait complete. 2020-07-18T03:33:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 Processor -- start 2020-07-18T03:33:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 -- start start 2020-07-18T03:33:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f39300ad100 0x7f393013d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39301433e0 0x7f393013cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f393012ea10 0x7f393012eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300667a0 con 0x7f39300ad100 2020-07-18T03:33:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3930066a00 con 0x7f39301433e0 2020-07-18T03:33:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f3934e56700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3930066ca0 con 0x7f393012ea10 2020-07-18T03:33:45.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.657+0000 7f392dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39301433e0 0x7f393013cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:45.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39301433e0 0x7f393013cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35162/0 (socket says 172.21.15.47:35162) 2020-07-18T03:33:45.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 -- 172.21.15.47:0/3594489764 learned_addr learned my addr 172.21.15.47:0/3594489764 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:45.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392e59c700 1 --2- 172.21.15.47:0/3594489764 >> v2:172.21.15.18:3300/0 conn(0x7f39300ad100 0x7f393013d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:45.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392ed9d700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f393012ea10 0x7f393012eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 -- 172.21.15.47:0/3594489764 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f393012ea10 msgr2=0x7f393012eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f393012ea10 0x7f393012eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 -- 172.21.15.47:0/3594489764 >> v2:172.21.15.18:3300/0 conn(0x7f39300ad100 msgr2=0x7f393013d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 --2- 172.21.15.47:0/3594489764 >> v2:172.21.15.18:3300/0 conn(0x7f39300ad100 0x7f393013d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 -- 172.21.15.47:0/3594489764 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393013d2d0 con 0x7f39301433e0 2020-07-18T03:33:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f392dd9b700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39301433e0 0x7f393013cde0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f3920002f30 tx=0x7f3920015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3920009850 con 0x7f39301433e0 2020-07-18T03:33:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392001e460 con 0x7f39301433e0 2020-07-18T03:33:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3930065120 con 0x7f39301433e0 2020-07-18T03:33:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f392001e7c0 con 0x7f39301433e0 2020-07-18T03:33:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.658+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39300b2760 con 0x7f39301433e0 2020-07-18T03:33:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.659+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930057450 con 0x7f39301433e0 2020-07-18T03:33:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.659+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3920025030 con 0x7f39301433e0 2020-07-18T03:33:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.659+0000 7f39267fc700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f390c04ff40 0x7f390c052400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.660+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f3920069400 con 0x7f39301433e0 2020-07-18T03:33:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.660+0000 7f392e59c700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f390c04ff40 0x7f390c052400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.660+0000 7f392e59c700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f390c04ff40 0x7f390c052400 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f391800d000 tx=0x7f3918000b80).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.663+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3920041b40 con 0x7f39301433e0 2020-07-18T03:33:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.872+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3930066e10 con 0x7f39301433e0 2020-07-18T03:33:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.873+0000 7f39267fc700 1 -- 172.21.15.47:0/3594489764 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f3920023090 con 0x7f39301433e0 2020-07-18T03:33:45.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:45.875 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:45.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f390c04ff40 msgr2=0x7f390c052400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:45.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f390c04ff40 0x7f390c052400 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f391800d000 tx=0x7f3918000b80).stop 2020-07-18T03:33:45.876 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39301433e0 msgr2=0x7f393013cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:45.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39301433e0 0x7f393013cde0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f3920002f30 tx=0x7f3920015000).stop 2020-07-18T03:33:45.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 shutdown_connections 2020-07-18T03:33:45.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f390c04ff40 0x7f390c052400 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.877 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f393012ea10 0x7f393012eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 --2- 172.21.15.47:0/3594489764 >> v2:172.21.15.18:3300/0 conn(0x7f39300ad100 0x7f393013d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 --2- 172.21.15.47:0/3594489764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39301433e0 0x7f393013cde0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 >> 172.21.15.47:0/3594489764 conn(0x7f39300b9340 msgr2=0x7f39301469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 shutdown_connections 2020-07-18T03:33:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:45.874+0000 7f3934e56700 1 -- 172.21.15.47:0/3594489764 wait complete. 2020-07-18T03:33:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:45.886 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:46.887 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.013+0000 7f78897d5700 1 Processor -- start 2020-07-18T03:33:47.015 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.013+0000 7f78897d5700 1 -- start start 2020-07-18T03:33:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f78897d5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78840b5370 0x7f788405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f78897d5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f78840b41c0 0x7f78840ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f78897d5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f78840b4620 0x7f78840ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f78897d5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78840b5a10 con 0x7f78840b4620 2020-07-18T03:33:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f78897d5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78840b0040 con 0x7f78840b5370 2020-07-18T03:33:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f78897d5700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840b3af0 con 0x7f78840b41c0 2020-07-18T03:33:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f7882ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78840b5370 0x7f788405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f7882ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78840b5370 0x7f788405fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35168/0 (socket says 172.21.15.47:35168) 2020-07-18T03:33:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.014+0000 7f7882ffd700 1 -- 172.21.15.47:0/3898628394 learned_addr learned my addr 172.21.15.47:0/3898628394 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78837fe700 1 --2- 172.21.15.47:0/3898628394 >> v2:172.21.15.18:3300/0 conn(0x7f78840b4620 0x7f78840ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78827fc700 1 --2- 172.21.15.47:0/3898628394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f78840b41c0 0x7f78840ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7882ffd700 1 -- 172.21.15.47:0/3898628394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f78840b41c0 msgr2=0x7f78840ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7882ffd700 1 --2- 172.21.15.47:0/3898628394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f78840b41c0 0x7f78840ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7882ffd700 1 -- 172.21.15.47:0/3898628394 >> v2:172.21.15.18:3300/0 conn(0x7f78840b4620 msgr2=0x7f78840ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7882ffd700 1 --2- 172.21.15.47:0/3898628394 >> v2:172.21.15.18:3300/0 conn(0x7f78840b4620 0x7f78840ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7882ffd700 1 -- 172.21.15.47:0/3898628394 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840adcb0 con 0x7f78840b5370 2020-07-18T03:33:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7882ffd700 1 --2- 172.21.15.47:0/3898628394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78840b5370 0x7f788405fff0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f7878002f30 tx=0x7f7878006f20).ready entity=mon.1 client_cookie=7263b308157095e4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7881ffb700 1 -- 172.21.15.47:0/3898628394 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787800cb70 con 0x7f78840b5370 2020-07-18T03:33:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7881ffb700 1 -- 172.21.15.47:0/3898628394 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787800ccd0 con 0x7f78840b5370 2020-07-18T03:33:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f7881ffb700 1 -- 172.21.15.47:0/3898628394 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7878010600 con 0x7f78840b5370 2020-07-18T03:33:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 -- 172.21.15.47:0/3898628394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78840b5370 msgr2=0x7f788405fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 --2- 172.21.15.47:0/3898628394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78840b5370 0x7f788405fff0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f7878002f30 tx=0x7f7878006f20).stop 2020-07-18T03:33:47.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 -- 172.21.15.47:0/3898628394 shutdown_connections 2020-07-18T03:33:47.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 --2- 172.21.15.47:0/3898628394 >> v2:172.21.15.18:3300/0 conn(0x7f78840b4620 0x7f78840ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 --2- 172.21.15.47:0/3898628394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78840b5370 0x7f788405fff0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 --2- 172.21.15.47:0/3898628394 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f78840b41c0 0x7f78840ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.022 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 -- 172.21.15.47:0/3898628394 >> 172.21.15.47:0/3898628394 conn(0x7f78840b9340 msgr2=0x7f78840b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:47.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 -- 172.21.15.47:0/3898628394 shutdown_connections 2020-07-18T03:33:47.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.015+0000 7f78897d5700 1 -- 172.21.15.47:0/3898628394 wait complete. 2020-07-18T03:33:47.025 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.017+0000 7f78897d5700 1 Processor -- start 2020-07-18T03:33:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.017+0000 7f78897d5700 1 -- start start 2020-07-18T03:33:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78897d5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f78840b41c0 0x7f7884140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78897d5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7884154db0 0x7f78841128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:47.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78897d5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7884151ce0 con 0x7f7884063470 2020-07-18T03:33:47.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78897d5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7884151b00 con 0x7f78840b41c0 2020-07-18T03:33:47.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78897d5700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7884151f80 con 0x7f7884154db0 2020-07-18T03:33:47.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78827fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7884063470 0x7f78841546a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44496/0 (socket says 172.21.15.47:44496) 2020-07-18T03:33:47.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78827fc700 1 -- 172.21.15.47:0/1382434463 learned_addr learned my addr 172.21.15.47:0/1382434463 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:47.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f7882ffd700 1 --2- 172.21.15.47:0/1382434463 >> v2:172.21.15.18:3300/0 conn(0x7f78840b41c0 0x7f7884140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:47.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f78837fe700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7884154db0 0x7f78841128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:47.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998949521 0 0) 0x7f7884151ce0 con 0x7f7884063470 2020-07-18T03:33:47.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7860003240 con 0x7f7884063470 2020-07-18T03:33:47.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f7882ffd700 1 -- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7884154db0 msgr2=0x7f78841128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:47.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f7882ffd700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7884154db0 0x7f78841128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f7882ffd700 1 -- 172.21.15.47:0/1382434463 >> v1:172.21.15.47:6789/0 conn(0x7f7884063470 legacy=0x7f78841546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f7882ffd700 1 -- 172.21.15.47:0/1382434463 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884141430 con 0x7f78840b41c0 2020-07-18T03:33:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.018+0000 7f7882ffd700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78840b41c0 0x7f7884140f40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f7878002f30 tx=0x7f7878017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:47.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.019+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7878019870 con 0x7f78840b41c0 2020-07-18T03:33:47.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.019+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78780106b0 con 0x7f78840b41c0 2020-07-18T03:33:47.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.019+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7884151300 con 0x7f78840b41c0 2020-07-18T03:33:47.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.019+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7878019bc0 con 0x7f78840b41c0 2020-07-18T03:33:47.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.019+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7884155400 con 0x7f78840b41c0 2020-07-18T03:33:47.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.020+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7884151490 con 0x7f78840b41c0 2020-07-18T03:33:47.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.020+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f787800e040 con 0x7f78840b41c0 2020-07-18T03:33:47.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.020+0000 7f786affd700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f78600503c0 0x7f7860052880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:47.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.020+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7878068b40 con 0x7f78840b41c0 2020-07-18T03:33:47.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.020+0000 7f78827fc700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f78600503c0 0x7f7860052880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:47.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.021+0000 7f78827fc700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f78600503c0 0x7f7860052880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f787400a000 tx=0x7f7874006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:47.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.024+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7878041200 con 0x7f78840b41c0 2020-07-18T03:33:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.243+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7884057450 con 0x7f78840b41c0 2020-07-18T03:33:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.244+0000 7f786affd700 1 -- 172.21.15.47:0/1382434463 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f7878014020 con 0x7f78840b41c0 2020-07-18T03:33:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:47.246 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f78600503c0 msgr2=0x7f7860052880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f78600503c0 0x7f7860052880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f787400a000 tx=0x7f7874006dd0).stop 2020-07-18T03:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78840b41c0 msgr2=0x7f7884140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78840b41c0 0x7f7884140f40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f7878002f30 tx=0x7f7878017000).stop 2020-07-18T03:33:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 shutdown_connections 2020-07-18T03:33:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7884154db0 0x7f78841128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78840b41c0 0x7f7884140f40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 --2- 172.21.15.47:0/1382434463 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f78600503c0 0x7f7860052880 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:47.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 >> 172.21.15.47:0/1382434463 conn(0x7f78840b9340 msgr2=0x7f7884141ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:47.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 shutdown_connections 2020-07-18T03:33:47.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:47.246+0000 7f78897d5700 1 -- 172.21.15.47:0/1382434463 wait complete. 2020-07-18T03:33:47.250 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:47.257 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:48.258 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.384+0000 7f0346c60700 1 Processor -- start 2020-07-18T03:33:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.384+0000 7f0346c60700 1 -- start start 2020-07-18T03:33:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f0346c60700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f0340147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f0346c60700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f03401433e0 0x7f0340143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f0346c60700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0340147330 con 0x7f03400ad100 2020-07-18T03:33:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f0346c60700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03400ae530 con 0x7f03401433e0 2020-07-18T03:33:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f0346c60700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03400b57e0 con 0x7f0340147a00 2020-07-18T03:33:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f03449fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f03400ad100 0x7f03400b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44502/0 (socket says 172.21.15.47:44502) 2020-07-18T03:33:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.385+0000 7f03449fc700 1 -- 172.21.15.47:0/1228592407 learned_addr learned my addr 172.21.15.47:0/1228592407 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:48.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03451fd700 1 --2- 172.21.15.47:0/1228592407 >> v2:172.21.15.18:3300/0 conn(0x7f03401433e0 0x7f0340143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033ffff700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f0340147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046456853 0 0) 0x7f0340147330 con 0x7f03400ad100 2020-07-18T03:33:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f032c003240 con 0x7f03400ad100 2020-07-18T03:33:48.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1760332797 0 0) 0x7f032c003240 con 0x7f03400ad100 2020-07-18T03:33:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 msgr2=0x7f0340147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f0340147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 >> v2:172.21.15.18:3300/0 conn(0x7f03401433e0 msgr2=0x7f0340143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 --2- 172.21.15.47:0/1228592407 >> v2:172.21.15.18:3300/0 conn(0x7f03401433e0 0x7f0340143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400b5a80 con 0x7f03400ad100 2020-07-18T03:33:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f03300016d0 con 0x7f03400ad100 2020-07-18T03:33:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 >> v1:172.21.15.47:6789/0 conn(0x7f03400ad100 legacy=0x7f03400b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032c003730 0x7f032c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f032c0044e0 0x7f032c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f032c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400ae530 con 0x7f032c0044e0 2020-07-18T03:33:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400b57e0 con 0x7f032c003730 2020-07-18T03:33:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c004270 con 0x7f0340147a00 2020-07-18T03:33:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032c003730 0x7f032c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f033ffff700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f032c0044e0 0x7f032c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03451fd700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f032c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 -- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 msgr2=0x7f032c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f032c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 -- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f032c0044e0 msgr2=0x7f032c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f032c0044e0 0x7f032c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 -- 172.21.15.47:0/1228592407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c003240 con 0x7f032c003730 2020-07-18T03:33:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03451fd700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0340147a00 0x7f032c004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:33:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.386+0000 7f03449fc700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032c003730 0x7f032c003b50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f03300013e0 tx=0x7f03400b03e0).ready entity=mon.1 client_cookie=4669d05380324176 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033000ce00 con 0x7f032c003730 2020-07-18T03:33:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03300072e0 con 0x7f032c003730 2020-07-18T03:33:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f033f7fe700 1 -- 172.21.15.47:0/1228592407 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03300057e0 con 0x7f032c003730 2020-07-18T03:33:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f0346c60700 1 -- 172.21.15.47:0/1228592407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032c003730 msgr2=0x7f032c003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f0346c60700 1 --2- 172.21.15.47:0/1228592407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032c003730 0x7f032c003b50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f03300013e0 tx=0x7f03400b03e0).stop 2020-07-18T03:33:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f03451fd700 1 -- 172.21.15.47:0/1228592407 reap_dead start 2020-07-18T03:33:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f0346c60700 1 -- 172.21.15.47:0/1228592407 shutdown_connections 2020-07-18T03:33:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f0346c60700 1 -- 172.21.15.47:0/1228592407 >> 172.21.15.47:0/1228592407 conn(0x7f03400b9340 msgr2=0x7f03400b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f0346c60700 1 -- 172.21.15.47:0/1228592407 shutdown_connections 2020-07-18T03:33:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.387+0000 7f0346c60700 1 -- 172.21.15.47:0/1228592407 wait complete. 2020-07-18T03:33:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.388+0000 7f0346c60700 1 Processor -- start 2020-07-18T03:33:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.388+0000 7f0346c60700 1 -- start start 2020-07-18T03:33:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f0346c60700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f034013e450 0x7f0340146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f0346c60700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f034013e830 0x7f034013ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f0346c60700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0340142820 con 0x7f0340061b80 2020-07-18T03:33:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f0346c60700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0340142610 con 0x7f034013e450 2020-07-18T03:33:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f0346c60700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0340142270 con 0x7f034013e830 2020-07-18T03:33:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f03449fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f034013e450 0x7f0340146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f03449fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f034013e450 0x7f0340146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35194/0 (socket says 172.21.15.47:35194) 2020-07-18T03:33:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f03449fc700 1 -- 172.21.15.47:0/56684451 learned_addr learned my addr 172.21.15.47:0/56684451 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f033ffff700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f034013e830 0x7f034013ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.389+0000 7f03449fc700 1 -- 172.21.15.47:0/56684451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f034013e830 msgr2=0x7f034013ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f03449fc700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f034013e830 0x7f034013ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f03449fc700 1 -- 172.21.15.47:0/56684451 >> v1:172.21.15.18:6789/0 conn(0x7f0340061b80 legacy=0x7f03400b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f03449fc700 1 -- 172.21.15.47:0/56684451 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0340146d80 con 0x7f034013e450 2020-07-18T03:33:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f03449fc700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034013e450 0x7f0340146890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f0340145c00 tx=0x7f0330015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0330013850 con 0x7f034013e450 2020-07-18T03:33:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03300139b0 con 0x7f034013e450 2020-07-18T03:33:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0340142cc0 con 0x7f034013e450 2020-07-18T03:33:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0330013d10 con 0x7f034013e450 2020-07-18T03:33:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.390+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034013b2e0 con 0x7f034013e450 2020-07-18T03:33:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.391+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034013b500 con 0x7f034013e450 2020-07-18T03:33:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.391+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0330026020 con 0x7f034013e450 2020-07-18T03:33:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.391+0000 7f033cff9700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0318050060 0x7f0318052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.391+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f03300696b0 con 0x7f034013e450 2020-07-18T03:33:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.391+0000 7f033ffff700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0318050060 0x7f0318052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.392+0000 7f033ffff700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0318050060 0x7f0318052520 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f0334008000 tx=0x7f0334008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.395+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0330041df0 con 0x7f034013e450 2020-07-18T03:33:48.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.609+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f034013b500 con 0x7f034013e450 2020-07-18T03:33:48.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.609+0000 7f033cff9700 1 -- 172.21.15.47:0/56684451 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f0330045410 con 0x7f034013e450 2020-07-18T03:33:48.611 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:48.611 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:48.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0318050060 msgr2=0x7f0318052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0318050060 0x7f0318052520 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f0334008000 tx=0x7f0334008040).stop 2020-07-18T03:33:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034013e450 msgr2=0x7f0340146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034013e450 0x7f0340146890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f0340145c00 tx=0x7f0330015000).stop 2020-07-18T03:33:48.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 shutdown_connections 2020-07-18T03:33:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f0318050060 0x7f0318052520 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034013e450 0x7f0340146890 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 --2- 172.21.15.47:0/56684451 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f034013e830 0x7f034013ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 >> 172.21.15.47:0/56684451 conn(0x7f03400b9340 msgr2=0x7f034005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 shutdown_connections 2020-07-18T03:33:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:48.611+0000 7f0346c60700 1 -- 172.21.15.47:0/56684451 wait complete. 2020-07-18T03:33:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:48.622 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:49.623 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.752+0000 7f9504274700 1 Processor -- start 2020-07-18T03:33:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.752+0000 7f9504274700 1 -- start start 2020-07-18T03:33:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.752+0000 7f9504274700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f94fc0ad100 0x7f94fc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.752+0000 7f9504274700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94fc147a00 0x7f94fc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc147330 con 0x7f94fc05f5a0 2020-07-18T03:33:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc0ae530 con 0x7f94fc0ad100 2020-07-18T03:33:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc0b57e0 con 0x7f94fc147a00 2020-07-18T03:33:49.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94fc147a00 0x7f94fc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94fc147a00 0x7f94fc147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35204/0 (socket says 172.21.15.47:35204) 2020-07-18T03:33:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 -- 172.21.15.47:0/3318489838 learned_addr learned my addr 172.21.15.47:0/3318489838 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9502010700 1 --2- 172.21.15.47:0/3318489838 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0ad100 0x7f94fc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 -- 172.21.15.47:0/3318489838 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0ad100 msgr2=0x7f94fc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 --2- 172.21.15.47:0/3318489838 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0ad100 0x7f94fc0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 -- 172.21.15.47:0/3318489838 >> v1:172.21.15.18:6789/0 conn(0x7f94fc05f5a0 legacy=0x7f94fc05f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 -- 172.21.15.47:0/3318489838 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0b5a80 con 0x7f94fc147a00 2020-07-18T03:33:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950180f700 1 --2- 172.21.15.47:0/3318489838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc147a00 0x7f94fc147e40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f94fc05ca30 tx=0x7f94f0006e90).ready entity=mon.1 client_cookie=f4de50848b7ae11c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950100e700 1 -- 172.21.15.47:0/3318489838 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f0009930 con 0x7f94fc147a00 2020-07-18T03:33:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950100e700 1 -- 172.21.15.47:0/3318489838 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f0009a90 con 0x7f94fc147a00 2020-07-18T03:33:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f950100e700 1 -- 172.21.15.47:0/3318489838 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f0009e30 con 0x7f94fc147a00 2020-07-18T03:33:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 -- 172.21.15.47:0/3318489838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc147a00 msgr2=0x7f94fc147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 --2- 172.21.15.47:0/3318489838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc147a00 0x7f94fc147e40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f94fc05ca30 tx=0x7f94f0006e90).stop 2020-07-18T03:33:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 -- 172.21.15.47:0/3318489838 shutdown_connections 2020-07-18T03:33:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 --2- 172.21.15.47:0/3318489838 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0ad100 0x7f94fc0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.753+0000 7f9504274700 1 --2- 172.21.15.47:0/3318489838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc147a00 0x7f94fc147e40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.754+0000 7f9504274700 1 -- 172.21.15.47:0/3318489838 >> 172.21.15.47:0/3318489838 conn(0x7f94fc0b9340 msgr2=0x7f94fc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.754+0000 7f9504274700 1 -- 172.21.15.47:0/3318489838 shutdown_connections 2020-07-18T03:33:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.754+0000 7f9504274700 1 -- 172.21.15.47:0/3318489838 wait complete. 2020-07-18T03:33:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.755+0000 7f9504274700 1 Processor -- start 2020-07-18T03:33:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.755+0000 7f9504274700 1 -- start start 2020-07-18T03:33:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f9504274700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f94fc0663e0 0x7f94fc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f9504274700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc065800 con 0x7f94fc05f5a0 2020-07-18T03:33:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f9504274700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc064350 con 0x7f94fc0ad100 2020-07-18T03:33:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f9504274700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc061bf0 con 0x7f94fc0663e0 2020-07-18T03:33:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f950180f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f94fc0ad100 0x7f94fc139f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44528/0 (socket says 172.21.15.47:44528) 2020-07-18T03:33:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f950180f700 1 -- 172.21.15.47:0/4147381647 learned_addr learned my addr 172.21.15.47:0/4147381647 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:49.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f9502811700 1 --2- 172.21.15.47:0/4147381647 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0663e0 0x7f94fc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426190881 0 0) 0x7f94fc064350 con 0x7f94fc0ad100 2020-07-18T03:33:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d4003240 con 0x7f94fc0ad100 2020-07-18T03:33:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1283496095 0 0) 0x7f94d4003240 con 0x7f94fc0ad100 2020-07-18T03:33:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94fc064350 con 0x7f94fc0ad100 2020-07-18T03:33:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f94f0009930 con 0x7f94fc0ad100 2020-07-18T03:33:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0663e0 msgr2=0x7f94fc15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 --2- 172.21.15.47:0/4147381647 >> v2:172.21.15.18:3300/0 conn(0x7f94fc0663e0 0x7f94fc15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 >> v1:172.21.15.47:6789/0 conn(0x7f94fc0ad100 legacy=0x7f94fc139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 >> v1:172.21.15.18:6789/0 conn(0x7f94fc05f5a0 legacy=0x7f94fc13bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d4003800 0x7f94d4003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94d40044e0 0x7f94d4004940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94d4004e30 0x7f94d400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc061bf0 con 0x7f94d40044e0 2020-07-18T03:33:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4004250 con 0x7f94d4003800 2020-07-18T03:33:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.756+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d400bc20 con 0x7f94d4004e30 2020-07-18T03:33:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d4003800 0x7f94d4003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f950180f700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94d40044e0 0x7f94d4004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502811700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94d4004e30 0x7f94d400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 -- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94d4004e30 msgr2=0x7f94d400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f94d4004e30 0x7f94d400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 -- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94d40044e0 msgr2=0x7f94d4004940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94d40044e0 0x7f94d4004940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc066880 con 0x7f94d4003800 2020-07-18T03:33:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc0b1530 con 0x7f94d4003800 2020-07-18T03:33:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502811700 1 -- 172.21.15.47:0/4147381647 reap_dead start 2020-07-18T03:33:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc0660b0 con 0x7f94d4003800 2020-07-18T03:33:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f9502010700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d4003800 0x7f94d4003be0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f94f8006ab0 tx=0x7f94f8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f8007980 con 0x7f94d4003800 2020-07-18T03:33:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f8007ae0 con 0x7f94d4003800 2020-07-18T03:33:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.757+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f8007e40 con 0x7f94d4003800 2020-07-18T03:33:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.758+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc057450 con 0x7f94d4003800 2020-07-18T03:33:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.758+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f94f801f060 con 0x7f94d4003800 2020-07-18T03:33:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.759+0000 7f94ee7fc700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f94d4053ce0 0x7f94d40561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.759+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f94f80698a0 con 0x7f94d4003800 2020-07-18T03:33:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.759+0000 7f950180f700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f94d4053ce0 0x7f94d40561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.759+0000 7f950180f700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f94d4053ce0 0x7f94d40561a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f94f001c000 tx=0x7f94f0015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.762+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f94f8041f60 con 0x7f94d4003800 2020-07-18T03:33:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.975+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94fc143e80 con 0x7f94d4003800 2020-07-18T03:33:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.975+0000 7f94ee7fc700 1 -- 172.21.15.47:0/4147381647 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f94f8045580 con 0x7f94d4003800 2020-07-18T03:33:49.977 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:49.977 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f94d4053ce0 msgr2=0x7f94d40561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f94d4053ce0 0x7f94d40561a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f94f001c000 tx=0x7f94f0015000).stop 2020-07-18T03:33:49.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d4003800 msgr2=0x7f94d4003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:49.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d4003800 0x7f94d4003be0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f94f8006ab0 tx=0x7f94f8008040).stop 2020-07-18T03:33:49.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 shutdown_connections 2020-07-18T03:33:49.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f94d4053ce0 0x7f94d40561a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 --2- 172.21.15.47:0/4147381647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d4003800 0x7f94d4003be0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 >> 172.21.15.47:0/4147381647 conn(0x7f94fc0b9340 msgr2=0x7f94fc1431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 shutdown_connections 2020-07-18T03:33:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:49.977+0000 7f9504274700 1 -- 172.21.15.47:0/4147381647 wait complete. 2020-07-18T03:33:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:49.988 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:50.493 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:33:50.497 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:33:50.989 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.115+0000 7fc55fc2a700 1 Processor -- start 2020-07-18T03:33:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.115+0000 7fc55fc2a700 1 -- start start 2020-07-18T03:33:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55fc2a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc5580ad9e0 0x7fc5581428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55fc2a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5580b3ac0 con 0x7fc5580b5370 2020-07-18T03:33:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55fc2a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5580b3d20 con 0x7fc5580b41c0 2020-07-18T03:33:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55fc2a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5580b5a10 con 0x7fc5580ad9e0 2020-07-18T03:33:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55d1c5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc5580b41c0 0x7fc5580ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44542/0 (socket says 172.21.15.47:44542) 2020-07-18T03:33:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55d1c5700 1 -- 172.21.15.47:0/2418828446 learned_addr learned my addr 172.21.15.47:0/2418828446 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.116+0000 7fc55e1c7700 1 --2- 172.21.15.47:0/2418828446 >> v2:172.21.15.18:3300/0 conn(0x7fc5580ad9e0 0x7fc5581428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644713168 0 0) 0x7fc5580b3d20 con 0x7fc5580b41c0 2020-07-18T03:33:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc544003240 con 0x7fc5580b41c0 2020-07-18T03:33:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670181409 0 0) 0x7fc5580b3ac0 con 0x7fc5580b5370 2020-07-18T03:33:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5580b3d20 con 0x7fc5580b5370 2020-07-18T03:33:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1861615315 0 0) 0x7fc544003240 con 0x7fc5580b41c0 2020-07-18T03:33:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 >> v2:172.21.15.18:3300/0 conn(0x7fc5580ad9e0 msgr2=0x7fc5581428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 --2- 172.21.15.47:0/2418828446 >> v2:172.21.15.18:3300/0 conn(0x7fc5580ad9e0 0x7fc5581428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 >> v1:172.21.15.18:6789/0 conn(0x7fc5580b5370 legacy=0x7fc55805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580b5750 con 0x7fc5580b41c0 2020-07-18T03:33:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fc54c0016d0 con 0x7fc5580b41c0 2020-07-18T03:33:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 >> v1:172.21.15.47:6789/0 conn(0x7fc5580b41c0 legacy=0x7fc5580ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc544003910 0x7fc544003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5440044e0 0x7fc544004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc544004e50 0x7fc54400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5580b5a10 con 0x7fc544004e50 2020-07-18T03:33:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54400bcb0 con 0x7fc544003910 2020-07-18T03:33:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54400be20 con 0x7fc5440044e0 2020-07-18T03:33:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55d9c6700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc544003910 0x7fc544003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55d1c5700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5440044e0 0x7fc544004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55e1c7700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc544004e50 0x7fc54400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55d9c6700 1 -- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5440044e0 msgr2=0x7fc544004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55d9c6700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc5440044e0 0x7fc544004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55d9c6700 1 -- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc544004e50 msgr2=0x7fc54400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.117+0000 7fc55d9c6700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc544004e50 0x7fc54400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55e1c7700 1 -- 172.21.15.47:0/2418828446 reap_dead start 2020-07-18T03:33:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55d9c6700 1 -- 172.21.15.47:0/2418828446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544003240 con 0x7fc544003910 2020-07-18T03:33:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55d9c6700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc544003910 0x7fc544003d10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fc544004450 tx=0x7fc550007ef0).ready entity=mon.1 client_cookie=930e35fa47bd2c4c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc550001690 con 0x7fc544003910 2020-07-18T03:33:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55000cb80 con 0x7fc544003910 2020-07-18T03:33:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55c9c4700 1 -- 172.21.15.47:0/2418828446 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc550013410 con 0x7fc544003910 2020-07-18T03:33:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 -- 172.21.15.47:0/2418828446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc544003910 msgr2=0x7fc544003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc544003910 0x7fc544003d10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fc544004450 tx=0x7fc550007ef0).stop 2020-07-18T03:33:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 -- 172.21.15.47:0/2418828446 shutdown_connections 2020-07-18T03:33:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 --2- 172.21.15.47:0/2418828446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc544003910 0x7fc544003d10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 -- 172.21.15.47:0/2418828446 >> 172.21.15.47:0/2418828446 conn(0x7fc5580b9340 msgr2=0x7fc5580b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 -- 172.21.15.47:0/2418828446 shutdown_connections 2020-07-18T03:33:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.118+0000 7fc55fc2a700 1 -- 172.21.15.47:0/2418828446 wait complete. 2020-07-18T03:33:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.120+0000 7fc55fc2a700 1 Processor -- start 2020-07-18T03:33:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.120+0000 7fc55fc2a700 1 -- start start 2020-07-18T03:33:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.120+0000 7fc55fc2a700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc558156540 0x7fc558156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.120+0000 7fc55fc2a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc558113040 con 0x7fc558141520 2020-07-18T03:33:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.120+0000 7fc55fc2a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5581132e0 con 0x7fc5581144b0 2020-07-18T03:33:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.120+0000 7fc55fc2a700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc558113580 con 0x7fc558156540 2020-07-18T03:33:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d1c5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc5581144b0 0x7fc558140e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44554/0 (socket says 172.21.15.47:44554) 2020-07-18T03:33:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d1c5700 1 -- 172.21.15.47:0/1590661757 learned_addr learned my addr 172.21.15.47:0/1590661757 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d9c6700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc558156540 0x7fc558156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116540802 0 0) 0x7fc5581132e0 con 0x7fc5581144b0 2020-07-18T03:33:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc530003240 con 0x7fc5581144b0 2020-07-18T03:33:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536954560 0 0) 0x7fc558113040 con 0x7fc558141520 2020-07-18T03:33:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5581132e0 con 0x7fc558141520 2020-07-18T03:33:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d9c6700 1 -- 172.21.15.47:0/1590661757 >> v1:172.21.15.47:6789/0 conn(0x7fc5581144b0 legacy=0x7fc558140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d9c6700 1 -- 172.21.15.47:0/1590661757 >> v1:172.21.15.18:6789/0 conn(0x7fc558141520 legacy=0x7fc5581148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d9c6700 1 -- 172.21.15.47:0/1590661757 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc558112d30 con 0x7fc558156540 2020-07-18T03:33:51.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55d9c6700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc558156540 0x7fc558156920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc544004450 tx=0x7fc550016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:51.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc55001f670 con 0x7fc558156540 2020-07-18T03:33:51.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55001f7d0 con 0x7fc558156540 2020-07-18T03:33:51.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc558113a20 con 0x7fc558156540 2020-07-18T03:33:51.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.121+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc55001fb30 con 0x7fc558156540 2020-07-18T03:33:51.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.122+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc558156fd0 con 0x7fc558156540 2020-07-18T03:33:51.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.123+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc558057450 con 0x7fc558156540 2020-07-18T03:33:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.123+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc550010030 con 0x7fc558156540 2020-07-18T03:33:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.123+0000 7fc555ffb700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc530050450 0x7fc530052910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.123+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc550069ef0 con 0x7fc558156540 2020-07-18T03:33:51.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.123+0000 7fc55d1c5700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc530050450 0x7fc530052910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:51.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.124+0000 7fc55d1c5700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc530050450 0x7fc530052910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fc54c00d000 tx=0x7fc5580b3eb0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:51.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.127+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc550042510 con 0x7fc558156540 2020-07-18T03:33:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.334+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5580b01a0 con 0x7fc558156540 2020-07-18T03:33:51.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.335+0000 7fc555ffb700 1 -- 172.21.15.47:0/1590661757 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fc550045b30 con 0x7fc558156540 2020-07-18T03:33:51.337 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:51.338 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc530050450 msgr2=0x7fc530052910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc530050450 0x7fc530052910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fc54c00d000 tx=0x7fc5580b3eb0).stop 2020-07-18T03:33:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc558156540 msgr2=0x7fc558156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc558156540 0x7fc558156920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc544004450 tx=0x7fc550016000).stop 2020-07-18T03:33:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 shutdown_connections 2020-07-18T03:33:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc558156540 0x7fc558156920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 --2- 172.21.15.47:0/1590661757 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc530050450 0x7fc530052910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 >> 172.21.15.47:0/1590661757 conn(0x7fc5580b9340 msgr2=0x7fc5580b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 shutdown_connections 2020-07-18T03:33:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:51.337+0000 7fc55fc2a700 1 -- 172.21.15.47:0/1590661757 wait complete. 2020-07-18T03:33:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:51.348 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:52.350 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.478+0000 7fec28bc1700 1 Processor -- start 2020-07-18T03:33:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.478+0000 7fec28bc1700 1 -- start start 2020-07-18T03:33:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.479+0000 7fec28bc1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fec240ad100 0x7fec240b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.479+0000 7fec28bc1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec24147a00 0x7fec24147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.479+0000 7fec28bc1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec24147330 con 0x7fec2405f5a0 2020-07-18T03:33:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.479+0000 7fec28bc1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240ae530 con 0x7fec240ad100 2020-07-18T03:33:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.479+0000 7fec28bc1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240b57e0 con 0x7fec24147a00 2020-07-18T03:33:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.479+0000 7fec21d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec24147a00 0x7fec24147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec24147a00 0x7fec24147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35240/0 (socket says 172.21.15.47:35240) 2020-07-18T03:33:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 -- 172.21.15.47:0/3077719078 learned_addr learned my addr 172.21.15.47:0/3077719078 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec2259c700 1 --2- 172.21.15.47:0/3077719078 >> v2:172.21.15.18:3300/0 conn(0x7fec240ad100 0x7fec240b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 -- 172.21.15.47:0/3077719078 >> v2:172.21.15.18:3300/0 conn(0x7fec240ad100 msgr2=0x7fec240b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 --2- 172.21.15.47:0/3077719078 >> v2:172.21.15.18:3300/0 conn(0x7fec240ad100 0x7fec240b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 -- 172.21.15.47:0/3077719078 >> v1:172.21.15.18:6789/0 conn(0x7fec2405f5a0 legacy=0x7fec2405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 -- 172.21.15.47:0/3077719078 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240b5a80 con 0x7fec24147a00 2020-07-18T03:33:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec21d9b700 1 --2- 172.21.15.47:0/3077719078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec24147a00 0x7fec24147e40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fec14002f30 tx=0x7fec14006e90).ready entity=mon.1 client_cookie=bea24389a379f4a0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec2159a700 1 -- 172.21.15.47:0/3077719078 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec140099c0 con 0x7fec24147a00 2020-07-18T03:33:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec2159a700 1 -- 172.21.15.47:0/3077719078 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec14009b20 con 0x7fec24147a00 2020-07-18T03:33:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec2159a700 1 -- 172.21.15.47:0/3077719078 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec14009e80 con 0x7fec24147a00 2020-07-18T03:33:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 -- 172.21.15.47:0/3077719078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec24147a00 msgr2=0x7fec24147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3077719078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec24147a00 0x7fec24147e40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fec14002f30 tx=0x7fec14006e90).stop 2020-07-18T03:33:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 -- 172.21.15.47:0/3077719078 shutdown_connections 2020-07-18T03:33:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3077719078 >> v2:172.21.15.18:3300/0 conn(0x7fec240ad100 0x7fec240b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3077719078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec24147a00 0x7fec24147e40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 -- 172.21.15.47:0/3077719078 >> 172.21.15.47:0/3077719078 conn(0x7fec240b9340 msgr2=0x7fec240b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 -- 172.21.15.47:0/3077719078 shutdown_connections 2020-07-18T03:33:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.480+0000 7fec28bc1700 1 -- 172.21.15.47:0/3077719078 wait complete. 2020-07-18T03:33:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.482+0000 7fec28bc1700 1 Processor -- start 2020-07-18T03:33:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.482+0000 7fec28bc1700 1 -- start start 2020-07-18T03:33:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec28bc1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fec2405f5a0 0x7fec24066070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec28bc1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec240ad100 0x7fec241558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec28bc1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec24155440 con 0x7fec2413d070 2020-07-18T03:33:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec28bc1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec24155160 con 0x7fec2405f5a0 2020-07-18T03:33:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec28bc1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec24066560 con 0x7fec240ad100 2020-07-18T03:33:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec22d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fec2413d070 0x7fec24137520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44570/0 (socket says 172.21.15.47:44570) 2020-07-18T03:33:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec22d9d700 1 -- 172.21.15.47:0/3297609988 learned_addr learned my addr 172.21.15.47:0/3297609988 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec2259c700 1 --2- 172.21.15.47:0/3297609988 >> v2:172.21.15.18:3300/0 conn(0x7fec2405f5a0 0x7fec24066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec21d9b700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec240ad100 0x7fec241558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859186441 0 0) 0x7fec24155440 con 0x7fec2413d070 2020-07-18T03:33:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec00003240 con 0x7fec2413d070 2020-07-18T03:33:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec2259c700 1 -- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec240ad100 msgr2=0x7fec241558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec2259c700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec240ad100 0x7fec241558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec2259c700 1 -- 172.21.15.47:0/3297609988 >> v1:172.21.15.47:6789/0 conn(0x7fec2413d070 legacy=0x7fec24137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec2259c700 1 -- 172.21.15.47:0/3297609988 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24138a80 con 0x7fec2405f5a0 2020-07-18T03:33:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec21d9b700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec240ad100 0x7fec241558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.483+0000 7fec2259c700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec2405f5a0 0x7fec24066070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fec0c00a000 tx=0x7fec0c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.484+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c00f830 con 0x7fec2405f5a0 2020-07-18T03:33:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.484+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec24137c30 con 0x7fec2405f5a0 2020-07-18T03:33:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.484+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00f990 con 0x7fec2405f5a0 2020-07-18T03:33:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.484+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c00fcf0 con 0x7fec2405f5a0 2020-07-18T03:33:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.484+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec24156010 con 0x7fec2405f5a0 2020-07-18T03:33:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.485+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec241552a0 con 0x7fec2405f5a0 2020-07-18T03:33:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.485+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fec0c01a030 con 0x7fec2405f5a0 2020-07-18T03:33:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.485+0000 7fec1a7fc700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec00050360 0x7fec00052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.485+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fec0c067b80 con 0x7fec2405f5a0 2020-07-18T03:33:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.485+0000 7fec21d9b700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec00050360 0x7fec00052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.486+0000 7fec21d9b700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec00050360 0x7fec00052820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fec1401c000 tx=0x7fec14015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.489+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fec241552a0 con 0x7fec2405f5a0 2020-07-18T03:33:52.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.697+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec241552a0 con 0x7fec2405f5a0 2020-07-18T03:33:52.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.698+0000 7fec1a7fc700 1 -- 172.21.15.47:0/3297609988 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fec241552a0 con 0x7fec2405f5a0 2020-07-18T03:33:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:52.699 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec00050360 msgr2=0x7fec00052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:52.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec00050360 0x7fec00052820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fec1401c000 tx=0x7fec14015000).stop 2020-07-18T03:33:52.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec2405f5a0 msgr2=0x7fec24066070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:52.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec2405f5a0 0x7fec24066070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fec0c00a000 tx=0x7fec0c008040).stop 2020-07-18T03:33:52.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 shutdown_connections 2020-07-18T03:33:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec00050360 0x7fec00052820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec2405f5a0 0x7fec24066070 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 --2- 172.21.15.47:0/3297609988 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec240ad100 0x7fec241558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 >> 172.21.15.47:0/3297609988 conn(0x7fec240b9340 msgr2=0x7fec241396d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 shutdown_connections 2020-07-18T03:33:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:52.699+0000 7fec28bc1700 1 -- 172.21.15.47:0/3297609988 wait complete. 2020-07-18T03:33:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:52.710 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:53.712 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.838+0000 7ff0d67cf700 1 Processor -- start 2020-07-18T03:33:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.838+0000 7ff0d67cf700 1 -- start start 2020-07-18T03:33:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0d67cf700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff0d0147070 0x7ff0d0062050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0d67cf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0d01474c0 0x7ff0d0062540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0d67cf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d0062f30 con 0x7ff0d0060080 2020-07-18T03:33:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0d67cf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d0062a30 con 0x7ff0d01474c0 2020-07-18T03:33:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0d67cf700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0062c90 con 0x7ff0d0147070 2020-07-18T03:33:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0d4d6c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0d01474c0 0x7ff0d0062540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0cffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0d0060080 0x7ff0d00b6b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44574/0 (socket says 172.21.15.47:44574) 2020-07-18T03:33:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.839+0000 7ff0cffff700 1 -- 172.21.15.47:0/123617851 learned_addr learned my addr 172.21.15.47:0/123617851 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0cf7fe700 1 --2- 172.21.15.47:0/123617851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff0d0147070 0x7ff0d0062050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/123617851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff0d0147070 msgr2=0x7ff0d0062050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d4d6c700 1 --2- 172.21.15.47:0/123617851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff0d0147070 0x7ff0d0062050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/123617851 >> v1:172.21.15.47:6789/0 conn(0x7ff0d0060080 legacy=0x7ff0d00b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0ceffd700 1 -- 172.21.15.47:0/123617851 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428043624 0 0) 0x7ff0d0062f30 con 0x7ff0d0060080 2020-07-18T03:33:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/123617851 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d014a6e0 con 0x7ff0d01474c0 2020-07-18T03:33:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d4d6c700 1 --2- 172.21.15.47:0/123617851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d01474c0 0x7ff0d0062540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7ff0c80032b0 tx=0x7ff0c80091d0).ready entity=mon.1 client_cookie=409db481e0f5bacf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0ceffd700 1 -- 172.21.15.47:0/123617851 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0c800ba50 con 0x7ff0d01474c0 2020-07-18T03:33:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0ceffd700 1 -- 172.21.15.47:0/123617851 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c800bbb0 con 0x7ff0d01474c0 2020-07-18T03:33:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0ceffd700 1 -- 172.21.15.47:0/123617851 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0c800bd20 con 0x7ff0d01474c0 2020-07-18T03:33:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 -- 172.21.15.47:0/123617851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d01474c0 msgr2=0x7ff0d0062540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/123617851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d01474c0 0x7ff0d0062540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7ff0c80032b0 tx=0x7ff0c80091d0).stop 2020-07-18T03:33:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 -- 172.21.15.47:0/123617851 shutdown_connections 2020-07-18T03:33:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/123617851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d01474c0 0x7ff0d0062540 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/123617851 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7ff0d0147070 0x7ff0d0062050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 -- 172.21.15.47:0/123617851 >> 172.21.15.47:0/123617851 conn(0x7ff0d00ba2a0 msgr2=0x7ff0d00b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 -- 172.21.15.47:0/123617851 shutdown_connections 2020-07-18T03:33:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.840+0000 7ff0d67cf700 1 -- 172.21.15.47:0/123617851 wait complete. 2020-07-18T03:33:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.842+0000 7ff0d67cf700 1 Processor -- start 2020-07-18T03:33:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.842+0000 7ff0d67cf700 1 -- start start 2020-07-18T03:33:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ff0d0060080 0x7ff0d0064b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0d0137ec0 0x7ff0d00634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d0159870 con 0x7ff0d0147070 2020-07-18T03:33:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d00b3170 con 0x7ff0d0060080 2020-07-18T03:33:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d00b3410 con 0x7ff0d0137ec0 2020-07-18T03:33:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0d0137ec0 0x7ff0d00634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0d0137ec0 0x7ff0d00634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35262/0 (socket says 172.21.15.47:35262) 2020-07-18T03:33:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/3436576592 learned_addr learned my addr 172.21.15.47:0/3436576592 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0cffff700 1 --2- 172.21.15.47:0/3436576592 >> v2:172.21.15.18:3300/0 conn(0x7ff0d0060080 0x7ff0d0064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/3436576592 >> v2:172.21.15.18:3300/0 conn(0x7ff0d0060080 msgr2=0x7ff0d0064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 --2- 172.21.15.47:0/3436576592 >> v2:172.21.15.18:3300/0 conn(0x7ff0d0060080 0x7ff0d0064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/3436576592 >> v1:172.21.15.18:6789/0 conn(0x7ff0d0147070 legacy=0x7ff0d01377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 -- 172.21.15.47:0/3436576592 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0158a80 con 0x7ff0d0137ec0 2020-07-18T03:33:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d4d6c700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0137ec0 0x7ff0d00634a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7ff0d0155320 tx=0x7ff0c8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0c80107a0 con 0x7ff0d0137ec0 2020-07-18T03:33:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8010900 con 0x7ff0d0137ec0 2020-07-18T03:33:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d0158cd0 con 0x7ff0d0137ec0 2020-07-18T03:33:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0c8010c60 con 0x7ff0d0137ec0 2020-07-18T03:33:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.843+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d0159220 con 0x7ff0d0137ec0 2020-07-18T03:33:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.844+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d00b4070 con 0x7ff0d0137ec0 2020-07-18T03:33:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.845+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7ff0c8027030 con 0x7ff0d0137ec0 2020-07-18T03:33:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.845+0000 7ff0affff700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff0a804fee0 0x7ff0a80523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.845+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff0c8069080 con 0x7ff0d0137ec0 2020-07-18T03:33:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.845+0000 7ff0cffff700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff0a804fee0 0x7ff0a80523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.845+0000 7ff0cffff700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff0a804fee0 0x7ff0a80523a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7ff0c000d000 tx=0x7ff0c0001160).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:53.848+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7ff0c80417c0 con 0x7ff0d0137ec0 2020-07-18T03:33:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.058+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0d0057450 con 0x7ff0d0137ec0 2020-07-18T03:33:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.058+0000 7ff0affff700 1 -- 172.21.15.47:0/3436576592 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7ff0c8044de0 con 0x7ff0d0137ec0 2020-07-18T03:33:54.061 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:54.061 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff0a804fee0 msgr2=0x7ff0a80523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff0a804fee0 0x7ff0a80523a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7ff0c000d000 tx=0x7ff0c0001160).stop 2020-07-18T03:33:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0137ec0 msgr2=0x7ff0d00634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0137ec0 0x7ff0d00634a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7ff0d0155320 tx=0x7ff0c8016000).stop 2020-07-18T03:33:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 shutdown_connections 2020-07-18T03:33:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7ff0a804fee0 0x7ff0a80523a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/3436576592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0137ec0 0x7ff0d00634a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 --2- 172.21.15.47:0/3436576592 >> v2:172.21.15.18:3300/0 conn(0x7ff0d0060080 0x7ff0d0064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 >> 172.21.15.47:0/3436576592 conn(0x7ff0d00ba2a0 msgr2=0x7ff0d00b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 shutdown_connections 2020-07-18T03:33:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:54.059+0000 7ff0d67cf700 1 -- 172.21.15.47:0/3436576592 wait complete. 2020-07-18T03:33:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:54.072 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:55.073 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.199+0000 7f22a5028700 1 Processor -- start 2020-07-18T03:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.199+0000 7f22a5028700 1 -- start start 2020-07-18T03:33:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a5028700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f22a00b2290 0x7f22a01437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a5028700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b6530 0x7f22a00b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a5028700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a0143ca0 con 0x7f22a00b4b30 2020-07-18T03:33:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a5028700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00b3130 con 0x7f22a00b2290 2020-07-18T03:33:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a5028700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00b33d0 con 0x7f22a00b6530 2020-07-18T03:33:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a4827700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22a00b4b30 0x7f22a00b4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44590/0 (socket says 172.21.15.47:44590) 2020-07-18T03:33:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f22a4827700 1 -- 172.21.15.47:0/1447897790 learned_addr learned my addr 172.21.15.47:0/1447897790 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229ffff700 1 --2- 172.21.15.47:0/1447897790 >> v2:172.21.15.18:3300/0 conn(0x7f22a00b2290 0x7f22a01437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1447897790 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b6530 0x7f22a00b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229effd700 1 -- 172.21.15.47:0/1447897790 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437374901 0 0) 0x7f22a0143ca0 con 0x7f22a00b4b30 2020-07-18T03:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229effd700 1 -- 172.21.15.47:0/1447897790 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f228c003240 con 0x7f22a00b4b30 2020-07-18T03:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229effd700 1 -- 172.21.15.47:0/1447897790 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3973393336 0 0) 0x7f228c003240 con 0x7f22a00b4b30 2020-07-18T03:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229f7fe700 1 -- 172.21.15.47:0/1447897790 >> v2:172.21.15.18:3300/0 conn(0x7f22a00b2290 msgr2=0x7f22a01437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1447897790 >> v2:172.21.15.18:3300/0 conn(0x7f22a00b2290 0x7f22a01437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229f7fe700 1 -- 172.21.15.47:0/1447897790 >> v1:172.21.15.47:6789/0 conn(0x7f22a00b4b30 legacy=0x7f22a00b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229f7fe700 1 -- 172.21.15.47:0/1447897790 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00b2670 con 0x7f22a00b6530 2020-07-18T03:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.200+0000 7f229ffff700 1 --2- 172.21.15.47:0/1447897790 >> v2:172.21.15.18:3300/0 conn(0x7f22a00b2290 0x7f22a01437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1447897790 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b6530 0x7f22a00b6990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f2298001bf0 tx=0x7f2294006dd0).ready entity=mon.2 client_cookie=91803b8ead90975 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f229effd700 1 -- 172.21.15.47:0/1447897790 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2298001a50 con 0x7f22a00b6530 2020-07-18T03:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f229effd700 1 -- 172.21.15.47:0/1447897790 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229400d570 con 0x7f22a00b6530 2020-07-18T03:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f229effd700 1 -- 172.21.15.47:0/1447897790 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f229400d6e0 con 0x7f22a00b6530 2020-07-18T03:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 -- 172.21.15.47:0/1447897790 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b6530 msgr2=0x7f22a00b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 --2- 172.21.15.47:0/1447897790 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b6530 0x7f22a00b6990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f2298001bf0 tx=0x7f2294006dd0).stop 2020-07-18T03:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 -- 172.21.15.47:0/1447897790 shutdown_connections 2020-07-18T03:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 --2- 172.21.15.47:0/1447897790 >> v2:172.21.15.18:3300/0 conn(0x7f22a00b2290 0x7f22a01437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 --2- 172.21.15.47:0/1447897790 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b6530 0x7f22a00b6990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 -- 172.21.15.47:0/1447897790 >> 172.21.15.47:0/1447897790 conn(0x7f22a00b9e50 msgr2=0x7f22a00b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 -- 172.21.15.47:0/1447897790 shutdown_connections 2020-07-18T03:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.201+0000 7f22a5028700 1 -- 172.21.15.47:0/1447897790 wait complete. 2020-07-18T03:33:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.202+0000 7f22a5028700 1 Processor -- start 2020-07-18T03:33:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 -- start start 2020-07-18T03:33:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 0x7f22a0145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b4b30 0x7f22a0062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f22a00668e0 0x7f22a0155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a01452c0 con 0x7f22a00668e0 2020-07-18T03:33:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a01454d0 con 0x7f22a00b2290 2020-07-18T03:33:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a5028700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a0145770 con 0x7f22a00b4b30 2020-07-18T03:33:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f229ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 0x7f22a0145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f229ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 0x7f22a0145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35272/0 (socket says 172.21.15.47:35272) 2020-07-18T03:33:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f229ffff700 1 -- 172.21.15.47:0/1684371770 learned_addr learned my addr 172.21.15.47:0/1684371770 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b4b30 0x7f22a0062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.203+0000 7f22a4827700 1 --2- 172.21.15.47:0/1684371770 >> v2:172.21.15.18:3300/0 conn(0x7f22a00668e0 0x7f22a0155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229f7fe700 1 -- 172.21.15.47:0/1684371770 >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 msgr2=0x7f22a0145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1684371770 >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 0x7f22a0145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229f7fe700 1 -- 172.21.15.47:0/1684371770 >> v2:172.21.15.18:3300/0 conn(0x7f22a00668e0 msgr2=0x7f22a0155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1684371770 >> v2:172.21.15.18:3300/0 conn(0x7f22a00668e0 0x7f22a0155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229f7fe700 1 -- 172.21.15.47:0/1684371770 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0063140 con 0x7f22a00b4b30 2020-07-18T03:33:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229ffff700 1 --2- 172.21.15.47:0/1684371770 >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 0x7f22a0145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f229f7fe700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b4b30 0x7f22a0062c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f2298001bf0 tx=0x7f2294015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2294011b40 con 0x7f22a00b4b30 2020-07-18T03:33:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2294011ca0 con 0x7f22a00b4b30 2020-07-18T03:33:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a0063330 con 0x7f22a00b4b30 2020-07-18T03:33:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f229401e560 con 0x7f22a00b4b30 2020-07-18T03:33:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.204+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a0155d00 con 0x7f22a00b4b30 2020-07-18T03:33:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.205+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a0057450 con 0x7f22a00b4b30 2020-07-18T03:33:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.205+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f229401a040 con 0x7f22a00b4b30 2020-07-18T03:33:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.206+0000 7f227ffff700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2278050060 0x7f2278052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.206+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f229406abb0 con 0x7f22a00b4b30 2020-07-18T03:33:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.206+0000 7f229ffff700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2278050060 0x7f2278052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.206+0000 7f229ffff700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2278050060 0x7f2278052520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f229000b000 tx=0x7f2290006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.210+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2294071080 con 0x7f22a00b4b30 2020-07-18T03:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.420+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22a0068c50 con 0x7f22a00b4b30 2020-07-18T03:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.421+0000 7f227ffff700 1 -- 172.21.15.47:0/1684371770 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f22940432f0 con 0x7f22a00b4b30 2020-07-18T03:33:55.423 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:55.423 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:55.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2278050060 msgr2=0x7f2278052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:55.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2278050060 0x7f2278052520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f229000b000 tx=0x7f2290006dd0).stop 2020-07-18T03:33:55.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b4b30 msgr2=0x7f22a0062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:55.425 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b4b30 0x7f22a0062c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f2298001bf0 tx=0x7f2294015000).stop 2020-07-18T03:33:55.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 shutdown_connections 2020-07-18T03:33:55.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2278050060 0x7f2278052520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 --2- 172.21.15.47:0/1684371770 >> v2:172.21.15.18:3300/0 conn(0x7f22a00668e0 0x7f22a0155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 --2- 172.21.15.47:0/1684371770 >> v2:172.21.15.47:3300/0 conn(0x7f22a00b2290 0x7f22a0145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 --2- 172.21.15.47:0/1684371770 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f22a00b4b30 0x7f22a0062c50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:55.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 >> 172.21.15.47:0/1684371770 conn(0x7f22a00b9e50 msgr2=0x7f22a00b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:55.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 shutdown_connections 2020-07-18T03:33:55.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:55.422+0000 7f22a5028700 1 -- 172.21.15.47:0/1684371770 wait complete. 2020-07-18T03:33:55.427 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:55.435 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:56.437 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.563+0000 7fd6309da700 1 Processor -- start 2020-07-18T03:33:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.563+0000 7fd6309da700 1 -- start start 2020-07-18T03:33:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd6309da700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd6309da700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 0x7fd62c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd6309da700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c147330 con 0x7fd62c147a00 2020-07-18T03:33:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd6309da700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0ae530 con 0x7fd62c0ad100 2020-07-18T03:33:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd6309da700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b57e0 con 0x7fd62c1443f0 2020-07-18T03:33:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd62ad9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 0x7fd62c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd62a59c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd62ad9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 0x7fd62c147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56114/0 (socket says 172.21.15.47:56114) 2020-07-18T03:33:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd629d9b700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fd62c147a00 0x7fd62c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35134/0 (socket says 172.21.15.47:35134) 2020-07-18T03:33:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd62a59c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:49192/0 (socket says 172.21.15.47:49192) 2020-07-18T03:33:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.564+0000 7fd62ad9d700 1 -- 172.21.15.47:0/4191869328 learned_addr learned my addr 172.21.15.47:0/4191869328 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62ad9d700 1 -- 172.21.15.47:0/4191869328 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 msgr2=0x7fd62c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62ad9d700 1 --2- 172.21.15.47:0/4191869328 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62ad9d700 1 -- 172.21.15.47:0/4191869328 >> v1:172.21.15.18:6789/0 conn(0x7fd62c147a00 legacy=0x7fd62c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62ad9d700 1 -- 172.21.15.47:0/4191869328 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0b5a80 con 0x7fd62c1443f0 2020-07-18T03:33:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62a59c700 1 --2- 172.21.15.47:0/4191869328 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62ad9d700 1 --2- 172.21.15.47:0/4191869328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 0x7fd62c147e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd6240032b0 tx=0x7fd624009270).ready entity=mon.2 client_cookie=34e05ebd1956d32b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62959a700 1 -- 172.21.15.47:0/4191869328 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd624009e60 con 0x7fd62c1443f0 2020-07-18T03:33:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62959a700 1 -- 172.21.15.47:0/4191869328 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62400bc00 con 0x7fd62c1443f0 2020-07-18T03:33:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd62959a700 1 -- 172.21.15.47:0/4191869328 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd62400bd70 con 0x7fd62c1443f0 2020-07-18T03:33:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 -- 172.21.15.47:0/4191869328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 msgr2=0x7fd62c147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 --2- 172.21.15.47:0/4191869328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 0x7fd62c147e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd6240032b0 tx=0x7fd624009270).stop 2020-07-18T03:33:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 -- 172.21.15.47:0/4191869328 shutdown_connections 2020-07-18T03:33:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 --2- 172.21.15.47:0/4191869328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c1443f0 0x7fd62c147e40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 --2- 172.21.15.47:0/4191869328 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 -- 172.21.15.47:0/4191869328 >> 172.21.15.47:0/4191869328 conn(0x7fd62c0b9340 msgr2=0x7fd62c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 -- 172.21.15.47:0/4191869328 shutdown_connections 2020-07-18T03:33:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.565+0000 7fd6309da700 1 -- 172.21.15.47:0/4191869328 wait complete. 2020-07-18T03:33:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.567+0000 7fd6309da700 1 Processor -- start 2020-07-18T03:33:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.567+0000 7fd6309da700 1 -- start start 2020-07-18T03:33:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd6309da700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd6309da700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 0x7fd62c0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd6309da700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c065650 con 0x7fd62c13c750 2020-07-18T03:33:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd6309da700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c13ccd0 con 0x7fd62c0ad100 2020-07-18T03:33:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd6309da700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c13cf70 con 0x7fd62c147a00 2020-07-18T03:33:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 0x7fd62c0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd62a59c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 0x7fd62c0b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56118/0 (socket says 172.21.15.47:56118) 2020-07-18T03:33:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd62ad9d700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fd62c13c750 0x7fd62c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35144/0 (socket says 172.21.15.47:35144) 2020-07-18T03:33:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 -- 172.21.15.47:0/4114796539 learned_addr learned my addr 172.21.15.47:0/4114796539 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 -- 172.21.15.47:0/4114796539 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 msgr2=0x7fd62c066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:56.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 --2- 172.21.15.47:0/4114796539 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 -- 172.21.15.47:0/4114796539 >> v1:172.21.15.18:6789/0 conn(0x7fd62c13c750 legacy=0x7fd62c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:56.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 -- 172.21.15.47:0/4114796539 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c13d210 con 0x7fd62c147a00 2020-07-18T03:33:56.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254966976 0 0) 0x7fd62c065650 con 0x7fd62c13c750 2020-07-18T03:33:56.579 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd62a59c700 1 --2- 172.21.15.47:0/4114796539 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:33:56.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.568+0000 7fd629d9b700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 0x7fd62c0b1420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd62c05ca30 tx=0x7fd61c007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:56.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.569+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd61c0072e0 con 0x7fd62c147a00 2020-07-18T03:33:56.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.569+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c005530 con 0x7fd62c147a00 2020-07-18T03:33:56.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.569+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd61c005890 con 0x7fd62c147a00 2020-07-18T03:33:56.580 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.569+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c152800 con 0x7fd62c147a00 2020-07-18T03:33:56.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.569+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c152d20 con 0x7fd62c147a00 2020-07-18T03:33:56.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.570+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c0adf10 con 0x7fd62c147a00 2020-07-18T03:33:56.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.570+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd61c01a060 con 0x7fd62c147a00 2020-07-18T03:33:56.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.570+0000 7fd61a7fc700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd60004fe60 0x7fd600052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:56.581 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.570+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd61c0674b0 con 0x7fd62c147a00 2020-07-18T03:33:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.570+0000 7fd62a59c700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd60004fe60 0x7fd600052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.571+0000 7fd62a59c700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd60004fe60 0x7fd600052320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd62000b000 tx=0x7fd620006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.574+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd61c03fc40 con 0x7fd62c147a00 2020-07-18T03:33:56.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.783+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62c057450 con 0x7fd62c147a00 2020-07-18T03:33:56.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.784+0000 7fd61a7fc700 1 -- 172.21.15.47:0/4114796539 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fd61c018070 con 0x7fd62c147a00 2020-07-18T03:33:56.786 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:56.787 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:56.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd60004fe60 msgr2=0x7fd600052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd60004fe60 0x7fd600052320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd62000b000 tx=0x7fd620006dd0).stop 2020-07-18T03:33:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 msgr2=0x7fd62c0b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 0x7fd62c0b1420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd62c05ca30 tx=0x7fd61c007c20).stop 2020-07-18T03:33:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 shutdown_connections 2020-07-18T03:33:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd60004fe60 0x7fd600052320 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 --2- 172.21.15.47:0/4114796539 >> v2:172.21.15.18:3300/0 conn(0x7fd62c0ad100 0x7fd62c066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 --2- 172.21.15.47:0/4114796539 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd62c147a00 0x7fd62c0b1420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:56.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 >> 172.21.15.47:0/4114796539 conn(0x7fd62c0b9340 msgr2=0x7fd62c1469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:56.795 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 shutdown_connections 2020-07-18T03:33:56.796 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:56.785+0000 7fd6309da700 1 -- 172.21.15.47:0/4114796539 wait complete. 2020-07-18T03:33:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:56.803 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:57.806 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.932+0000 7f9a588ab700 1 Processor -- start 2020-07-18T03:33:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.932+0000 7f9a588ab700 1 -- start start 2020-07-18T03:33:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.933+0000 7f9a588ab700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9a540628a0 0x7f9a5413db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.933+0000 7f9a588ab700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a54061b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.933+0000 7f9a588ab700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a541441f0 con 0x7f9a54062000 2020-07-18T03:33:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.933+0000 7f9a588ab700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a54143fe0 con 0x7f9a540628a0 2020-07-18T03:33:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.933+0000 7f9a588ab700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54144490 con 0x7f9a540ad990 2020-07-18T03:33:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9a540628a0 0x7f9a5413db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9a540628a0 0x7f9a5413db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:49206/0 (socket says 172.21.15.47:49206) 2020-07-18T03:33:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 -- 172.21.15.47:0/251144683 learned_addr learned my addr 172.21.15.47:0/251144683 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a52d9d700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f9a54062000 0x7f9a54147280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35152/0 (socket says 172.21.15.47:35152) 2020-07-18T03:33:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a51d9b700 1 --2- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a54061b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 -- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 msgr2=0x7f9a54061b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 --2- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a54061b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 -- 172.21.15.47:0/251144683 >> v1:172.21.15.18:6789/0 conn(0x7f9a54062000 legacy=0x7f9a54147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 -- 172.21.15.47:0/251144683 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54143c80 con 0x7f9a540628a0 2020-07-18T03:33:57.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.934+0000 7f9a5259c700 1 --2- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a540628a0 0x7f9a5413db90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f9a3c002f30 tx=0x7f9a3c006f20).ready entity=mon.0 client_cookie=9d13a35c7447fae7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:57.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a5159a700 1 -- 172.21.15.47:0/251144683 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3c00dda0 con 0x7f9a540628a0 2020-07-18T03:33:57.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a5159a700 1 -- 172.21.15.47:0/251144683 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c010470 con 0x7f9a540628a0 2020-07-18T03:33:57.941 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a5159a700 1 -- 172.21.15.47:0/251144683 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3c0107d0 con 0x7f9a540628a0 2020-07-18T03:33:57.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 -- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a540628a0 msgr2=0x7f9a5413db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:57.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 --2- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a540628a0 0x7f9a5413db90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f9a3c002f30 tx=0x7f9a3c006f20).stop 2020-07-18T03:33:57.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 -- 172.21.15.47:0/251144683 shutdown_connections 2020-07-18T03:33:57.942 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 --2- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a540628a0 0x7f9a5413db90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:57.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 --2- 172.21.15.47:0/251144683 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a54061b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:57.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 -- 172.21.15.47:0/251144683 >> 172.21.15.47:0/251144683 conn(0x7f9a540b92e0 msgr2=0x7f9a540b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:57.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 -- 172.21.15.47:0/251144683 shutdown_connections 2020-07-18T03:33:57.943 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.935+0000 7f9a588ab700 1 -- 172.21.15.47:0/251144683 wait complete. 2020-07-18T03:33:57.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.936+0000 7f9a588ab700 1 Processor -- start 2020-07-18T03:33:57.946 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.936+0000 7f9a588ab700 1 -- start start 2020-07-18T03:33:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a588ab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a54062000 0x7f9a54154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a588ab700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a540b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a588ab700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9a5413dc00 0x7f9a5413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a588ab700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a54154420 con 0x7f9a5413dc00 2020-07-18T03:33:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a588ab700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a541546c0 con 0x7f9a54062000 2020-07-18T03:33:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a588ab700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54154960 con 0x7f9a540ad990 2020-07-18T03:33:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a5259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a54062000 0x7f9a54154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a5259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a54062000 0x7f9a54154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35300/0 (socket says 172.21.15.47:35300) 2020-07-18T03:33:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a5259c700 1 -- 172.21.15.47:0/1823146021 learned_addr learned my addr 172.21.15.47:0/1823146021 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a51d9b700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a540b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a52d9d700 1 --2- 172.21.15.47:0/1823146021 >> v2:172.21.15.18:3300/0 conn(0x7f9a5413dc00 0x7f9a5413ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.937+0000 7f9a5259c700 1 -- 172.21.15.47:0/1823146021 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 msgr2=0x7f9a540b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a5259c700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a540b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a5259c700 1 -- 172.21.15.47:0/1823146021 >> v2:172.21.15.18:3300/0 conn(0x7f9a5413dc00 msgr2=0x7f9a5413ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a5259c700 1 --2- 172.21.15.47:0/1823146021 >> v2:172.21.15.18:3300/0 conn(0x7f9a5413dc00 0x7f9a5413ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a5259c700 1 -- 172.21.15.47:0/1823146021 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a540b8a10 con 0x7f9a54062000 2020-07-18T03:33:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a5259c700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a54062000 0x7f9a54154e00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f9a3c002f30 tx=0x7f9a3c018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3c018760 con 0x7f9a54062000 2020-07-18T03:33:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0188c0 con 0x7f9a54062000 2020-07-18T03:33:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5413e0b0 con 0x7f9a54062000 2020-07-18T03:33:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3c018c20 con 0x7f9a54062000 2020-07-18T03:33:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.938+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a54142d50 con 0x7f9a54062000 2020-07-18T03:33:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.939+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a540690d0 con 0x7f9a54062000 2020-07-18T03:33:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.940+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9a3c021070 con 0x7f9a54062000 2020-07-18T03:33:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.940+0000 7f9a4a7fc700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a300500f0 0x7f9a300525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.940+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9a3c06c180 con 0x7f9a54062000 2020-07-18T03:33:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.940+0000 7f9a51d9b700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a300500f0 0x7f9a300525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.940+0000 7f9a51d9b700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a300500f0 0x7f9a300525b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f9a4400b000 tx=0x7f9a440091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:57.943+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9a3c0483a0 con 0x7f9a54062000 2020-07-18T03:33:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.153+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a5406a100 con 0x7f9a54062000 2020-07-18T03:33:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.154+0000 7f9a4a7fc700 1 -- 172.21.15.47:0/1823146021 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f9a3c047f90 con 0x7f9a54062000 2020-07-18T03:33:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:58.157 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:58.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a300500f0 msgr2=0x7f9a300525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:58.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a300500f0 0x7f9a300525b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f9a4400b000 tx=0x7f9a440091d0).stop 2020-07-18T03:33:58.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a54062000 msgr2=0x7f9a54154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a54062000 0x7f9a54154e00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f9a3c002f30 tx=0x7f9a3c018000).stop 2020-07-18T03:33:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 shutdown_connections 2020-07-18T03:33:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a300500f0 0x7f9a300525b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.155+0000 7f9a588ab700 1 --2- 172.21.15.47:0/1823146021 >> v2:172.21.15.18:3300/0 conn(0x7f9a5413dc00 0x7f9a5413ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.156+0000 7f9a588ab700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a54062000 0x7f9a54154e00 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.156+0000 7f9a588ab700 1 --2- 172.21.15.47:0/1823146021 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a540ad990 0x7f9a540b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.156+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 >> 172.21.15.47:0/1823146021 conn(0x7f9a540b92e0 msgr2=0x7f9a540b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.156+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 shutdown_connections 2020-07-18T03:33:58.162 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:58.156+0000 7f9a588ab700 1 -- 172.21.15.47:0/1823146021 wait complete. 2020-07-18T03:33:58.162 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:58.168 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:33:59.169 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:33:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.296+0000 7f7f41f12700 1 Processor -- start 2020-07-18T03:33:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.296+0000 7f7f41f12700 1 -- start start 2020-07-18T03:33:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f41f12700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f3c0b7420 0x7f7f3c0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f41f12700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c0b95a0 con 0x7f7f3c0b82f0 2020-07-18T03:33:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f41f12700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c0b5c80 con 0x7f7f3c0b42a0 2020-07-18T03:33:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f41f12700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3c0b5a90 con 0x7f7f3c0b7420 2020-07-18T03:33:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7f3c0b42a0 0x7f7f3c0b6660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44628/0 (socket says 172.21.15.47:44628) 2020-07-18T03:33:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3b7fe700 1 -- 172.21.15.47:0/3766861090 learned_addr learned my addr 172.21.15.47:0/3766861090 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:33:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3affd700 1 --2- 172.21.15.47:0/3766861090 >> v2:172.21.15.18:3300/0 conn(0x7f7f3c0b7420 0x7f7f3c0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190832297 0 0) 0x7f7f3c0b5c80 con 0x7f7f3c0b42a0 2020-07-18T03:33:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f28003240 con 0x7f7f3c0b42a0 2020-07-18T03:33:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331728775 0 0) 0x7f7f3c0b95a0 con 0x7f7f3c0b82f0 2020-07-18T03:33:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3c0b5c80 con 0x7f7f3c0b82f0 2020-07-18T03:33:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1062986046 0 0) 0x7f7f28003240 con 0x7f7f3c0b42a0 2020-07-18T03:33:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 >> v2:172.21.15.18:3300/0 conn(0x7f7f3c0b7420 msgr2=0x7f7f3c0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 --2- 172.21.15.47:0/3766861090 >> v2:172.21.15.18:3300/0 conn(0x7f7f3c0b7420 0x7f7f3c0b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 >> v1:172.21.15.18:6789/0 conn(0x7f7f3c0b82f0 legacy=0x7f7f3c14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0b5f20 con 0x7f7f3c0b42a0 2020-07-18T03:33:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7f2c0016d0 con 0x7f7f3c0b42a0 2020-07-18T03:33:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 >> v1:172.21.15.47:6789/0 conn(0x7f7f3c0b42a0 legacy=0x7f7f3c0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f280038c0 0x7f7f28003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f280044e0 0x7f7f28004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f28004e50 0x7f7f2800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2800bc20 con 0x7f7f280044e0 2020-07-18T03:33:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2800bd60 con 0x7f7f280038c0 2020-07-18T03:33:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.297+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2800bef0 con 0x7f7f28004e50 2020-07-18T03:33:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f280038c0 0x7f7f28003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3affd700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f280044e0 0x7f7f28004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3bfff700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f28004e50 0x7f7f2800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 -- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f28004e50 msgr2=0x7f7f2800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f28004e50 0x7f7f2800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 -- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f280044e0 msgr2=0x7f7f28004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:33:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f280044e0 0x7f7f28004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 -- 172.21.15.47:0/3766861090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28003240 con 0x7f7f280038c0 2020-07-18T03:33:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3bfff700 1 -- 172.21.15.47:0/3766861090 reap_dead start 2020-07-18T03:33:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3b7fe700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f280038c0 0x7f7f28003cc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f7f2c0014a0 tx=0x7f7f3c0b60b0).ready entity=mon.1 client_cookie=9eb671d3bcfed1a0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f2c00cde0 con 0x7f7f280038c0 2020-07-18T03:33:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0070d0 con 0x7f7f280038c0 2020-07-18T03:33:59.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f3a7fc700 1 -- 172.21.15.47:0/3766861090 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f2c007430 con 0x7f7f280038c0 2020-07-18T03:33:59.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 -- 172.21.15.47:0/3766861090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f280038c0 msgr2=0x7f7f28003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f280038c0 0x7f7f28003cc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f7f2c0014a0 tx=0x7f7f3c0b60b0).stop 2020-07-18T03:33:59.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 -- 172.21.15.47:0/3766861090 shutdown_connections 2020-07-18T03:33:59.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 --2- 172.21.15.47:0/3766861090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f280038c0 0x7f7f28003cc0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 -- 172.21.15.47:0/3766861090 >> 172.21.15.47:0/3766861090 conn(0x7f7f3c05f5a0 msgr2=0x7f7f3c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 -- 172.21.15.47:0/3766861090 shutdown_connections 2020-07-18T03:33:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.298+0000 7f7f41f12700 1 -- 172.21.15.47:0/3766861090 wait complete. 2020-07-18T03:33:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.300+0000 7f7f41f12700 1 Processor -- start 2020-07-18T03:33:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.300+0000 7f7f41f12700 1 -- start start 2020-07-18T03:33:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f41f12700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f3c1419f0 0x7f7f3c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f41f12700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 0x7f7f3c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f41f12700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c0b0f40 con 0x7f7f3c0aec10 2020-07-18T03:33:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f41f12700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3c0b1560 con 0x7f7f3c1419f0 2020-07-18T03:33:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f41f12700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c0ae110 con 0x7f7f3c141dd0 2020-07-18T03:33:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 0x7f7f3c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3b7fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f3c1419f0 0x7f7f3c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 0x7f7f3c158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56152/0 (socket says 172.21.15.47:56152) 2020-07-18T03:33:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 -- 172.21.15.47:0/513901917 learned_addr learned my addr 172.21.15.47:0/513901917 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:33:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443140983 0 0) 0x7f7f3c0b0f40 con 0x7f7f3c0aec10 2020-07-18T03:33:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f14003240 con 0x7f7f3c0aec10 2020-07-18T03:33:59.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 -- 172.21.15.47:0/513901917 >> v2:172.21.15.18:3300/0 conn(0x7f7f3c1419f0 msgr2=0x7f7f3c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 --2- 172.21.15.47:0/513901917 >> v2:172.21.15.18:3300/0 conn(0x7f7f3c1419f0 0x7f7f3c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 -- 172.21.15.47:0/513901917 >> v1:172.21.15.18:6789/0 conn(0x7f7f3c0aec10 legacy=0x7f7f3c155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.326 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 -- 172.21.15.47:0/513901917 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0ae250 con 0x7f7f3c141dd0 2020-07-18T03:33:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.301+0000 7f7f3affd700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 0x7f7f3c158ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f7f30008000 tx=0x7f7f30008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.302+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f30018070 con 0x7f7f3c141dd0 2020-07-18T03:33:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.302+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000f760 con 0x7f7f3c141dd0 2020-07-18T03:33:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.302+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c158fb0 con 0x7f7f3c141dd0 2020-07-18T03:33:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.302+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3000fac0 con 0x7f7f3c141dd0 2020-07-18T03:33:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.302+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c155710 con 0x7f7f3c141dd0 2020-07-18T03:33:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.303+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c0b4e90 con 0x7f7f3c141dd0 2020-07-18T03:33:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.303+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7f3000fc20 con 0x7f7f3c141dd0 2020-07-18T03:33:59.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.303+0000 7f7f1b7fe700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f140503e0 0x7f7f140528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:33:59.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.303+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7f300680a0 con 0x7f7f3c141dd0 2020-07-18T03:33:59.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.303+0000 7f7f3b7fe700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f140503e0 0x7f7f140528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:33:59.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.303+0000 7f7f3b7fe700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f140503e0 0x7f7f140528a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f7f2c019000 tx=0x7f7f2c013000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:33:59.330 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.307+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7f300407e0 con 0x7f7f3c141dd0 2020-07-18T03:33:59.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.516+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f3c0b16a0 con 0x7f7f3c141dd0 2020-07-18T03:33:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.516+0000 7f7f1b7fe700 1 -- 172.21.15.47:0/513901917 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f7f30043e00 con 0x7f7f3c141dd0 2020-07-18T03:33:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:33:59.519 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:33:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f140503e0 msgr2=0x7f7f140528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f140503e0 0x7f7f140528a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f7f2c019000 tx=0x7f7f2c013000).stop 2020-07-18T03:33:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 msgr2=0x7f7f3c158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:33:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 0x7f7f3c158ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f7f30008000 tx=0x7f7f30008040).stop 2020-07-18T03:33:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 shutdown_connections 2020-07-18T03:33:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7f140503e0 0x7f7f140528a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 --2- 172.21.15.47:0/513901917 >> v2:172.21.15.18:3300/0 conn(0x7f7f3c1419f0 0x7f7f3c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 --2- 172.21.15.47:0/513901917 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f3c141dd0 0x7f7f3c158ac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:33:59.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 >> 172.21.15.47:0/513901917 conn(0x7f7f3c05f5a0 msgr2=0x7f7f3c0b9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:33:59.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 shutdown_connections 2020-07-18T03:33:59.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:33:59.518+0000 7f7f41f12700 1 -- 172.21.15.47:0/513901917 wait complete. 2020-07-18T03:33:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:33:59.530 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:00.532 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.659+0000 7f141d72e700 1 Processor -- start 2020-07-18T03:34:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.659+0000 7f141d72e700 1 -- start start 2020-07-18T03:34:00.663 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f141d72e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14180ad100 0x7f14180b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f141d72e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1418147330 con 0x7f14181443f0 2020-07-18T03:34:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f141d72e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180603c0 con 0x7f1418147a00 2020-07-18T03:34:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f141d72e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180b57e0 con 0x7f14180ad100 2020-07-18T03:34:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f14167fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1418147a00 0x7f141805f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44648/0 (socket says 172.21.15.47:44648) 2020-07-18T03:34:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1416ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14180ad100 0x7f14180b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f14167fc700 1 -- 172.21.15.47:0/102939804 learned_addr learned my addr 172.21.15.47:0/102939804 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1416ffd700 1 -- 172.21.15.47:0/102939804 >> v1:172.21.15.47:6789/0 conn(0x7f1418147a00 legacy=0x7f141805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1416ffd700 1 -- 172.21.15.47:0/102939804 >> v1:172.21.15.18:6789/0 conn(0x7f14181443f0 legacy=0x7f141805fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1416ffd700 1 -- 172.21.15.47:0/102939804 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180b5a80 con 0x7f14180ad100 2020-07-18T03:34:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1416ffd700 1 --2- 172.21.15.47:0/102939804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180ad100 0x7f14180b4c10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f14100032b0 tx=0x7f14100091d0).ready entity=mon.1 client_cookie=d329401ea5bf8d67 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1415ffb700 1 -- 172.21.15.47:0/102939804 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f141000ba50 con 0x7f14180ad100 2020-07-18T03:34:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.660+0000 7f1415ffb700 1 -- 172.21.15.47:0/102939804 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000bbb0 con 0x7f14180ad100 2020-07-18T03:34:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f1415ffb700 1 -- 172.21.15.47:0/102939804 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f141000bd20 con 0x7f14180ad100 2020-07-18T03:34:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 -- 172.21.15.47:0/102939804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180ad100 msgr2=0x7f14180b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 --2- 172.21.15.47:0/102939804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180ad100 0x7f14180b4c10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f14100032b0 tx=0x7f14100091d0).stop 2020-07-18T03:34:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 -- 172.21.15.47:0/102939804 shutdown_connections 2020-07-18T03:34:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 --2- 172.21.15.47:0/102939804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180ad100 0x7f14180b4c10 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 -- 172.21.15.47:0/102939804 >> 172.21.15.47:0/102939804 conn(0x7f14180b9340 msgr2=0x7f14180b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 -- 172.21.15.47:0/102939804 shutdown_connections 2020-07-18T03:34:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.661+0000 7f141d72e700 1 -- 172.21.15.47:0/102939804 wait complete. 2020-07-18T03:34:00.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.662+0000 7f141d72e700 1 Processor -- start 2020-07-18T03:34:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.662+0000 7f141d72e700 1 -- start start 2020-07-18T03:34:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f141d72e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f14181443f0 0x7f1418151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f141d72e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14180b2170 0x7f1418150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f141d72e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1418151020 con 0x7f1418147a00 2020-07-18T03:34:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f141d72e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1418061bb0 con 0x7f14181443f0 2020-07-18T03:34:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f141d72e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1418061e50 con 0x7f14180b2170 2020-07-18T03:34:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f14177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14180b2170 0x7f1418150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f14177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14180b2170 0x7f1418150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35336/0 (socket says 172.21.15.47:35336) 2020-07-18T03:34:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f14177fe700 1 -- 172.21.15.47:0/2602879791 learned_addr learned my addr 172.21.15.47:0/2602879791 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:00.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f1416ffd700 1 --2- 172.21.15.47:0/2602879791 >> v2:172.21.15.18:3300/0 conn(0x7f14181443f0 0x7f1418151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:00.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f14177fe700 1 -- 172.21.15.47:0/2602879791 >> v2:172.21.15.18:3300/0 conn(0x7f14181443f0 msgr2=0x7f1418151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f14177fe700 1 --2- 172.21.15.47:0/2602879791 >> v2:172.21.15.18:3300/0 conn(0x7f14181443f0 0x7f1418151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:00.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.663+0000 7f14177fe700 1 -- 172.21.15.47:0/2602879791 >> v1:172.21.15.18:6789/0 conn(0x7f1418147a00 legacy=0x7f1418142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f14177fe700 1 -- 172.21.15.47:0/2602879791 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180b25e0 con 0x7f14180b2170 2020-07-18T03:34:00.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f1416ffd700 1 --2- 172.21.15.47:0/2602879791 >> v2:172.21.15.18:3300/0 conn(0x7f14181443f0 0x7f1418151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:00.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f14177fe700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180b2170 0x7f1418150b30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f1418140f00 tx=0x7f1408007b10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:00.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f140800e4a0 con 0x7f14180b2170 2020-07-18T03:34:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140800e600 con 0x7f14180b2170 2020-07-18T03:34:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f140800e960 con 0x7f14180b2170 2020-07-18T03:34:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f141d72e700 1 -- 172.21.15.47:0/2602879791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14181427a0 con 0x7f14180b2170 2020-07-18T03:34:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.664+0000 7f141d72e700 1 -- 172.21.15.47:0/2602879791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1418140a80 con 0x7f14180b2170 2020-07-18T03:34:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.665+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e8004d90 con 0x7f14180b2170 2020-07-18T03:34:00.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.665+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1408019030 con 0x7f14180b2170 2020-07-18T03:34:00.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.666+0000 7f140effd700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f13f404fff0 0x7f13f40524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:00.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.666+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1408067170 con 0x7f14180b2170 2020-07-18T03:34:00.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.666+0000 7f1416ffd700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f13f404fff0 0x7f13f40524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:00.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.666+0000 7f1416ffd700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f13f404fff0 0x7f13f40524b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1410017000 tx=0x7f1410016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:00.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.669+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f140803f8b0 con 0x7f14180b2170 2020-07-18T03:34:00.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.879+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e80015d0 con 0x7f14180b2170 2020-07-18T03:34:00.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.880+0000 7f140effd700 1 -- 172.21.15.47:0/2602879791 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f1408042ed0 con 0x7f14180b2170 2020-07-18T03:34:00.882 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:00.883 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:00.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f13f404fff0 msgr2=0x7f13f40524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f13f404fff0 0x7f13f40524b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1410017000 tx=0x7f1410016000).stop 2020-07-18T03:34:00.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180b2170 msgr2=0x7f1418150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:00.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180b2170 0x7f1418150b30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f1418140f00 tx=0x7f1408007b10).stop 2020-07-18T03:34:00.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 shutdown_connections 2020-07-18T03:34:00.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f13f404fff0 0x7f13f40524b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:00.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 --2- 172.21.15.47:0/2602879791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14180b2170 0x7f1418150b30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:00.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 --2- 172.21.15.47:0/2602879791 >> v2:172.21.15.18:3300/0 conn(0x7f14181443f0 0x7f1418151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 >> 172.21.15.47:0/2602879791 conn(0x7f14180b9340 msgr2=0x7f14181431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 shutdown_connections 2020-07-18T03:34:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:00.881+0000 7f140cff9700 1 -- 172.21.15.47:0/2602879791 wait complete. 2020-07-18T03:34:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:34:00.893 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:01.895 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:02.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.021+0000 7fa92d386700 1 Processor -- start 2020-07-18T03:34:02.023 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.021+0000 7fa92d386700 1 -- start start 2020-07-18T03:34:02.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa92d386700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9280b5370 0x7fa92805fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:02.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa92d386700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9280b41c0 0x7fa9280ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa92d386700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa9280b4620 0x7fa9280ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa92d386700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280b5a10 con 0x7fa9280b4620 2020-07-18T03:34:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa92d386700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280b0040 con 0x7fa9280b41c0 2020-07-18T03:34:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa92d386700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280b3af0 con 0x7fa9280b5370 2020-07-18T03:34:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9280b41c0 0x7fa9280ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9280b41c0 0x7fa9280ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35346/0 (socket says 172.21.15.47:35346) 2020-07-18T03:34:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 -- 172.21.15.47:0/698193992 learned_addr learned my addr 172.21.15.47:0/698193992 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa926ffd700 1 --2- 172.21.15.47:0/698193992 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9280b5370 0x7fa92805fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9277fe700 1 --2- 172.21.15.47:0/698193992 >> v2:172.21.15.18:3300/0 conn(0x7fa9280b4620 0x7fa9280ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 -- 172.21.15.47:0/698193992 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9280b5370 msgr2=0x7fa92805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 --2- 172.21.15.47:0/698193992 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9280b5370 0x7fa92805fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 -- 172.21.15.47:0/698193992 >> v2:172.21.15.18:3300/0 conn(0x7fa9280b4620 msgr2=0x7fa9280ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 --2- 172.21.15.47:0/698193992 >> v2:172.21.15.18:3300/0 conn(0x7fa9280b4620 0x7fa9280ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 -- 172.21.15.47:0/698193992 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280adcb0 con 0x7fa9280b41c0 2020-07-18T03:34:02.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9277fe700 1 --2- 172.21.15.47:0/698193992 >> v2:172.21.15.18:3300/0 conn(0x7fa9280b4620 0x7fa9280ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:02.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.022+0000 7fa9267fc700 1 --2- 172.21.15.47:0/698193992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b41c0 0x7fa9280ad2d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fa918002f30 tx=0x7fa918006e90).ready entity=mon.1 client_cookie=e2ac7ccc49bc319d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:02.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa925ffb700 1 -- 172.21.15.47:0/698193992 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa918009920 con 0x7fa9280b41c0 2020-07-18T03:34:02.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa925ffb700 1 -- 172.21.15.47:0/698193992 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918009a80 con 0x7fa9280b41c0 2020-07-18T03:34:02.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa925ffb700 1 -- 172.21.15.47:0/698193992 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa918009de0 con 0x7fa9280b41c0 2020-07-18T03:34:02.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 -- 172.21.15.47:0/698193992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b41c0 msgr2=0x7fa9280ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:02.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 --2- 172.21.15.47:0/698193992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b41c0 0x7fa9280ad2d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fa918002f30 tx=0x7fa918006e90).stop 2020-07-18T03:34:02.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 -- 172.21.15.47:0/698193992 shutdown_connections 2020-07-18T03:34:02.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 --2- 172.21.15.47:0/698193992 >> v2:172.21.15.18:3300/0 conn(0x7fa9280b4620 0x7fa9280ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 --2- 172.21.15.47:0/698193992 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9280b5370 0x7fa92805fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 --2- 172.21.15.47:0/698193992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b41c0 0x7fa9280ad2d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 -- 172.21.15.47:0/698193992 >> 172.21.15.47:0/698193992 conn(0x7fa9280b9340 msgr2=0x7fa9280b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:02.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 -- 172.21.15.47:0/698193992 shutdown_connections 2020-07-18T03:34:02.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.023+0000 7fa92d386700 1 -- 172.21.15.47:0/698193992 wait complete. 2020-07-18T03:34:02.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.024+0000 7fa92d386700 1 Processor -- start 2020-07-18T03:34:02.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.024+0000 7fa92d386700 1 -- start start 2020-07-18T03:34:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa92d386700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9280b5370 0x7fa92813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa92d386700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa928061bc0 0x7fa92813c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa92d386700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa928066200 con 0x7fa9281548a0 2020-07-18T03:34:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa92d386700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa928066430 con 0x7fa928061bc0 2020-07-18T03:34:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa92d386700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280666d0 con 0x7fa9280b5370 2020-07-18T03:34:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa926ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9280b5370 0x7fa92813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa926ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9280b5370 0x7fa92813bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35348/0 (socket says 172.21.15.47:35348) 2020-07-18T03:34:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa926ffd700 1 -- 172.21.15.47:0/3273175536 learned_addr learned my addr 172.21.15.47:0/3273175536 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa9267fc700 1 --2- 172.21.15.47:0/3273175536 >> v2:172.21.15.18:3300/0 conn(0x7fa928061bc0 0x7fa92813c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.025+0000 7fa926ffd700 1 -- 172.21.15.47:0/3273175536 >> v2:172.21.15.18:3300/0 conn(0x7fa928061bc0 msgr2=0x7fa92813c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa926ffd700 1 --2- 172.21.15.47:0/3273175536 >> v2:172.21.15.18:3300/0 conn(0x7fa928061bc0 0x7fa92813c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa926ffd700 1 -- 172.21.15.47:0/3273175536 >> v1:172.21.15.18:6789/0 conn(0x7fa9281548a0 legacy=0x7fa928154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa926ffd700 1 -- 172.21.15.47:0/3273175536 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928155400 con 0x7fa9280b5370 2020-07-18T03:34:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa9267fc700 1 --2- 172.21.15.47:0/3273175536 >> v2:172.21.15.18:3300/0 conn(0x7fa928061bc0 0x7fa92813c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa926ffd700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b5370 0x7fa92813bb60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fa9280b8490 tx=0x7fa91c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa91c009980 con 0x7fa9280b5370 2020-07-18T03:34:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c009ae0 con 0x7fa9280b5370 2020-07-18T03:34:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa928062420 con 0x7fa9280b5370 2020-07-18T03:34:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa91c009e40 con 0x7fa9280b5370 2020-07-18T03:34:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.026+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9280628f0 con 0x7fa9280b5370 2020-07-18T03:34:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.027+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928057450 con 0x7fa9280b5370 2020-07-18T03:34:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.027+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa91c013480 con 0x7fa9280b5370 2020-07-18T03:34:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.027+0000 7fa90effd700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa904050060 0x7fa904052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.027+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa91c067ff0 con 0x7fa9280b5370 2020-07-18T03:34:02.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.028+0000 7fa9267fc700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa904050060 0x7fa904052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:02.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.028+0000 7fa9267fc700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa904050060 0x7fa904052520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa918017000 tx=0x7fa918015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:02.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.031+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa91c0406d0 con 0x7fa9280b5370 2020-07-18T03:34:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.242+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9280b01a0 con 0x7fa9280b5370 2020-07-18T03:34:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.243+0000 7fa90effd700 1 -- 172.21.15.47:0/3273175536 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fa91c043cf0 con 0x7fa9280b5370 2020-07-18T03:34:02.246 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:02.246 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa904050060 msgr2=0x7fa904052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa904050060 0x7fa904052520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa918017000 tx=0x7fa918015000).stop 2020-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b5370 msgr2=0x7fa92813bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b5370 0x7fa92813bb60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fa9280b8490 tx=0x7fa91c008040).stop 2020-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 shutdown_connections 2020-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa904050060 0x7fa904052520 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 --2- 172.21.15.47:0/3273175536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9280b5370 0x7fa92813bb60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 --2- 172.21.15.47:0/3273175536 >> v2:172.21.15.18:3300/0 conn(0x7fa928061bc0 0x7fa92813c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.244+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 >> 172.21.15.47:0/3273175536 conn(0x7fa9280b9340 msgr2=0x7fa9280b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.245+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 shutdown_connections 2020-07-18T03:34:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:02.245+0000 7fa92d386700 1 -- 172.21.15.47:0/3273175536 wait complete. 2020-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:34:02.257 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:03.259 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.385+0000 7fd0f671d700 1 Processor -- start 2020-07-18T03:34:03.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.385+0000 7fd0f671d700 1 -- start start 2020-07-18T03:34:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f671d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f00ad100 0x7fd0f00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f671d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd0f0147a00 0x7fd0f0147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f671d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f0147330 con 0x7fd0f005f5a0 2020-07-18T03:34:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f671d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f00ae530 con 0x7fd0f0147a00 2020-07-18T03:34:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f671d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00b57e0 con 0x7fd0f00ad100 2020-07-18T03:34:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0ef7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd0f0147a00 0x7fd0f0147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f4cba700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd0f005f5a0 0x7fd0f005fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44678/0 (socket says 172.21.15.47:44678) 2020-07-18T03:34:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0f4cba700 1 -- 172.21.15.47:0/70523328 learned_addr learned my addr 172.21.15.47:0/70523328 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0effff700 1 --2- 172.21.15.47:0/70523328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f00ad100 0x7fd0f00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0ef7fe700 1 -- 172.21.15.47:0/70523328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f00ad100 msgr2=0x7fd0f00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0ef7fe700 1 --2- 172.21.15.47:0/70523328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f00ad100 0x7fd0f00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0ef7fe700 1 -- 172.21.15.47:0/70523328 >> v1:172.21.15.47:6789/0 conn(0x7fd0f005f5a0 legacy=0x7fd0f005fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.386+0000 7fd0ef7fe700 1 -- 172.21.15.47:0/70523328 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00b5a80 con 0x7fd0f0147a00 2020-07-18T03:34:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0ef7fe700 1 --2- 172.21.15.47:0/70523328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f0147a00 0x7fd0f0147e40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7fd0e4002f30 tx=0x7fd0e4006dd0).ready entity=mon.1 client_cookie=71cde5f8c234a1f5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0eeffd700 1 -- 172.21.15.47:0/70523328 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e40097f0 con 0x7fd0f0147a00 2020-07-18T03:34:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0eeffd700 1 -- 172.21.15.47:0/70523328 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e4009950 con 0x7fd0f0147a00 2020-07-18T03:34:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0eeffd700 1 -- 172.21.15.47:0/70523328 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e4009cb0 con 0x7fd0f0147a00 2020-07-18T03:34:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 -- 172.21.15.47:0/70523328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f0147a00 msgr2=0x7fd0f0147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 --2- 172.21.15.47:0/70523328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f0147a00 0x7fd0f0147e40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7fd0e4002f30 tx=0x7fd0e4006dd0).stop 2020-07-18T03:34:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 -- 172.21.15.47:0/70523328 shutdown_connections 2020-07-18T03:34:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 --2- 172.21.15.47:0/70523328 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f00ad100 0x7fd0f00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 --2- 172.21.15.47:0/70523328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f0147a00 0x7fd0f0147e40 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 -- 172.21.15.47:0/70523328 >> 172.21.15.47:0/70523328 conn(0x7fd0f00b9340 msgr2=0x7fd0f00b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:03.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 -- 172.21.15.47:0/70523328 shutdown_connections 2020-07-18T03:34:03.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.387+0000 7fd0f671d700 1 -- 172.21.15.47:0/70523328 wait complete. 2020-07-18T03:34:03.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.388+0000 7fd0f671d700 1 Processor -- start 2020-07-18T03:34:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f671d700 1 -- start start 2020-07-18T03:34:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f671d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 0x7fd0f0061ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f671d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd0f00ad100 0x7fd0f0138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f671d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f0061fc0 con 0x7fd0f01426d0 2020-07-18T03:34:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f671d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f0139ba0 con 0x7fd0f00ad100 2020-07-18T03:34:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f671d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0143a90 con 0x7fd0f005f5a0 2020-07-18T03:34:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f4cba700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd0f01426d0 0x7fd0f013b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44684/0 (socket says 172.21.15.47:44684) 2020-07-18T03:34:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.389+0000 7fd0f4cba700 1 -- 172.21.15.47:0/376363924 learned_addr learned my addr 172.21.15.47:0/376363924 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764681667 0 0) 0x7fd0f0061fc0 con 0x7fd0f01426d0 2020-07-18T03:34:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c8003240 con 0x7fd0f01426d0 2020-07-18T03:34:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0effff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 0x7fd0f0061ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0ef7fe700 1 --2- 172.21.15.47:0/376363924 >> v2:172.21.15.18:3300/0 conn(0x7fd0f00ad100 0x7fd0f0138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3616616274 0 0) 0x7fd0c8003240 con 0x7fd0f01426d0 2020-07-18T03:34:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd0f0061fc0 con 0x7fd0f01426d0 2020-07-18T03:34:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fd0e8001be0 con 0x7fd0f01426d0 2020-07-18T03:34:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 msgr2=0x7fd0f0061ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 0x7fd0f0061ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 >> v2:172.21.15.18:3300/0 conn(0x7fd0f00ad100 msgr2=0x7fd0f0138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 --2- 172.21.15.47:0/376363924 >> v2:172.21.15.18:3300/0 conn(0x7fd0f00ad100 0x7fd0f0138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 >> v1:172.21.15.47:6789/0 conn(0x7fd0f01426d0 legacy=0x7fd0f013b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0c8003800 0x7fd0c8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 0x7fd0c80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0c80049d0 0x7fd0c8004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0143a90 con 0x7fd0c8003800 2020-07-18T03:34:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c800b730 con 0x7fd0c80049d0 2020-07-18T03:34:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c800b870 con 0x7fd0f005f5a0 2020-07-18T03:34:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0f4cba700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0c80049d0 0x7fd0c8004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0effff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0c8003800 0x7fd0c8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.390+0000 7fd0ef7fe700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 0x7fd0c80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f4cba700 1 -- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 msgr2=0x7fd0c80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f4cba700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd0f005f5a0 0x7fd0c80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f4cba700 1 -- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0c8003800 msgr2=0x7fd0c8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f4cba700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0c8003800 0x7fd0c8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f4cba700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0146d70 con 0x7fd0c80049d0 2020-07-18T03:34:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f01394e0 con 0x7fd0c80049d0 2020-07-18T03:34:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f0143710 con 0x7fd0c80049d0 2020-07-18T03:34:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0f4cba700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0c80049d0 0x7fd0c8004e10 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fd0e80058b0 tx=0x7fd0e8007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e8005950 con 0x7fd0c80049d0 2020-07-18T03:34:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8005ab0 con 0x7fd0c80049d0 2020-07-18T03:34:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.391+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e8005e10 con 0x7fd0c80049d0 2020-07-18T03:34:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.392+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f0057450 con 0x7fd0c80049d0 2020-07-18T03:34:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.392+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd0e8010030 con 0x7fd0c80049d0 2020-07-18T03:34:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.392+0000 7fd0cffff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd0c8058530 0x7fd0c805a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.392+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd0e80684d0 con 0x7fd0c80049d0 2020-07-18T03:34:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.392+0000 7fd0effff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd0c8058530 0x7fd0c805a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.393+0000 7fd0effff700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd0c8058530 0x7fd0c805a9f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fd0e000c000 tx=0x7fd0e0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:03.412 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.396+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd0e8040b90 con 0x7fd0c80049d0 2020-07-18T03:34:03.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.606+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0f006a100 con 0x7fd0c80049d0 2020-07-18T03:34:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.606+0000 7fd0cffff700 1 -- 172.21.15.47:0/376363924 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fd0e80441b0 con 0x7fd0c80049d0 2020-07-18T03:34:03.609 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:03.609 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd0c8058530 msgr2=0x7fd0c805a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd0c8058530 0x7fd0c805a9f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fd0e000c000 tx=0x7fd0e0008040).stop 2020-07-18T03:34:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0c80049d0 msgr2=0x7fd0c8004e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 --2- 172.21.15.47:0/376363924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0c80049d0 0x7fd0c8004e10 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fd0e80058b0 tx=0x7fd0e8007c00).stop 2020-07-18T03:34:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f4cba700 1 -- 172.21.15.47:0/376363924 reap_dead start 2020-07-18T03:34:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f4cba700 1 -- 172.21.15.47:0/376363924 reap_dead start 2020-07-18T03:34:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 shutdown_connections 2020-07-18T03:34:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 >> 172.21.15.47:0/376363924 conn(0x7fd0f00b9340 msgr2=0x7fd0f00b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 shutdown_connections 2020-07-18T03:34:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:03.608+0000 7fd0f671d700 1 -- 172.21.15.47:0/376363924 wait complete. 2020-07-18T03:34:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:34:03.621 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:04.623 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:04.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.749+0000 7fd4ac939700 1 Processor -- start 2020-07-18T03:34:04.751 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.749+0000 7fd4ac939700 1 -- start start 2020-07-18T03:34:04.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4ac939700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80b4410 0x7fd4a8146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:04.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4ac939700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4a80ba600 0x7fd4a8061f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:04.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4ac939700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a80b72a0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4ac939700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a80b8da0 con 0x7fd4a80ba600 2020-07-18T03:34:04.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4ac939700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a80b7540 con 0x7fd4a80b4410 2020-07-18T03:34:04.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4a80ba600 0x7fd4a8061f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:04.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4a80ba600 0x7fd4a8061f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35378/0 (socket says 172.21.15.47:35378) 2020-07-18T03:34:04.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 -- 172.21.15.47:0/4010983678 learned_addr learned my addr 172.21.15.47:0/4010983678 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:04.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a77fe700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80b4410 0x7fd4a8146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:04.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 -- 172.21.15.47:0/4010983678 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80b4410 msgr2=0x7fd4a8146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:04.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80b4410 0x7fd4a8146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:04.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 -- 172.21.15.47:0/4010983678 >> v1:172.21.15.18:6789/0 conn(0x7fd4a80ba1c0 legacy=0x7fd4a8061800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:04.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 -- 172.21.15.47:0/4010983678 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8146fb0 con 0x7fd4a80ba600 2020-07-18T03:34:04.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a77fe700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80b4410 0x7fd4a8146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:04.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.750+0000 7fd4a7fff700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a80ba600 0x7fd4a8061f10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fd4a80b5740 tx=0x7fd4a0009270).ready entity=mon.1 client_cookie=f22addf16ffa86d9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:04.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4a67fc700 1 -- 172.21.15.47:0/4010983678 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4a000bbc0 con 0x7fd4a80ba600 2020-07-18T03:34:04.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4a67fc700 1 -- 172.21.15.47:0/4010983678 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a000bd20 con 0x7fd4a80ba600 2020-07-18T03:34:04.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4a67fc700 1 -- 172.21.15.47:0/4010983678 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4a0011400 con 0x7fd4a80ba600 2020-07-18T03:34:04.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 -- 172.21.15.47:0/4010983678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a80ba600 msgr2=0x7fd4a8061f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:04.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a80ba600 0x7fd4a8061f10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fd4a80b5740 tx=0x7fd4a0009270).stop 2020-07-18T03:34:04.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 -- 172.21.15.47:0/4010983678 shutdown_connections 2020-07-18T03:34:04.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a80ba600 0x7fd4a8061f10 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:04.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 --2- 172.21.15.47:0/4010983678 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80b4410 0x7fd4a8146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:04.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 -- 172.21.15.47:0/4010983678 >> 172.21.15.47:0/4010983678 conn(0x7fd4a805f5a0 msgr2=0x7fd4a805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:04.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 -- 172.21.15.47:0/4010983678 shutdown_connections 2020-07-18T03:34:04.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.751+0000 7fd4ac939700 1 -- 172.21.15.47:0/4010983678 wait complete. 2020-07-18T03:34:04.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.752+0000 7fd4ac939700 1 Processor -- start 2020-07-18T03:34:04.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4ac939700 1 -- start start 2020-07-18T03:34:04.762 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4ac939700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80ba1c0 0x7fd4a8159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:04.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4ac939700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a8159580 con 0x7fd4a80b4410 2020-07-18T03:34:04.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4ac939700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a81551f0 con 0x7fd4a80b0c20 2020-07-18T03:34:04.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4ac939700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8155490 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4a7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd4a80b0c20 0x7fd4a814b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44704/0 (socket says 172.21.15.47:44704) 2020-07-18T03:34:04.763 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4a7fff700 1 -- 172.21.15.47:0/2515183970 learned_addr learned my addr 172.21.15.47:0/2515183970 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.753+0000 7fd4a6ffd700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80ba1c0 0x7fd4a8159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440677678 0 0) 0x7fd4a81551f0 con 0x7fd4a80b0c20 2020-07-18T03:34:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd480003240 con 0x7fd4a80b0c20 2020-07-18T03:34:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578436600 0 0) 0x7fd4a8159580 con 0x7fd4a80b4410 2020-07-18T03:34:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a81551f0 con 0x7fd4a80b4410 2020-07-18T03:34:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4a6ffd700 1 -- 172.21.15.47:0/2515183970 >> v1:172.21.15.47:6789/0 conn(0x7fd4a80b0c20 legacy=0x7fd4a814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4a6ffd700 1 -- 172.21.15.47:0/2515183970 >> v1:172.21.15.18:6789/0 conn(0x7fd4a80b4410 legacy=0x7fd4a81549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1698131199 0 0) 0x7fd480003240 con 0x7fd4a80b0c20 2020-07-18T03:34:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4a6ffd700 1 -- 172.21.15.47:0/2515183970 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8155730 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4a6ffd700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80ba1c0 0x7fd4a8159090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fd49c001f50 tx=0x7fd49c007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49c013410 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c013570 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49c0138d0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a814b8b0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.754+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a8155e10 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.755+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a81560f0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.756+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd49c010030 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.756+0000 7fd4877fe700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd480050490 0x7fd480052950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.756+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd49c0675d0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.756+0000 7fd4a77fe700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd480050490 0x7fd480052950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.756+0000 7fd4a77fe700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd480050490 0x7fd480052950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fd498002f30 tx=0x7fd498008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.760+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd4a81560f0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.980+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a81560f0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.980+0000 7fd4877fe700 1 -- 172.21.15.47:0/2515183970 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7fd4a81560f0 con 0x7fd4a80ba1c0 2020-07-18T03:34:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:04.983 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:04.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd480050490 msgr2=0x7fd480052950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:04.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd480050490 0x7fd480052950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fd498002f30 tx=0x7fd498008040).stop 2020-07-18T03:34:04.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80ba1c0 msgr2=0x7fd4a8159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:04.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80ba1c0 0x7fd4a8159090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fd49c001f50 tx=0x7fd49c007b70).stop 2020-07-18T03:34:04.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 shutdown_connections 2020-07-18T03:34:04.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd480050490 0x7fd480052950 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:04.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 --2- 172.21.15.47:0/2515183970 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd4a80ba1c0 0x7fd4a8159090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:04.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 >> 172.21.15.47:0/2515183970 conn(0x7fd4a805f5a0 msgr2=0x7fd4a805d380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:04.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 shutdown_connections 2020-07-18T03:34:04.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:04.982+0000 7fd4ac939700 1 -- 172.21.15.47:0/2515183970 wait complete. 2020-07-18T03:34:04.986 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:34:04.994 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:05.996 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:06.122 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.120+0000 7f048856b700 1 Processor -- start 2020-07-18T03:34:06.123 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.120+0000 7f048856b700 1 -- start start 2020-07-18T03:34:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f048856b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f048005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f048856b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f04800b41c0 0x7f04800ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f048856b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04800b3ac0 con 0x7f04800b4620 2020-07-18T03:34:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f048856b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04800b5a10 con 0x7f04800b41c0 2020-07-18T03:34:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f048856b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04800b0040 con 0x7f04800b5370 2020-07-18T03:34:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f0486b08700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f04800b4620 0x7f048005f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44708/0 (socket says 172.21.15.47:44708) 2020-07-18T03:34:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.121+0000 7f0486b08700 1 -- 172.21.15.47:0/2088414684 learned_addr learned my addr 172.21.15.47:0/2088414684 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 --2- 172.21.15.47:0/2088414684 >> v2:172.21.15.18:3300/0 conn(0x7f04800b41c0 0x7f04800ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0486307700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f048005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216984387 0 0) 0x7f04800b3ac0 con 0x7f04800b4620 2020-07-18T03:34:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f046c003240 con 0x7f04800b4620 2020-07-18T03:34:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 637978010 0 0) 0x7f046c003240 con 0x7f04800b4620 2020-07-18T03:34:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 msgr2=0x7f048005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f048005fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 >> v2:172.21.15.18:3300/0 conn(0x7f04800b41c0 msgr2=0x7f04800ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 --2- 172.21.15.47:0/2088414684 >> v2:172.21.15.18:3300/0 conn(0x7f04800b41c0 0x7f04800ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800b3d20 con 0x7f04800b4620 2020-07-18T03:34:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f04700016d0 con 0x7f04800b4620 2020-07-18T03:34:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 >> v1:172.21.15.47:6789/0 conn(0x7f04800b4620 legacy=0x7f048005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f046c003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f046c003c40 0x7f046c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f046c0049d0 0x7f046c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04800b5a10 con 0x7f046c0049d0 2020-07-18T03:34:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04800b0040 con 0x7f046c003c40 2020-07-18T03:34:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046c004200 con 0x7f04800b5370 2020-07-18T03:34:06.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f046c003c40 0x7f046c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0486307700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f046c003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0486b08700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f046c0049d0 0x7f046c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 -- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 msgr2=0x7f046c003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f046c003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 -- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f046c0049d0 msgr2=0x7f046c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f046c0049d0 0x7f046c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0485b06700 1 -- 172.21.15.47:0/2088414684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c003240 con 0x7f046c003c40 2020-07-18T03:34:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.122+0000 7f0486307700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f04800b5370 0x7f046c003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f0485b06700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f046c003c40 0x7f046c0044e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f047400c000 tx=0x7f0474008040).ready entity=mon.1 client_cookie=b80c6c680f13580a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047400ebc0 con 0x7f046c003c40 2020-07-18T03:34:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047400ed20 con 0x7f046c003c40 2020-07-18T03:34:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f0485305700 1 -- 172.21.15.47:0/2088414684 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04740125e0 con 0x7f046c003c40 2020-07-18T03:34:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f048856b700 1 -- 172.21.15.47:0/2088414684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f046c003c40 msgr2=0x7f046c0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f048856b700 1 --2- 172.21.15.47:0/2088414684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f046c003c40 0x7f046c0044e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f047400c000 tx=0x7f0474008040).stop 2020-07-18T03:34:06.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f0486b08700 1 -- 172.21.15.47:0/2088414684 reap_dead start 2020-07-18T03:34:06.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f048856b700 1 -- 172.21.15.47:0/2088414684 shutdown_connections 2020-07-18T03:34:06.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f048856b700 1 -- 172.21.15.47:0/2088414684 >> 172.21.15.47:0/2088414684 conn(0x7f04800b9340 msgr2=0x7f04800b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:06.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f048856b700 1 -- 172.21.15.47:0/2088414684 shutdown_connections 2020-07-18T03:34:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.123+0000 7f048856b700 1 -- 172.21.15.47:0/2088414684 wait complete. 2020-07-18T03:34:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.124+0000 7f048856b700 1 Processor -- start 2020-07-18T03:34:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.124+0000 7f048856b700 1 -- start start 2020-07-18T03:34:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f048856b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0480134820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f048856b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f0480134d10 0x7f0480063450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f048856b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0480062840 con 0x7f0480154320 2020-07-18T03:34:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f048856b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0480150c10 con 0x7f0480134d10 2020-07-18T03:34:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f048856b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0480062a70 con 0x7f0480154700 2020-07-18T03:34:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f0486307700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0480154320 0x7f0480154b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44720/0 (socket says 172.21.15.47:44720) 2020-07-18T03:34:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f0486307700 1 -- 172.21.15.47:0/1818013858 learned_addr learned my addr 172.21.15.47:0/1818013858 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644576604 0 0) 0x7f0480062840 con 0x7f0480154320 2020-07-18T03:34:06.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0458003240 con 0x7f0480154320 2020-07-18T03:34:06.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f0486b08700 1 --2- 172.21.15.47:0/1818013858 >> v2:172.21.15.18:3300/0 conn(0x7f0480134d10 0x7f0480063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.125+0000 7f0485b06700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0480134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3467071447 0 0) 0x7f0458003240 con 0x7f0480154320 2020-07-18T03:34:06.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0480062840 con 0x7f0480154320 2020-07-18T03:34:06.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f0478009bc0 con 0x7f0480154320 2020-07-18T03:34:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 msgr2=0x7f0480134820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0480134820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 >> v2:172.21.15.18:3300/0 conn(0x7f0480134d10 msgr2=0x7f0480063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 --2- 172.21.15.47:0/1818013858 >> v2:172.21.15.18:3300/0 conn(0x7f0480134d10 0x7f0480063450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 >> v1:172.21.15.47:6789/0 conn(0x7f0480154320 legacy=0x7f0480154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0458003800 0x7f0458003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04580044e0 0x7f0458004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0458004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0480062a70 con 0x7f04580044e0 2020-07-18T03:34:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458005300 con 0x7f0458003800 2020-07-18T03:34:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045800b730 con 0x7f0480154700 2020-07-18T03:34:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0458003800 0x7f0458003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0485b06700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04580044e0 0x7f0458004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486b08700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0458004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 -- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 msgr2=0x7f0458004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0458004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 -- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04580044e0 msgr2=0x7f0458004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04580044e0 0x7f0458004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480062d10 con 0x7f0458003800 2020-07-18T03:34:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486b08700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f0480154700 0x7f0458004e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0480062fb0 con 0x7f0458003800 2020-07-18T03:34:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0480134820 con 0x7f0458003800 2020-07-18T03:34:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.126+0000 7f0486307700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0458003800 0x7f0458003c20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f047800a000 tx=0x7f0478008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.127+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047800c8a0 con 0x7f0458003800 2020-07-18T03:34:06.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.127+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047800ca00 con 0x7f0458003800 2020-07-18T03:34:06.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.127+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f047800cd60 con 0x7f0458003800 2020-07-18T03:34:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.127+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0480134b00 con 0x7f0458003800 2020-07-18T03:34:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.128+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f0478014470 con 0x7f0458003800 2020-07-18T03:34:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.128+0000 7f047e7fc700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f04580583a0 0x7f045805a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.128+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f04780695f0 con 0x7f0458003800 2020-07-18T03:34:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.128+0000 7f0485b06700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f04580583a0 0x7f045805a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.128+0000 7f0485b06700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f04580583a0 0x7f045805a860 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f047401c000 tx=0x7f0474019000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.132+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f0478041cd0 con 0x7f0458003800 2020-07-18T03:34:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.340+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0480057450 con 0x7f0458003800 2020-07-18T03:34:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.340+0000 7f047e7fc700 1 -- 172.21.15.47:0/1818013858 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f04780452f0 con 0x7f0458003800 2020-07-18T03:34:06.344 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:06.344 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f04580583a0 msgr2=0x7f045805a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f04580583a0 0x7f045805a860 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f047401c000 tx=0x7f0474019000).stop 2020-07-18T03:34:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0458003800 msgr2=0x7f0458003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 --2- 172.21.15.47:0/1818013858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0458003800 0x7f0458003c20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f047800a000 tx=0x7f0478008040).stop 2020-07-18T03:34:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f0486b08700 1 -- 172.21.15.47:0/1818013858 reap_dead start 2020-07-18T03:34:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f0486b08700 1 -- 172.21.15.47:0/1818013858 reap_dead start 2020-07-18T03:34:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 shutdown_connections 2020-07-18T03:34:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 >> 172.21.15.47:0/1818013858 conn(0x7f04800b9340 msgr2=0x7f04800b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 shutdown_connections 2020-07-18T03:34:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:06.342+0000 7f048856b700 1 -- 172.21.15.47:0/1818013858 wait complete. 2020-07-18T03:34:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:34:06.354 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:07.356 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 Processor -- start 2020-07-18T03:34:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 -- start start 2020-07-18T03:34:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f040ad100 0x7f7f040b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f041443f0 0x7f7f04147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f04147330 con 0x7f7f04147a00 2020-07-18T03:34:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040ae530 con 0x7f7f041443f0 2020-07-18T03:34:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.483+0000 7f7f0a973700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040b57e0 con 0x7f7f040ad100 2020-07-18T03:34:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f08f10700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f041443f0 0x7f7f04147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f03fff700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f040ad100 0x7f7f040b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f037fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f7f04147a00 0x7f7f0405f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35268/0 (socket says 172.21.15.47:35268) 2020-07-18T03:34:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f037fe700 1 -- 172.21.15.47:0/2860542112 learned_addr learned my addr 172.21.15.47:0/2860542112 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f08f10700 1 -- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f040ad100 msgr2=0x7f7f040b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f08f10700 1 --2- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f040ad100 0x7f7f040b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f08f10700 1 -- 172.21.15.47:0/2860542112 >> v1:172.21.15.18:6789/0 conn(0x7f7f04147a00 legacy=0x7f7f0405f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f08f10700 1 -- 172.21.15.47:0/2860542112 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040b5a80 con 0x7f7f041443f0 2020-07-18T03:34:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f02ffd700 1 -- 172.21.15.47:0/2860542112 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426979973 0 0) 0x7f7f04147330 con 0x7f7f04147a00 2020-07-18T03:34:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f03fff700 1 --2- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f040ad100 0x7f7f040b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.484+0000 7f7f08f10700 1 --2- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f041443f0 0x7f7f04147e40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f7efc0032b0 tx=0x7f7efc0091d0).ready entity=mon.0 client_cookie=1d56c47de1a1a62b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f02ffd700 1 -- 172.21.15.47:0/2860542112 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7efc00b9a0 con 0x7f7f041443f0 2020-07-18T03:34:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f02ffd700 1 -- 172.21.15.47:0/2860542112 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00bb00 con 0x7f7f041443f0 2020-07-18T03:34:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f02ffd700 1 -- 172.21.15.47:0/2860542112 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7efc00bc70 con 0x7f7f041443f0 2020-07-18T03:34:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 -- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f041443f0 msgr2=0x7f7f04147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 --2- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f041443f0 0x7f7f04147e40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f7efc0032b0 tx=0x7f7efc0091d0).stop 2020-07-18T03:34:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 -- 172.21.15.47:0/2860542112 shutdown_connections 2020-07-18T03:34:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 --2- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f041443f0 0x7f7f04147e40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 --2- 172.21.15.47:0/2860542112 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7f040ad100 0x7f7f040b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 -- 172.21.15.47:0/2860542112 >> 172.21.15.47:0/2860542112 conn(0x7f7f040b9340 msgr2=0x7f7f040b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 -- 172.21.15.47:0/2860542112 shutdown_connections 2020-07-18T03:34:07.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.485+0000 7f7f0a973700 1 -- 172.21.15.47:0/2860542112 wait complete. 2020-07-18T03:34:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.486+0000 7f7f0a973700 1 Processor -- start 2020-07-18T03:34:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.486+0000 7f7f0a973700 1 -- start start 2020-07-18T03:34:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f0a973700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f0413a0c0 0x7f7f040661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f0a973700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f04064980 con 0x7f7f04147a00 2020-07-18T03:34:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f0a973700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f04064c20 con 0x7f7f040ad100 2020-07-18T03:34:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f0a973700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f04064360 con 0x7f7f0413a0c0 2020-07-18T03:34:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f03fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7f040ad100 0x7f7f041485b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44736/0 (socket says 172.21.15.47:44736) 2020-07-18T03:34:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f03fff700 1 -- 172.21.15.47:0/3509348211 learned_addr learned my addr 172.21.15.47:0/3509348211 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:07.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7f08f10700 1 --2- 172.21.15.47:0/3509348211 >> v2:172.21.15.18:3300/0 conn(0x7f7f0413a0c0 0x7f7f040661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:07.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235008896 0 0) 0x7f7f04064c20 con 0x7f7f040ad100 2020-07-18T03:34:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.487+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc003240 con 0x7f7f040ad100 2020-07-18T03:34:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565012129 0 0) 0x7f7f04064980 con 0x7f7f04147a00 2020-07-18T03:34:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f04064c20 con 0x7f7f04147a00 2020-07-18T03:34:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7f08f10700 1 -- 172.21.15.47:0/3509348211 >> v1:172.21.15.47:6789/0 conn(0x7f7f040ad100 legacy=0x7f7f041485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7f08f10700 1 -- 172.21.15.47:0/3509348211 >> v1:172.21.15.18:6789/0 conn(0x7f7f04147a00 legacy=0x7f7f04156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7f08f10700 1 -- 172.21.15.47:0/3509348211 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040644a0 con 0x7f7f0413a0c0 2020-07-18T03:34:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7f08f10700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f0413a0c0 0x7f7f040661e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f7efc0032b0 tx=0x7f7efc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7efc0107a0 con 0x7f7f0413a0c0 2020-07-18T03:34:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f040636e0 con 0x7f7f0413a0c0 2020-07-18T03:34:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc010900 con 0x7f7f0413a0c0 2020-07-18T03:34:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7efc010c60 con 0x7f7f0413a0c0 2020-07-18T03:34:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.488+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f04156e40 con 0x7f7f0413a0c0 2020-07-18T03:34:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.489+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f04064df0 con 0x7f7f0413a0c0 2020-07-18T03:34:07.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.489+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7efc027030 con 0x7f7f0413a0c0 2020-07-18T03:34:07.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.490+0000 7f7ee3fff700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7edc050440 0x7f7edc052900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:07.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.490+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f7efc068c70 con 0x7f7f0413a0c0 2020-07-18T03:34:07.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.490+0000 7f7f03fff700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7edc050440 0x7f7edc052900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:07.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.490+0000 7f7f03fff700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7edc050440 0x7f7edc052900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f7ef400a000 tx=0x7f7ef4006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:07.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.494+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7efc044d30 con 0x7f7f0413a0c0 2020-07-18T03:34:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.705+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f04057450 con 0x7f7f0413a0c0 2020-07-18T03:34:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.705+0000 7f7ee3fff700 1 -- 172.21.15.47:0/3509348211 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2140 (secure 0 0 0) 0x7f7efc015090 con 0x7f7f0413a0c0 2020-07-18T03:34:07.708 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:07.708 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:33:18.448750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7edc050440 msgr2=0x7f7edc052900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7edc050440 0x7f7edc052900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f7ef400a000 tx=0x7f7ef4006dd0).stop 2020-07-18T03:34:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f0413a0c0 msgr2=0x7f7f040661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f0413a0c0 0x7f7f040661e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f7efc0032b0 tx=0x7f7efc016000).stop 2020-07-18T03:34:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 shutdown_connections 2020-07-18T03:34:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f0413a0c0 0x7f7f040661e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 --2- 172.21.15.47:0/3509348211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7edc050440 0x7f7edc052900 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 >> 172.21.15.47:0/3509348211 conn(0x7f7f040b9340 msgr2=0x7f7f041469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 shutdown_connections 2020-07-18T03:34:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:07.707+0000 7f7f0a973700 1 -- 172.21.15.47:0/3509348211 wait complete. 2020-07-18T03:34:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2020-07-18T03:34:07.721 INFO:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2020-07-18T03:34:08.723 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.850+0000 7fbd56c68700 1 Processor -- start 2020-07-18T03:34:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.850+0000 7fbd56c68700 1 -- start start 2020-07-18T03:34:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.850+0000 7fbd56c68700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd500ad9e0 0x7fbd501428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.850+0000 7fbd56c68700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd500b3ac0 con 0x7fbd500b41c0 2020-07-18T03:34:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.850+0000 7fbd56c68700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd500b3d20 con 0x7fbd500b5370 2020-07-18T03:34:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.850+0000 7fbd56c68700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd500b5a10 con 0x7fbd500ad9e0 2020-07-18T03:34:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd500ad9e0 0x7fbd501428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd500ad9e0 0x7fbd501428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35428/0 (socket says 172.21.15.47:35428) 2020-07-18T03:34:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 -- 172.21.15.47:0/3556953754 learned_addr learned my addr 172.21.15.47:0/3556953754 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482429241 0 0) 0x7fbd500b3d20 con 0x7fbd500b5370 2020-07-18T03:34:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd34003240 con 0x7fbd500b5370 2020-07-18T03:34:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927924223 0 0) 0x7fbd500b3ac0 con 0x7fbd500b41c0 2020-07-18T03:34:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd500b3d20 con 0x7fbd500b41c0 2020-07-18T03:34:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 -- 172.21.15.47:0/3556953754 >> v1:172.21.15.47:6789/0 conn(0x7fbd500b5370 legacy=0x7fbd5005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 -- 172.21.15.47:0/3556953754 >> v1:172.21.15.18:6789/0 conn(0x7fbd500b41c0 legacy=0x7fbd500ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 -- 172.21.15.47:0/3556953754 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500b5750 con 0x7fbd500ad9e0 2020-07-18T03:34:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd55205700 1 --2- 172.21.15.47:0/3556953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd500ad9e0 0x7fbd501428f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fbd500afb30 tx=0x7fbd48009270).ready entity=mon.1 client_cookie=4ce364fa9668b560 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd4800bd00 con 0x7fbd500ad9e0 2020-07-18T03:34:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.851+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48009e90 con 0x7fbd500ad9e0 2020-07-18T03:34:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd4f7fe700 1 -- 172.21.15.47:0/3556953754 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd48010600 con 0x7fbd500ad9e0 2020-07-18T03:34:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 -- 172.21.15.47:0/3556953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd500ad9e0 msgr2=0x7fbd501428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 --2- 172.21.15.47:0/3556953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd500ad9e0 0x7fbd501428f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fbd500afb30 tx=0x7fbd48009270).stop 2020-07-18T03:34:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 -- 172.21.15.47:0/3556953754 shutdown_connections 2020-07-18T03:34:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 --2- 172.21.15.47:0/3556953754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd500ad9e0 0x7fbd501428f0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 -- 172.21.15.47:0/3556953754 >> 172.21.15.47:0/3556953754 conn(0x7fbd500b9340 msgr2=0x7fbd500b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 -- 172.21.15.47:0/3556953754 shutdown_connections 2020-07-18T03:34:08.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.852+0000 7fbd56c68700 1 -- 172.21.15.47:0/3556953754 wait complete. 2020-07-18T03:34:08.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.853+0000 7fbd56c68700 1 Processor -- start 2020-07-18T03:34:08.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.853+0000 7fbd56c68700 1 -- start start 2020-07-18T03:34:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd56c68700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbd500b5370 0x7fbd5014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:08.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd56c68700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd50131b30 0x7fbd50131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:08.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd56c68700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd50132490 con 0x7fbd500b41c0 2020-07-18T03:34:08.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd56c68700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd50066710 con 0x7fbd500b5370 2020-07-18T03:34:08.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd56c68700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd50061db0 con 0x7fbd50131b30 2020-07-18T03:34:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd54a04700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbd500b41c0 0x7fbd5013d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44750/0 (socket says 172.21.15.47:44750) 2020-07-18T03:34:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd54a04700 1 -- 172.21.15.47:0/551343003 learned_addr learned my addr 172.21.15.47:0/551343003 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.854+0000 7fbd55205700 1 --2- 172.21.15.47:0/551343003 >> v2:172.21.15.47:3300/0 conn(0x7fbd50131b30 0x7fbd50131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235912203 0 0) 0x7fbd34003240 con 0x7fbd500b41c0 2020-07-18T03:34:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd28003240 con 0x7fbd500b41c0 2020-07-18T03:34:08.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd55205700 1 -- 172.21.15.47:0/551343003 >> v2:172.21.15.18:3300/0 conn(0x7fbd500b5370 msgr2=0x7fbd5014aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:08.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd55205700 1 --2- 172.21.15.47:0/551343003 >> v2:172.21.15.18:3300/0 conn(0x7fbd500b5370 0x7fbd5014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-07-18T03:34:08.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd55205700 1 -- 172.21.15.47:0/551343003 >> v1:172.21.15.47:6789/0 conn(0x7fbd500b41c0 legacy=0x7fbd5013d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:08.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd55205700 1 -- 172.21.15.47:0/551343003 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd50064380 con 0x7fbd50131b30 2020-07-18T03:34:08.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd55205700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd50131b30 0x7fbd50131fa0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fbd500afb30 tx=0x7fbd48017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:08.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd4801a880 con 0x7fbd50131b30 2020-07-18T03:34:08.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4801a9e0 con 0x7fbd50131b30 2020-07-18T03:34:08.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd500662a0 con 0x7fbd50131b30 2020-07-18T03:34:08.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd4801ad40 con 0x7fbd50131b30 2020-07-18T03:34:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.855+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd50062490 con 0x7fbd50131b30 2020-07-18T03:34:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.856+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd50057450 con 0x7fbd50131b30 2020-07-18T03:34:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.856+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbd4802a070 con 0x7fbd50131b30 2020-07-18T03:34:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.857+0000 7fbd4cff9700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbd280503f0 0x7fbd280528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.857+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fbd480699b0 con 0x7fbd50131b30 2020-07-18T03:34:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.857+0000 7fbd54a04700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbd280503f0 0x7fbd280528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.857+0000 7fbd54a04700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbd280503f0 0x7fbd280528b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fbd440010d0 tx=0x7fbd440026d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:08.860+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbd480420c0 con 0x7fbd50131b30 2020-07-18T03:34:09.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.078+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd50066b60 con 0x7fbd50131b30 2020-07-18T03:34:09.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.079+0000 7fbd4cff9700 1 -- 172.21.15.47:0/551343003 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fbd48014070 con 0x7fbd50131b30 2020-07-18T03:34:09.081 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:09.082 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:34:08.307773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbd280503f0 msgr2=0x7fbd280528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbd280503f0 0x7fbd280528b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fbd440010d0 tx=0x7fbd440026d0).stop 2020-07-18T03:34:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd50131b30 msgr2=0x7fbd50131fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd50131b30 0x7fbd50131fa0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fbd500afb30 tx=0x7fbd48017000).stop 2020-07-18T03:34:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 shutdown_connections 2020-07-18T03:34:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd50131b30 0x7fbd50131fa0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 --2- 172.21.15.47:0/551343003 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbd280503f0 0x7fbd280528b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 --2- 172.21.15.47:0/551343003 >> v2:172.21.15.18:3300/0 conn(0x7fbd500b5370 0x7fbd5014aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-07-18T03:34:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 >> 172.21.15.47:0/551343003 conn(0x7fbd500b9340 msgr2=0x7fbd500b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 shutdown_connections 2020-07-18T03:34:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:09.080+0000 7fbd56c68700 1 -- 172.21.15.47:0/551343003 wait complete. 2020-07-18T03:34:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2020-07-18T03:34:09.093 INFO:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2020-07-18T03:34:10.095 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:10.223 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.221+0000 7fdfb8134700 1 Processor -- start 2020-07-18T03:34:10.224 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.221+0000 7fdfb8134700 1 -- start start 2020-07-18T03:34:10.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb8134700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00637d0 0x7fdfb0063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb8134700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb0146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb8134700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb00b6470 con 0x7fdfb00b5d30 2020-07-18T03:34:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb8134700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb00b61d0 con 0x7fdfb01464c0 2020-07-18T03:34:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb8134700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb014a5b0 con 0x7fdfb00637d0 2020-07-18T03:34:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb56cf700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00637d0 0x7fdfb0063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb66d1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb0146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb5ed0700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fdfb00b5d30 0x7fdfb00ba460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35296/0 (socket says 172.21.15.47:35296) 2020-07-18T03:34:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.222+0000 7fdfb5ed0700 1 -- 172.21.15.47:0/989125939 learned_addr learned my addr 172.21.15.47:0/989125939 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb4ece700 1 -- 172.21.15.47:0/989125939 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962768723 0 0) 0x7fdfb00b6470 con 0x7fdfb00b5d30 2020-07-18T03:34:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb4ece700 1 -- 172.21.15.47:0/989125939 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf98003240 con 0x7fdfb00b5d30 2020-07-18T03:34:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb56cf700 1 -- 172.21.15.47:0/989125939 >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 msgr2=0x7fdfb0146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb56cf700 1 --2- 172.21.15.47:0/989125939 >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb0146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb56cf700 1 -- 172.21.15.47:0/989125939 >> v1:172.21.15.18:6789/0 conn(0x7fdfb00b5d30 legacy=0x7fdfb00ba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb56cf700 1 -- 172.21.15.47:0/989125939 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0066ac0 con 0x7fdfb00637d0 2020-07-18T03:34:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb56cf700 1 --2- 172.21.15.47:0/989125939 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00637d0 0x7fdfb0063c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fdfac0032b0 tx=0x7fdfac0091d0).ready entity=mon.2 client_cookie=16f349f021298891 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb4ece700 1 -- 172.21.15.47:0/989125939 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfac00b940 con 0x7fdfb00637d0 2020-07-18T03:34:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb4ece700 1 -- 172.21.15.47:0/989125939 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac00baa0 con 0x7fdfb00637d0 2020-07-18T03:34:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb4ece700 1 -- 172.21.15.47:0/989125939 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfac00bc10 con 0x7fdfb00637d0 2020-07-18T03:34:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 -- 172.21.15.47:0/989125939 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00637d0 msgr2=0x7fdfb0063c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 --2- 172.21.15.47:0/989125939 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00637d0 0x7fdfb0063c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fdfac0032b0 tx=0x7fdfac0091d0).stop 2020-07-18T03:34:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 -- 172.21.15.47:0/989125939 shutdown_connections 2020-07-18T03:34:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 --2- 172.21.15.47:0/989125939 >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb0146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 --2- 172.21.15.47:0/989125939 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00637d0 0x7fdfb0063c10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 -- 172.21.15.47:0/989125939 >> 172.21.15.47:0/989125939 conn(0x7fdfb00b7100 msgr2=0x7fdfb00b9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 -- 172.21.15.47:0/989125939 shutdown_connections 2020-07-18T03:34:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.224+0000 7fdfb8134700 1 -- 172.21.15.47:0/989125939 wait complete. 2020-07-18T03:34:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.225+0000 7fdfb8134700 1 Processor -- start 2020-07-18T03:34:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.225+0000 7fdfb8134700 1 -- start start 2020-07-18T03:34:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb8134700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb01550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb8134700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 0x7fdfb0143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb8134700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb0155770 con 0x7fdfb00b5d30 2020-07-18T03:34:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb8134700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb0155a10 con 0x7fdfb01464c0 2020-07-18T03:34:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb8134700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0155cb0 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 0x7fdfb0143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb56cf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb01550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 0x7fdfb0143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56282/0 (socket says 172.21.15.47:56282) 2020-07-18T03:34:10.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 -- 172.21.15.47:0/2210000945 learned_addr learned my addr 172.21.15.47:0/2210000945 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:10.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239509020 0 0) 0x7fdfb0155770 con 0x7fdfb00b5d30 2020-07-18T03:34:10.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf88003240 con 0x7fdfb00b5d30 2020-07-18T03:34:10.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 -- 172.21.15.47:0/2210000945 >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 msgr2=0x7fdfb01550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 --2- 172.21.15.47:0/2210000945 >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb01550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 -- 172.21.15.47:0/2210000945 >> v1:172.21.15.18:6789/0 conn(0x7fdfb00b5d30 legacy=0x7fdfb0112f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.226+0000 7fdfb66d1700 1 -- 172.21.15.47:0/2210000945 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0115fa0 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.242 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.227+0000 7fdfb66d1700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 0x7fdfb0143d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdfa800c000 tx=0x7fdfa8006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.227+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfa800ca10 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.227+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa800cb70 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.227+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb0144260 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.243 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.227+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfa800ee80 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.227+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb0114f90 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.228+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb01555c0 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.228+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fdfa801c460 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.228+0000 7fdf9dffb700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdf880503e0 0x7fdf880528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.228+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fdfa8068110 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.230+0000 7fdfb5ed0700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdf880503e0 0x7fdf880528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.230+0000 7fdfb5ed0700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdf880503e0 0x7fdf880528a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fdfa0001110 tx=0x7fdfa0007880).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.233+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fdfa80407c0 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.444 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.443+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfb0057450 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.445 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.444+0000 7fdf9dffb700 1 -- 172.21.15.47:0/2210000945 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fdfa8043de0 con 0x7fdfb00b3cc0 2020-07-18T03:34:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:10.446 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:34:09.311823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:10.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdf880503e0 msgr2=0x7fdf880528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdf880503e0 0x7fdf880528a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fdfa0001110 tx=0x7fdfa0007880).stop 2020-07-18T03:34:10.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 msgr2=0x7fdfb0143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 0x7fdfb0143d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdfa800c000 tx=0x7fdfa8006dd0).stop 2020-07-18T03:34:10.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 shutdown_connections 2020-07-18T03:34:10.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fdf880503e0 0x7fdf880528a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 --2- 172.21.15.47:0/2210000945 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdfb00b3cc0 0x7fdfb0143d70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 --2- 172.21.15.47:0/2210000945 >> v2:172.21.15.18:3300/0 conn(0x7fdfb01464c0 0x7fdfb01550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 >> 172.21.15.47:0/2210000945 conn(0x7fdfb00b7100 msgr2=0x7fdfb0149ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:10.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 shutdown_connections 2020-07-18T03:34:10.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.445+0000 7fdfb8134700 1 -- 172.21.15.47:0/2210000945 wait complete. 2020-07-18T03:34:10.452 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 36 2020-07-18T03:34:10.464 INFO:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2020-07-18T03:34:10.465 INFO:tasks.cephfs.filesystem:reached state 'up:active' in 50.834195375442505s 2020-07-18T03:34:10.466 INFO:tasks.cephfs.test_client_recovery:Reached active... 2020-07-18T03:34:10.467 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:10.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.581+0000 7f552d975700 1 Processor -- start 2020-07-18T03:34:10.583 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.581+0000 7f552d975700 1 -- start start 2020-07-18T03:34:10.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f552d975700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f552d975700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55280641d0 0x7f552814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f552d975700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5528060ff0 con 0x7f5528063d20 2020-07-18T03:34:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f552d975700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5528147530 con 0x7f5528147150 2020-07-18T03:34:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f552d975700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5528147740 con 0x7f55280641d0 2020-07-18T03:34:10.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f55277fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55280641d0 0x7f552814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f55277fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55280641d0 0x7f552814ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35460/0 (socket says 172.21.15.47:35460) 2020-07-18T03:34:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f55277fe700 1 -- 172.21.15.47:0/2966539398 learned_addr learned my addr 172.21.15.47:0/2966539398 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f5526ffd700 1 --2- 172.21.15.47:0/2966539398 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f55277fe700 1 -- 172.21.15.47:0/2966539398 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 msgr2=0x7f55281445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f55277fe700 1 --2- 172.21.15.47:0/2966539398 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.582+0000 7f55277fe700 1 -- 172.21.15.47:0/2966539398 >> v1:172.21.15.18:6789/0 conn(0x7f5528063d20 legacy=0x7f552814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f55277fe700 1 -- 172.21.15.47:0/2966539398 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55281479e0 con 0x7f55280641d0 2020-07-18T03:34:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f55277fe700 1 --2- 172.21.15.47:0/2966539398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55280641d0 0x7f552814ae30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f5520002f30 tx=0x7f5520006e90).ready entity=mon.1 client_cookie=9ebd77e1571f7a0e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f5525ffb700 1 -- 172.21.15.47:0/2966539398 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55200099c0 con 0x7f55280641d0 2020-07-18T03:34:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f5525ffb700 1 -- 172.21.15.47:0/2966539398 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520009b20 con 0x7f55280641d0 2020-07-18T03:34:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f5525ffb700 1 -- 172.21.15.47:0/2966539398 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5520009e80 con 0x7f55280641d0 2020-07-18T03:34:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 -- 172.21.15.47:0/2966539398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55280641d0 msgr2=0x7f552814ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 --2- 172.21.15.47:0/2966539398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55280641d0 0x7f552814ae30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f5520002f30 tx=0x7f5520006e90).stop 2020-07-18T03:34:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 -- 172.21.15.47:0/2966539398 shutdown_connections 2020-07-18T03:34:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 --2- 172.21.15.47:0/2966539398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55280641d0 0x7f552814ae30 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 --2- 172.21.15.47:0/2966539398 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 -- 172.21.15.47:0/2966539398 >> 172.21.15.47:0/2966539398 conn(0x7f552805f5a0 msgr2=0x7f552805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:10.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 -- 172.21.15.47:0/2966539398 shutdown_connections 2020-07-18T03:34:10.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.583+0000 7f552d975700 1 -- 172.21.15.47:0/2966539398 wait complete. 2020-07-18T03:34:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f552d975700 1 Processor -- start 2020-07-18T03:34:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f552d975700 1 -- start start 2020-07-18T03:34:10.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f552d975700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f552d975700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552814cd00 con 0x7f5528063d20 2020-07-18T03:34:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f552d975700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552814cfa0 con 0x7f552813df00 2020-07-18T03:34:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f552d975700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552814d240 con 0x7f5528147150 2020-07-18T03:34:10.605 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f55277fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f552813df00 0x7f552814d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44786/0 (socket says 172.21.15.47:44786) 2020-07-18T03:34:10.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f55277fe700 1 -- 172.21.15.47:0/2879373260 learned_addr learned my addr 172.21.15.47:0/2879373260 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:10.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.585+0000 7f55267fc700 1 --2- 172.21.15.47:0/2879373260 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466077293 0 0) 0x7f552814cfa0 con 0x7f552813df00 2020-07-18T03:34:10.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5504003240 con 0x7f552813df00 2020-07-18T03:34:10.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731656164 0 0) 0x7f552814cd00 con 0x7f5528063d20 2020-07-18T03:34:10.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f552814cfa0 con 0x7f5528063d20 2020-07-18T03:34:10.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1029841535 0 0) 0x7f5504003240 con 0x7f552813df00 2020-07-18T03:34:10.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f552814cd00 con 0x7f552813df00 2020-07-18T03:34:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f55200099c0 con 0x7f552813df00 2020-07-18T03:34:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 msgr2=0x7f55281415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 --2- 172.21.15.47:0/2879373260 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 >> v1:172.21.15.47:6789/0 conn(0x7f552813df00 legacy=0x7f552814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 >> v1:172.21.15.18:6789/0 conn(0x7f5528063d20 legacy=0x7f5528145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5504003970 0x7f5504003d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55040044e0 0x7f5504004940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5504004e30 0x7f550400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552814d240 con 0x7f55040044e0 2020-07-18T03:34:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550400bc20 con 0x7f5504003970 2020-07-18T03:34:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550400bd60 con 0x7f5504004e30 2020-07-18T03:34:10.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f55267fc700 1 --2- 172.21.15.47:0/2879373260 >> v2:172.21.15.18:3300/0 conn(0x7f5528147150 0x7f55281415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:10.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f5526ffd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5504003970 0x7f5504003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f55267fc700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55040044e0 0x7f5504004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f55277fe700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5504004e30 0x7f550400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f5526ffd700 1 -- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5504004e30 msgr2=0x7f550400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f5526ffd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5504004e30 0x7f550400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.614 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f5526ffd700 1 -- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55040044e0 msgr2=0x7f5504004940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f5526ffd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55040044e0 0x7f5504004940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f5526ffd700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552813f4c0 con 0x7f5504003970 2020-07-18T03:34:10.615 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f55277fe700 1 -- 172.21.15.47:0/2879373260 reap_dead start 2020-07-18T03:34:10.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f55267fc700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55040044e0 0x7f5504004940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:10.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55280b1b10 con 0x7f5504003970 2020-07-18T03:34:10.616 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.586+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552813e530 con 0x7f5504003970 2020-07-18T03:34:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.587+0000 7f5526ffd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5504003970 0x7f5504003d70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f55040054b0 tx=0x7f551c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.587+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f551c006210 con 0x7f5504003970 2020-07-18T03:34:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.587+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c006370 con 0x7f5504003970 2020-07-18T03:34:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.587+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f551c0096f0 con 0x7f5504003970 2020-07-18T03:34:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.587+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55280b1ca0 con 0x7f5504003970 2020-07-18T03:34:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.588+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f551c009850 con 0x7f5504003970 2020-07-18T03:34:10.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.588+0000 7f550effd700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5504053eb0 0x7f5504056370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.588+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f551c067d50 con 0x7f5504003970 2020-07-18T03:34:10.619 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.588+0000 7f55267fc700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5504053eb0 0x7f5504056370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.620 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.588+0000 7f55267fc700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5504053eb0 0x7f5504056370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f5518009000 tx=0x7f5518009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.592+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f551c040460 con 0x7f5504003970 2020-07-18T03:34:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.748+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55280688f0 con 0x7f5504003970 2020-07-18T03:34:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.749+0000 7f550effd700 1 -- 172.21.15.47:0/2879373260 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16467 (secure 0 0 0) 0x7f551c043a80 con 0x7f5504003970 2020-07-18T03:34:10.751 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:10.751 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:07.302099+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:32:57.493054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-07-18T03:32:58.764476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"192.168.0.1:0/1325450171":"2020-07-18T04:34:06.628279+0000","172.21.15.47:6835/2273595707":"2020-07-19T03:33:14.874762+0000","172.21.15.47:6834/2273595707":"2020-07-19T03:33:14.874762+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5504053eb0 msgr2=0x7f5504056370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5504053eb0 0x7f5504056370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f5518009000 tx=0x7f5518009040).stop 2020-07-18T03:34:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5504003970 msgr2=0x7f5504003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5504003970 0x7f5504003d70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f55040054b0 tx=0x7f551c008040).stop 2020-07-18T03:34:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 shutdown_connections 2020-07-18T03:34:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5504053eb0 0x7f5504056370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 --2- 172.21.15.47:0/2879373260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5504003970 0x7f5504003d70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 >> 172.21.15.47:0/2879373260 conn(0x7f552805f5a0 msgr2=0x7f5528062340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 shutdown_connections 2020-07-18T03:34:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.750+0000 7f552d975700 1 -- 172.21.15.47:0/2879373260 wait complete. 2020-07-18T03:34:10.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:32:57.493054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:32:58.764476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:34:10.765 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:34:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.882+0000 7fe4b2bc6700 1 Processor -- start 2020-07-18T03:34:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.882+0000 7fe4b2bc6700 1 -- start start 2020-07-18T03:34:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0b59b0 0x7fe4ac0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4ac0b6300 0x7fe4ac0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac147de0 con 0x7fe4ac0ad330 2020-07-18T03:34:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ac0ae000 con 0x7fe4ac0b6300 2020-07-18T03:34:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac148010 con 0x7fe4ac0b59b0 2020-07-18T03:34:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4ac0b6300 0x7fe4ac0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4ac0b6300 0x7fe4ac0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35480/0 (socket says 172.21.15.47:35480) 2020-07-18T03:34:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 -- 172.21.15.47:0/180512525 learned_addr learned my addr 172.21.15.47:0/180512525 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4abfff700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0b59b0 0x7fe4ac0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 -- 172.21.15.47:0/180512525 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0b59b0 msgr2=0x7fe4ac0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:10.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0b59b0 0x7fe4ac0b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 -- 172.21.15.47:0/180512525 >> v1:172.21.15.18:6789/0 conn(0x7fe4ac0ad330 legacy=0x7fe4ac143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:10.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 -- 172.21.15.47:0/180512525 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac0ad710 con 0x7fe4ac0b6300 2020-07-18T03:34:10.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4abfff700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0b59b0 0x7fe4ac0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:10.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b1163700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0b6300 0x7fe4ac0b3a10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fe4ac0ae4e0 tx=0x7fe4a4009270).ready entity=mon.1 client_cookie=ffefd524356922b7 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4ab7fe700 1 -- 172.21.15.47:0/180512525 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a400bbc0 con 0x7fe4ac0b6300 2020-07-18T03:34:10.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4ab7fe700 1 -- 172.21.15.47:0/180512525 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a400bd20 con 0x7fe4ac0b6300 2020-07-18T03:34:10.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4ab7fe700 1 -- 172.21.15.47:0/180512525 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a4011400 con 0x7fe4ac0b6300 2020-07-18T03:34:10.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/180512525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0b6300 msgr2=0x7fe4ac0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0b6300 0x7fe4ac0b3a10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fe4ac0ae4e0 tx=0x7fe4a4009270).stop 2020-07-18T03:34:10.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/180512525 shutdown_connections 2020-07-18T03:34:10.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac0b6300 0x7fe4ac0b3a10 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 --2- 172.21.15.47:0/180512525 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0b59b0 0x7fe4ac0b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/180512525 >> 172.21.15.47:0/180512525 conn(0x7fe4ac05f5a0 msgr2=0x7fe4ac05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:10.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/180512525 shutdown_connections 2020-07-18T03:34:10.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.883+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/180512525 wait complete. 2020-07-18T03:34:10.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.885+0000 7fe4b2bc6700 1 Processor -- start 2020-07-18T03:34:10.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.885+0000 7fe4b2bc6700 1 -- start start 2020-07-18T03:34:10.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b2bc6700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 0x7fe4ac14a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b2bc6700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe4ac0b59b0 0x7fe4ac158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b2bc6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac158590 con 0x7fe4ac0af9d0 2020-07-18T03:34:10.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b2bc6700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ac158830 con 0x7fe4ac0b59b0 2020-07-18T03:34:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b2bc6700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac158ad0 con 0x7fe4ac0ad330 2020-07-18T03:34:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b1163700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe4ac0af9d0 0x7fe4ac13de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44806/0 (socket says 172.21.15.47:44806) 2020-07-18T03:34:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b1163700 1 -- 172.21.15.47:0/1792799623 learned_addr learned my addr 172.21.15.47:0/1792799623 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656644170 0 0) 0x7fe4ac158590 con 0x7fe4ac0af9d0 2020-07-18T03:34:10.897 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4abfff700 1 --2- 172.21.15.47:0/1792799623 >> v2:172.21.15.18:3300/0 conn(0x7fe4ac0b59b0 0x7fe4ac158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4b0962700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 0x7fe4ac14a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe484003240 con 0x7fe4ac0af9d0 2020-07-18T03:34:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 4237870081 0 0) 0x7fe484003240 con 0x7fe4ac0af9d0 2020-07-18T03:34:10.898 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4ac158590 con 0x7fe4ac0af9d0 2020-07-18T03:34:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fe4a400bbc0 con 0x7fe4ac0af9d0 2020-07-18T03:34:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 msgr2=0x7fe4ac14a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 0x7fe4ac14a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 >> v2:172.21.15.18:3300/0 conn(0x7fe4ac0b59b0 msgr2=0x7fe4ac158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 --2- 172.21.15.47:0/1792799623 >> v2:172.21.15.18:3300/0 conn(0x7fe4ac0b59b0 0x7fe4ac158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 >> v1:172.21.15.47:6789/0 conn(0x7fe4ac0af9d0 legacy=0x7fe4ac13de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe484003800 0x7fe484003c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4840044e0 0x7fe484004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 0x7fe484004e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac158830 con 0x7fe4840044e0 2020-07-18T03:34:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac158ad0 con 0x7fe484003800 2020-07-18T03:34:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.886+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe484005300 con 0x7fe4ac0ad330 2020-07-18T03:34:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe484003800 0x7fe484003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4abfff700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4840044e0 0x7fe484004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b1163700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 0x7fe484004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 -- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 msgr2=0x7fe484004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe4ac0ad330 0x7fe484004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 -- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4840044e0 msgr2=0x7fe484004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4840044e0 0x7fe484004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac143f10 con 0x7fe484003800 2020-07-18T03:34:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac0afe80 con 0x7fe484003800 2020-07-18T03:34:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac13e520 con 0x7fe484003800 2020-07-18T03:34:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4b0962700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe484003800 0x7fe484003c20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fe49c00d000 tx=0x7fe49c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe49c00d5f0 con 0x7fe484003800 2020-07-18T03:34:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c00d750 con 0x7fe484003800 2020-07-18T03:34:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.887+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe49c00dab0 con 0x7fe484003800 2020-07-18T03:34:10.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.888+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac05b120 con 0x7fe484003800 2020-07-18T03:34:10.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.888+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe49c00dc10 con 0x7fe484003800 2020-07-18T03:34:10.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.889+0000 7fe4a8ff9700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe4840581d0 0x7fe48405a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:10.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.889+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fe49c067a60 con 0x7fe484003800 2020-07-18T03:34:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.889+0000 7fe4abfff700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe4840581d0 0x7fe48405a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:10.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.889+0000 7fe4abfff700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe4840581d0 0x7fe48405a690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fe4a000d000 tx=0x7fe4a0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:10.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:10.892+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe49c040170 con 0x7fe484003800 2020-07-18T03:34:11.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.048+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe4ac0690d0 con 0x7fe484003800 2020-07-18T03:34:11.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.048+0000 7fe4a8ff9700 1 -- 172.21.15.47:0/1792799623 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v69) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe49c007c70 con 0x7fe484003800 2020-07-18T03:34:11.050 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:34:11.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe4840581d0 msgr2=0x7fe48405a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b2bc6700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe4840581d0 0x7fe48405a690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fe4a000d000 tx=0x7fe4a0008040).stop 2020-07-18T03:34:11.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe484003800 msgr2=0x7fe484003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b2bc6700 1 --2- 172.21.15.47:0/1792799623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe484003800 0x7fe484003c20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fe49c00d000 tx=0x7fe49c007b50).stop 2020-07-18T03:34:11.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b1163700 1 -- 172.21.15.47:0/1792799623 reap_dead start 2020-07-18T03:34:11.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b1163700 1 -- 172.21.15.47:0/1792799623 reap_dead start 2020-07-18T03:34:11.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.049+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 shutdown_connections 2020-07-18T03:34:11.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.050+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 >> 172.21.15.47:0/1792799623 conn(0x7fe4ac05f5a0 msgr2=0x7fe4ac147460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.050+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 shutdown_connections 2020-07-18T03:34:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.050+0000 7fe4b2bc6700 1 -- 172.21.15.47:0/1792799623 wait complete. 2020-07-18T03:34:11.061 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:34:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.178+0000 7fa94ffae700 1 Processor -- start 2020-07-18T03:34:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94ffae700 1 -- start start 2020-07-18T03:34:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94ffae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa948142a20 0x7fa948142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94ffae700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa948061c60 0x7fa9480620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94ffae700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9480af4b0 con 0x7fa9480b5dc0 2020-07-18T03:34:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94ffae700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9480b6d80 con 0x7fa948142a20 2020-07-18T03:34:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94ffae700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480af710 con 0x7fa948061c60 2020-07-18T03:34:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.179+0000 7fa94dd4a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa948142a20 0x7fa948142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa948142a20 0x7fa948142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35498/0 (socket says 172.21.15.47:35498) 2020-07-18T03:34:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 -- 172.21.15.47:0/699729482 learned_addr learned my addr 172.21.15.47:0/699729482 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94d549700 1 --2- 172.21.15.47:0/699729482 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa948061c60 0x7fa9480620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 -- 172.21.15.47:0/699729482 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa948061c60 msgr2=0x7fa9480620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 --2- 172.21.15.47:0/699729482 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa948061c60 0x7fa9480620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 -- 172.21.15.47:0/699729482 >> v1:172.21.15.18:6789/0 conn(0x7fa9480b5dc0 legacy=0x7fa948146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 -- 172.21.15.47:0/699729482 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480af9b0 con 0x7fa948142a20 2020-07-18T03:34:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94dd4a700 1 --2- 172.21.15.47:0/699729482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa948142a20 0x7fa948142e00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fa93c002f30 tx=0x7fa93c006e90).ready entity=mon.1 client_cookie=f217e9e837a067d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94cd48700 1 -- 172.21.15.47:0/699729482 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93c009970 con 0x7fa948142a20 2020-07-18T03:34:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94cd48700 1 -- 172.21.15.47:0/699729482 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c009ad0 con 0x7fa948142a20 2020-07-18T03:34:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94cd48700 1 -- 172.21.15.47:0/699729482 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93c009e30 con 0x7fa948142a20 2020-07-18T03:34:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 -- 172.21.15.47:0/699729482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa948142a20 msgr2=0x7fa948142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 --2- 172.21.15.47:0/699729482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa948142a20 0x7fa948142e00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fa93c002f30 tx=0x7fa93c006e90).stop 2020-07-18T03:34:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 -- 172.21.15.47:0/699729482 shutdown_connections 2020-07-18T03:34:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 --2- 172.21.15.47:0/699729482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa948142a20 0x7fa948142e00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 --2- 172.21.15.47:0/699729482 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa948061c60 0x7fa9480620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 -- 172.21.15.47:0/699729482 >> 172.21.15.47:0/699729482 conn(0x7fa94805f5a0 msgr2=0x7fa94805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 -- 172.21.15.47:0/699729482 shutdown_connections 2020-07-18T03:34:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.180+0000 7fa94ffae700 1 -- 172.21.15.47:0/699729482 wait complete. 2020-07-18T03:34:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.182+0000 7fa94ffae700 1 Processor -- start 2020-07-18T03:34:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.182+0000 7fa94ffae700 1 -- start start 2020-07-18T03:34:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.182+0000 7fa94ffae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9480b5dc0 0x7fa948155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.182+0000 7fa94ffae700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa948143310 0x7fa9480b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94ffae700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9481463f0 con 0x7fa948061c60 2020-07-18T03:34:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94ffae700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9480b0670 con 0x7fa948143310 2020-07-18T03:34:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94ffae700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9480b0910 con 0x7fa9480b5dc0 2020-07-18T03:34:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94dd4a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa948061c60 0x7fa9480b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44822/0 (socket says 172.21.15.47:44822) 2020-07-18T03:34:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94dd4a700 1 -- 172.21.15.47:0/300476959 learned_addr learned my addr 172.21.15.47:0/300476959 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94d549700 1 --2- 172.21.15.47:0/300476959 >> v2:172.21.15.47:3300/0 conn(0x7fa9480b5dc0 0x7fa948155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94e54b700 1 --2- 172.21.15.47:0/300476959 >> v2:172.21.15.18:3300/0 conn(0x7fa948143310 0x7fa9480b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738977724 0 0) 0x7fa9481463f0 con 0x7fa948061c60 2020-07-18T03:34:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa920003240 con 0x7fa948061c60 2020-07-18T03:34:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94d549700 1 -- 172.21.15.47:0/300476959 >> v2:172.21.15.18:3300/0 conn(0x7fa948143310 msgr2=0x7fa9480b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94d549700 1 --2- 172.21.15.47:0/300476959 >> v2:172.21.15.18:3300/0 conn(0x7fa948143310 0x7fa9480b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94d549700 1 -- 172.21.15.47:0/300476959 >> v1:172.21.15.47:6789/0 conn(0x7fa948061c60 legacy=0x7fa9480b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94d549700 1 -- 172.21.15.47:0/300476959 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480b41a0 con 0x7fa9480b5dc0 2020-07-18T03:34:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94d549700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9480b5dc0 0x7fa948155940 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7fa948144c50 tx=0x7fa944008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa94400f830 con 0x7fa9480b5dc0 2020-07-18T03:34:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400f990 con 0x7fa9480b5dc0 2020-07-18T03:34:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94813c9c0 con 0x7fa9480b5dc0 2020-07-18T03:34:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa94400fcf0 con 0x7fa9480b5dc0 2020-07-18T03:34:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.183+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94813ce30 con 0x7fa9480b5dc0 2020-07-18T03:34:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.184+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9480b04b0 con 0x7fa9480b5dc0 2020-07-18T03:34:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.184+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa9440123e0 con 0x7fa9480b5dc0 2020-07-18T03:34:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.185+0000 7fa939ffb700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa920050380 0x7fa920052840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.185+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fa944067890 con 0x7fa9480b5dc0 2020-07-18T03:34:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.185+0000 7fa94dd4a700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa920050380 0x7fa920052840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.185+0000 7fa94dd4a700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa920050380 0x7fa920052840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fa93c006f90 tx=0x7fa93c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.188+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa94403ff40 con 0x7fa9480b5dc0 2020-07-18T03:34:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.345+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa94805b120 con 0x7fa9480b5dc0 2020-07-18T03:34:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.346+0000 7fa939ffb700 1 -- 172.21.15.47:0/300476959 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa944043560 con 0x7fa9480b5dc0 2020-07-18T03:34:11.347 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa920050380 msgr2=0x7fa920052840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa920050380 0x7fa920052840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fa93c006f90 tx=0x7fa93c015000).stop 2020-07-18T03:34:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9480b5dc0 msgr2=0x7fa948155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9480b5dc0 0x7fa948155940 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7fa948144c50 tx=0x7fa944008040).stop 2020-07-18T03:34:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 shutdown_connections 2020-07-18T03:34:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 --2- 172.21.15.47:0/300476959 >> v2:172.21.15.18:3300/0 conn(0x7fa948143310 0x7fa9480b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa920050380 0x7fa920052840 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 --2- 172.21.15.47:0/300476959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9480b5dc0 0x7fa948155940 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 >> 172.21.15.47:0/300476959 conn(0x7fa94805f5a0 msgr2=0x7fa94813ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 shutdown_connections 2020-07-18T03:34:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.347+0000 7fa94ffae700 1 -- 172.21.15.47:0/300476959 wait complete. 2020-07-18T03:34:11.359 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:34:11.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.475+0000 7f7203145700 1 Processor -- start 2020-07-18T03:34:11.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f7203145700 1 -- start start 2020-07-18T03:34:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f7203145700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71fc0ad330 0x7f71fc143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f7203145700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71fc0b59b0 0x7f71fc0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f7203145700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc147de0 con 0x7f71fc0b6300 2020-07-18T03:34:11.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f7203145700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc0ae000 con 0x7f71fc0b59b0 2020-07-18T03:34:11.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f7203145700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc148010 con 0x7f71fc0ad330 2020-07-18T03:34:11.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.476+0000 7f71fbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71fc0b59b0 0x7f71fc0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71fc0b59b0 0x7f71fc0b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35510/0 (socket says 172.21.15.47:35510) 2020-07-18T03:34:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 -- 172.21.15.47:0/2924742841 learned_addr learned my addr 172.21.15.47:0/2924742841 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7200ee1700 1 --2- 172.21.15.47:0/2924742841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71fc0ad330 0x7f71fc143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 -- 172.21.15.47:0/2924742841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71fc0ad330 msgr2=0x7f71fc143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 --2- 172.21.15.47:0/2924742841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71fc0ad330 0x7f71fc143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fb7fe700 1 -- 172.21.15.47:0/2924742841 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764439677 0 0) 0x7f71fc147de0 con 0x7f71fc0b6300 2020-07-18T03:34:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 -- 172.21.15.47:0/2924742841 >> v1:172.21.15.18:6789/0 conn(0x7f71fc0b6300 legacy=0x7f71fc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 -- 172.21.15.47:0/2924742841 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0ad710 con 0x7f71fc0b59b0 2020-07-18T03:34:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fbfff700 1 --2- 172.21.15.47:0/2924742841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc0b59b0 0x7f71fc0b5e10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f71f0002f30 tx=0x7f71f0006e90).ready entity=mon.1 client_cookie=384eb4388cf9bee3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fb7fe700 1 -- 172.21.15.47:0/2924742841 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f0009a40 con 0x7f71fc0b59b0 2020-07-18T03:34:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fb7fe700 1 -- 172.21.15.47:0/2924742841 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0009ba0 con 0x7f71fc0b59b0 2020-07-18T03:34:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f71fb7fe700 1 -- 172.21.15.47:0/2924742841 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f0010470 con 0x7f71fc0b59b0 2020-07-18T03:34:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 -- 172.21.15.47:0/2924742841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc0b59b0 msgr2=0x7f71fc0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 --2- 172.21.15.47:0/2924742841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc0b59b0 0x7f71fc0b5e10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f71f0002f30 tx=0x7f71f0006e90).stop 2020-07-18T03:34:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 -- 172.21.15.47:0/2924742841 shutdown_connections 2020-07-18T03:34:11.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 --2- 172.21.15.47:0/2924742841 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f71fc0ad330 0x7f71fc143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 --2- 172.21.15.47:0/2924742841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc0b59b0 0x7f71fc0b5e10 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 -- 172.21.15.47:0/2924742841 >> 172.21.15.47:0/2924742841 conn(0x7f71fc05f5a0 msgr2=0x7f71fc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 -- 172.21.15.47:0/2924742841 shutdown_connections 2020-07-18T03:34:11.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.477+0000 7f7203145700 1 -- 172.21.15.47:0/2924742841 wait complete. 2020-07-18T03:34:11.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.479+0000 7f7203145700 1 Processor -- start 2020-07-18T03:34:11.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.479+0000 7f7203145700 1 -- start start 2020-07-18T03:34:11.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.479+0000 7f7203145700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f71fc0b6300 0x7f71fc064e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7203145700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71fc13ad90 0x7f71fc137a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7203145700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0643d0 con 0x7f71fc0ad330 2020-07-18T03:34:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7203145700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc064740 con 0x7f71fc0b6300 2020-07-18T03:34:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7203145700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc138580 con 0x7f71fc13ad90 2020-07-18T03:34:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7200ee1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f71fc0ad330 0x7f71fc063cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44836/0 (socket says 172.21.15.47:44836) 2020-07-18T03:34:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7200ee1700 1 -- 172.21.15.47:0/2529284573 learned_addr learned my addr 172.21.15.47:0/2529284573 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f72016e2700 1 --2- 172.21.15.47:0/2529284573 >> v2:172.21.15.47:3300/0 conn(0x7f71fc13ad90 0x7f71fc137a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f71fbfff700 1 --2- 172.21.15.47:0/2529284573 >> v2:172.21.15.18:3300/0 conn(0x7f71fc0b6300 0x7f71fc064e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523419239 0 0) 0x7f71fc0643d0 con 0x7f71fc0ad330 2020-07-18T03:34:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d4003240 con 0x7f71fc0ad330 2020-07-18T03:34:11.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f72016e2700 1 -- 172.21.15.47:0/2529284573 >> v2:172.21.15.18:3300/0 conn(0x7f71fc0b6300 msgr2=0x7f71fc064e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f72016e2700 1 --2- 172.21.15.47:0/2529284573 >> v2:172.21.15.18:3300/0 conn(0x7f71fc0b6300 0x7f71fc064e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f72016e2700 1 -- 172.21.15.47:0/2529284573 >> v1:172.21.15.47:6789/0 conn(0x7f71fc0ad330 legacy=0x7f71fc063cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f72016e2700 1 -- 172.21.15.47:0/2529284573 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc1386c0 con 0x7f71fc13ad90 2020-07-18T03:34:11.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f72016e2700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc13ad90 0x7f71fc137a60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f71fc063ac0 tx=0x7f71f4007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f4010510 con 0x7f71fc13ad90 2020-07-18T03:34:11.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4010670 con 0x7f71fc13ad90 2020-07-18T03:34:11.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc13a180 con 0x7f71fc13ad90 2020-07-18T03:34:11.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f40107e0 con 0x7f71fc13ad90 2020-07-18T03:34:11.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.480+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc13a460 con 0x7f71fc13ad90 2020-07-18T03:34:11.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.481+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc0b4300 con 0x7f71fc13ad90 2020-07-18T03:34:11.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.482+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f71f4010070 con 0x7f71fc13ad90 2020-07-18T03:34:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.482+0000 7f71f8ff9700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71d4050380 0x7f71d4052840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.482+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f71f4067700 con 0x7f71fc13ad90 2020-07-18T03:34:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.482+0000 7f7200ee1700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71d4050380 0x7f71d4052840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.482+0000 7f7200ee1700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71d4050380 0x7f71d4052840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f71ec00a000 tx=0x7f71ec008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.485+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f71f403fe10 con 0x7f71fc13ad90 2020-07-18T03:34:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.643+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71fc064880 con 0x7f71fc13ad90 2020-07-18T03:34:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.643+0000 7f71f8ff9700 1 -- 172.21.15.47:0/2529284573 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+10 (secure 0 0 0) 0x7f71f4043430 con 0x7f71fc13ad90 2020-07-18T03:34:11.645 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71d4050380 msgr2=0x7f71d4052840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71d4050380 0x7f71d4052840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f71ec00a000 tx=0x7f71ec008040).stop 2020-07-18T03:34:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc13ad90 msgr2=0x7f71fc137a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc13ad90 0x7f71fc137a60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f71fc063ac0 tx=0x7f71f4007c10).stop 2020-07-18T03:34:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 shutdown_connections 2020-07-18T03:34:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71fc13ad90 0x7f71fc137a60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 --2- 172.21.15.47:0/2529284573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f71d4050380 0x7f71d4052840 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 --2- 172.21.15.47:0/2529284573 >> v2:172.21.15.18:3300/0 conn(0x7f71fc0b6300 0x7f71fc064e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 >> 172.21.15.47:0/2529284573 conn(0x7f71fc05f5a0 msgr2=0x7f71fc149860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 shutdown_connections 2020-07-18T03:34:11.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.645+0000 7f7203145700 1 -- 172.21.15.47:0/2529284573 wait complete. 2020-07-18T03:34:11.657 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:34:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.773+0000 7fe998380700 1 Processor -- start 2020-07-18T03:34:11.775 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.773+0000 7fe998380700 1 -- start start 2020-07-18T03:34:11.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe998380700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe9900ad330 0x7fe990143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe998380700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9900b3a10 0x7fe9900b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe998380700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe990147de0 con 0x7fe9900b59b0 2020-07-18T03:34:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe998380700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900ae000 con 0x7fe9900ad330 2020-07-18T03:34:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe998380700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe990148010 con 0x7fe9900b3a10 2020-07-18T03:34:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9900b3a10 0x7fe9900b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9900b3a10 0x7fe9900b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35528/0 (socket says 172.21.15.47:35528) 2020-07-18T03:34:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 -- 172.21.15.47:0/197550675 learned_addr learned my addr 172.21.15.47:0/197550675 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99611c700 1 --2- 172.21.15.47:0/197550675 >> v2:172.21.15.18:3300/0 conn(0x7fe9900ad330 0x7fe990143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 -- 172.21.15.47:0/197550675 >> v2:172.21.15.18:3300/0 conn(0x7fe9900ad330 msgr2=0x7fe990143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 --2- 172.21.15.47:0/197550675 >> v2:172.21.15.18:3300/0 conn(0x7fe9900ad330 0x7fe990143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 -- 172.21.15.47:0/197550675 >> v1:172.21.15.18:6789/0 conn(0x7fe9900b59b0 legacy=0x7fe9900b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99691d700 1 -- 172.21.15.47:0/197550675 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900ad710 con 0x7fe9900b3a10 2020-07-18T03:34:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.774+0000 7fe99611c700 1 --2- 172.21.15.47:0/197550675 >> v2:172.21.15.18:3300/0 conn(0x7fe9900ad330 0x7fe990143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe99691d700 1 --2- 172.21.15.47:0/197550675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9900b3a10 0x7fe9900b3e80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fe9900ae4e0 tx=0x7fe988009270).ready entity=mon.1 client_cookie=9f8de75c3c6ba572 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe99511a700 1 -- 172.21.15.47:0/197550675 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe98800bbc0 con 0x7fe9900b3a10 2020-07-18T03:34:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe99511a700 1 -- 172.21.15.47:0/197550675 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800bd20 con 0x7fe9900b3a10 2020-07-18T03:34:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe99511a700 1 -- 172.21.15.47:0/197550675 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe98800be90 con 0x7fe9900b3a10 2020-07-18T03:34:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 -- 172.21.15.47:0/197550675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9900b3a10 msgr2=0x7fe9900b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 --2- 172.21.15.47:0/197550675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9900b3a10 0x7fe9900b3e80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fe9900ae4e0 tx=0x7fe988009270).stop 2020-07-18T03:34:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 -- 172.21.15.47:0/197550675 shutdown_connections 2020-07-18T03:34:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 --2- 172.21.15.47:0/197550675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9900b3a10 0x7fe9900b3e80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 --2- 172.21.15.47:0/197550675 >> v2:172.21.15.18:3300/0 conn(0x7fe9900ad330 0x7fe990143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 -- 172.21.15.47:0/197550675 >> 172.21.15.47:0/197550675 conn(0x7fe99005f5a0 msgr2=0x7fe99005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 -- 172.21.15.47:0/197550675 shutdown_connections 2020-07-18T03:34:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.775+0000 7fe998380700 1 -- 172.21.15.47:0/197550675 wait complete. 2020-07-18T03:34:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.776+0000 7fe998380700 1 Processor -- start 2020-07-18T03:34:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.776+0000 7fe998380700 1 -- start start 2020-07-18T03:34:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe998380700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900b59b0 0x7fe9901339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe998380700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe990134380 con 0x7fe9900ad330 2020-07-18T03:34:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe998380700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900b9b70 con 0x7fe990133ed0 2020-07-18T03:34:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe998380700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe990063b00 con 0x7fe9900b59b0 2020-07-18T03:34:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe99691d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe990133ed0 0x7fe9900b7180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44854/0 (socket says 172.21.15.47:44854) 2020-07-18T03:34:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe99691d700 1 -- 172.21.15.47:0/3983038398 learned_addr learned my addr 172.21.15.47:0/3983038398 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe99591b700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900b59b0 0x7fe9901339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189023025 0 0) 0x7fe9900b9b70 con 0x7fe990133ed0 2020-07-18T03:34:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe968003240 con 0x7fe990133ed0 2020-07-18T03:34:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818886897 0 0) 0x7fe990134380 con 0x7fe9900ad330 2020-07-18T03:34:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.777+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9900b9b70 con 0x7fe9900ad330 2020-07-18T03:34:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1357952652 0 0) 0x7fe968003240 con 0x7fe990133ed0 2020-07-18T03:34:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe990134380 con 0x7fe990133ed0 2020-07-18T03:34:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fe98800bbc0 con 0x7fe990133ed0 2020-07-18T03:34:11.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe99591b700 1 -- 172.21.15.47:0/3983038398 >> v1:172.21.15.47:6789/0 conn(0x7fe990133ed0 legacy=0x7fe9900b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe99591b700 1 -- 172.21.15.47:0/3983038398 >> v1:172.21.15.18:6789/0 conn(0x7fe9900ad330 legacy=0x7fe990132d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.803 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe99591b700 1 -- 172.21.15.47:0/3983038398 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990063830 con 0x7fe9900b59b0 2020-07-18T03:34:11.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe99591b700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900b59b0 0x7fe9901339e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fe98c001f50 tx=0x7fe98c007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe98c007030 con 0x7fe9900b59b0 2020-07-18T03:34:11.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c007190 con 0x7fe9900b59b0 2020-07-18T03:34:11.804 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe990142d80 con 0x7fe9900b59b0 2020-07-18T03:34:11.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe98c0064f0 con 0x7fe9900b59b0 2020-07-18T03:34:11.805 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.778+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe990064630 con 0x7fe9900b59b0 2020-07-18T03:34:11.806 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.779+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9900b3730 con 0x7fe9900b59b0 2020-07-18T03:34:11.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.779+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe98c0066f0 con 0x7fe9900b59b0 2020-07-18T03:34:11.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.780+0000 7fe9827fc700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe968050720 0x7fe968052be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:11.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.780+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fe98c0695e0 con 0x7fe9900b59b0 2020-07-18T03:34:11.807 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.780+0000 7fe99611c700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe968050720 0x7fe968052be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:11.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.780+0000 7fe99611c700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe968050720 0x7fe968052be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fe984002f30 tx=0x7fe984008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:11.808 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.784+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe98c041c10 con 0x7fe9900b59b0 2020-07-18T03:34:11.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.944+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe99005b120 con 0x7fe9900b59b0 2020-07-18T03:34:11.945 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.944+0000 7fe9827fc700 1 -- 172.21.15.47:0/3983038398 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe98c045230 con 0x7fe9900b59b0 2020-07-18T03:34:11.946 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:11.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe968050720 msgr2=0x7fe968052be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe968050720 0x7fe968052be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fe984002f30 tx=0x7fe984008040).stop 2020-07-18T03:34:11.947 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900b59b0 msgr2=0x7fe9901339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:11.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900b59b0 0x7fe9901339e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fe98c001f50 tx=0x7fe98c007b50).stop 2020-07-18T03:34:11.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 shutdown_connections 2020-07-18T03:34:11.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe968050720 0x7fe968052be0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 --2- 172.21.15.47:0/3983038398 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe9900b59b0 0x7fe9901339e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:11.948 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 >> 172.21.15.47:0/3983038398 conn(0x7fe99005f5a0 msgr2=0x7fe990146a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:11.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 shutdown_connections 2020-07-18T03:34:11.949 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:11.946+0000 7fe998380700 1 -- 172.21.15.47:0/3983038398 wait complete. 2020-07-18T03:34:11.958 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:34:11.959 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:12.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.073+0000 7f957985b700 1 Processor -- start 2020-07-18T03:34:12.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.073+0000 7f957985b700 1 -- start start 2020-07-18T03:34:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.074+0000 7f957985b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f95740b5370 0x7f957405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.074+0000 7f957985b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95740b3ac0 con 0x7f95740ad9e0 2020-07-18T03:34:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.074+0000 7f957985b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95740b3d20 con 0x7f95740b41c0 2020-07-18T03:34:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.074+0000 7f957985b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95740b5a10 con 0x7f95740b5370 2020-07-18T03:34:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.074+0000 7f95727fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f95740b41c0 0x7f95740ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44858/0 (socket says 172.21.15.47:44858) 2020-07-18T03:34:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.074+0000 7f95727fc700 1 -- 172.21.15.47:0/1070853030 learned_addr learned my addr 172.21.15.47:0/1070853030 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9572ffd700 1 --2- 172.21.15.47:0/1070853030 >> v2:172.21.15.18:3300/0 conn(0x7f95740b5370 0x7f957405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434594806 0 0) 0x7f95740b3d20 con 0x7f95740b41c0 2020-07-18T03:34:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9554003240 con 0x7f95740b41c0 2020-07-18T03:34:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799318274 0 0) 0x7f95740b3ac0 con 0x7f95740ad9e0 2020-07-18T03:34:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95740b3d20 con 0x7f95740ad9e0 2020-07-18T03:34:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9572ffd700 1 -- 172.21.15.47:0/1070853030 >> v1:172.21.15.47:6789/0 conn(0x7f95740b41c0 legacy=0x7f95740ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9572ffd700 1 -- 172.21.15.47:0/1070853030 >> v1:172.21.15.18:6789/0 conn(0x7f95740ad9e0 legacy=0x7f95741428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9572ffd700 1 -- 172.21.15.47:0/1070853030 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740b5750 con 0x7f95740b5370 2020-07-18T03:34:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9572ffd700 1 --2- 172.21.15.47:0/1070853030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95740b5370 0x7f957405fff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f95740afb30 tx=0x7f9564006dd0).ready entity=mon.0 client_cookie=b7f80eeb1d595ad3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95640096f0 con 0x7f95740b5370 2020-07-18T03:34:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9564009850 con 0x7f95740b5370 2020-07-18T03:34:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f9571ffb700 1 -- 172.21.15.47:0/1070853030 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9564009bb0 con 0x7f95740b5370 2020-07-18T03:34:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 -- 172.21.15.47:0/1070853030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95740b5370 msgr2=0x7f957405fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 --2- 172.21.15.47:0/1070853030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95740b5370 0x7f957405fff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f95740afb30 tx=0x7f9564006dd0).stop 2020-07-18T03:34:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 -- 172.21.15.47:0/1070853030 shutdown_connections 2020-07-18T03:34:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 --2- 172.21.15.47:0/1070853030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95740b5370 0x7f957405fff0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 -- 172.21.15.47:0/1070853030 >> 172.21.15.47:0/1070853030 conn(0x7f95740b9340 msgr2=0x7f95740b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 -- 172.21.15.47:0/1070853030 shutdown_connections 2020-07-18T03:34:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.075+0000 7f957985b700 1 -- 172.21.15.47:0/1070853030 wait complete. 2020-07-18T03:34:12.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.077+0000 7f957985b700 1 Processor -- start 2020-07-18T03:34:12.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.077+0000 7f957985b700 1 -- start start 2020-07-18T03:34:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f957985b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95740b41c0 0x7f9574153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f957985b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95741579a0 con 0x7f9574157500 2020-07-18T03:34:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f957985b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9574153630 con 0x7f95740ad9e0 2020-07-18T03:34:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f957985b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95741538d0 con 0x7f95740b41c0 2020-07-18T03:34:12.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f95727fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95740b41c0 0x7f9574153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f9572ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f95740ad9e0 0x7f9574134940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44864/0 (socket says 172.21.15.47:44864) 2020-07-18T03:34:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f9572ffd700 1 -- 172.21.15.47:0/3151304115 learned_addr learned my addr 172.21.15.47:0/3151304115 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f95727fc700 1 -- 172.21.15.47:0/3151304115 >> v1:172.21.15.47:6789/0 conn(0x7f95740ad9e0 legacy=0x7f9574134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f95727fc700 1 -- 172.21.15.47:0/3151304115 >> v1:172.21.15.18:6789/0 conn(0x7f9574157500 legacy=0x7f95740b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f95727fc700 1 -- 172.21.15.47:0/3151304115 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740b63d0 con 0x7f95740b41c0 2020-07-18T03:34:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044220971 0 0) 0x7f9574153630 con 0x7f95740ad9e0 2020-07-18T03:34:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f95727fc700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95740b41c0 0x7f9574153d70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f955c0010d0 tx=0x7f955c007960).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.078+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f955c0072e0 con 0x7f95740b41c0 2020-07-18T03:34:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.079+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c007440 con 0x7f95740b41c0 2020-07-18T03:34:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.079+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f955c0057e0 con 0x7f95740b41c0 2020-07-18T03:34:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.079+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95741452a0 con 0x7f95740b41c0 2020-07-18T03:34:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.079+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9574154420 con 0x7f95740b41c0 2020-07-18T03:34:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.080+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574057450 con 0x7f95740b41c0 2020-07-18T03:34:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.080+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f955c010070 con 0x7f95740b41c0 2020-07-18T03:34:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.080+0000 7f956affd700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f955004fec0 0x7f9550052380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.080+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f955c067320 con 0x7f95740b41c0 2020-07-18T03:34:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.080+0000 7f9572ffd700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f955004fec0 0x7f9550052380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.080+0000 7f9572ffd700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f955004fec0 0x7f9550052380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9564003070 tx=0x7f9564016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:12.099 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.083+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f955c03fa30 con 0x7f95740b41c0 2020-07-18T03:34:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.290+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f957405caf0 con 0x7f95740b41c0 2020-07-18T03:34:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.291+0000 7f956affd700 1 -- 172.21.15.47:0/3151304115 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f955c043050 con 0x7f95740b41c0 2020-07-18T03:34:12.292 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:12.293 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:34:09.311823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f955004fec0 msgr2=0x7f9550052380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f955004fec0 0x7f9550052380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9564003070 tx=0x7f9564016000).stop 2020-07-18T03:34:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95740b41c0 msgr2=0x7f9574153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95740b41c0 0x7f9574153d70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f955c0010d0 tx=0x7f955c007960).stop 2020-07-18T03:34:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 shutdown_connections 2020-07-18T03:34:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f955004fec0 0x7f9550052380 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 --2- 172.21.15.47:0/3151304115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95740b41c0 0x7f9574153d70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 >> 172.21.15.47:0/3151304115 conn(0x7f95740b9340 msgr2=0x7f957405fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 shutdown_connections 2020-07-18T03:34:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:12.292+0000 7f957985b700 1 -- 172.21.15.47:0/3151304115 wait complete. 2020-07-18T03:34:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 36 2020-07-18T03:34:12.453 INFO:tasks.ceph.mds.a.smithi047.stderr:2020-07-18T03:34:12.452+0000 7f0a5434b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:34:12.514 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:32:59.940158+0000', 'modified': '2020-07-18T03:34:09.311823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6228}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6228': {'gid': 6228, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.47:6835/1654366847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1654366847}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1654366847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:34:12.515 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:34:12.516 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-07-18T03:34:12.656 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6228,"want_state":"up:active","state":"up:active","rank_uptime":55.196629227999999,"mdsmap_epoch":36,"osdmap_epoch":69,"osdmap_epoch_barrier":69,"uptime":56.022775361000001} 2020-07-18T03:34:12.657 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6228, "mdsmap_epoch": 36, "osdmap_epoch": 69, "osdmap_epoch_barrier": 69, "rank_uptime": 55.196629228, "state": "up:active", "uptime": 56.022775361, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:34:12.658 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:34:12.658 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:34:12.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:12.662 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:34:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:12.679 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:12.679 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:12.679 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:12.679 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:12.680 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:12.680 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:12.680 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:12.680 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:12.681 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42338sec preferred_lft 42338sec 2020-07-18T03:34:12.681 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:34:12.681 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:12.681 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:12.681 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:12.682 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:12.682 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:12.682 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:34:12.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:12.684 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:34:12.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:12.751 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:34:12.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:12.823 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:34:12.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:12.895 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:34:12.966 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:34:12.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:12.968 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:34:13.024 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:34:13.024 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:34:13.024 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:34:13.025 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:34:13.025 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:34:13.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.026 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:34:13.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.097 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:34:13.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.172 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:34:13.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.244 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:34:13.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.305 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:34:13.359 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 2020-07-18T03:34:13.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.360 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:34:13.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.433 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0' 2020-07-18T03:34:13.504 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2020-07-18T03:34:13.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.505 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0' 2020-07-18T03:34:13.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.578 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:34:13.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.665 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:34:13.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.724 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:34:13.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.772 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:34:13.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.822 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:34:13.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:13.863 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:34:13.979 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:34:13.980 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:34:13.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:13.981 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:13.997 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:34:14.068 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:34:14.069 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:34:14.148 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:34:14.149 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:34:14.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:14.150 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:34:14.203 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:34:14.204 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:14.246 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:34:14.280 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:34:14.280 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:34:14.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:34:14.345+0000 7fd67a73e1c0 -1 init, newargv = 0x56465d1b5db0 newargc=15ceph-fuse[50097]: starting ceph client 2020-07-18T03:34:14.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr: 2020-07-18T03:34:14.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[50097]: starting fuse 2020-07-18T03:34:14.353 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:34:14.356 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:34:14.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:14.358 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:34:14.412 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:34:14.412 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:34:14.413 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:34:14.414 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:34:14.414 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:34:14.415 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:34:14.415 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:34:14.416 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:14.416 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:34:14.417 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:34:14.417 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:34:14.418 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:34:14.419 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:14.420 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:34:14.420 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:34:14.421 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:34:14.421 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:34:14.422 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:14.422 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:34:14.423 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:34:14.423 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:34:14.424 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:34:14.424 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:34:14.428 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:34:14.429 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:34:14.429 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:34:14.430 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:14.431 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:34:14.431 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:34:14.544 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.50097.asok 2020-07-18T03:34:14.724 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.50097.asok 2020-07-18T03:34:14.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:14.725 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.50097.asok status 2020-07-18T03:34:14.876 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:34:14.877 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:34:14.877 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:34:14.877 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:34:14.877 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:34:14.878 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:34:14.878 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:34:14.878 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "50097", 2020-07-18T03:34:14.878 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:34:14.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:14.879 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:34:14.879 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:34:14.879 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6907, 2020-07-18T03:34:14.879 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:34:14.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:34:14.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:34:14.880 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6907 2020-07-18T03:34:14.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:14.880 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:34:14.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:14.881 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:34:14.881 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 831035420 2020-07-18T03:34:14.881 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:14.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:14.881 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:34:14.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:14.882 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:34:14.882 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 831035420 2020-07-18T03:34:14.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:14.882 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6907 192.168.0.1:0/831035420", 2020-07-18T03:34:14.882 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/831035420", 2020-07-18T03:34:14.883 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:34:14.883 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 36, 2020-07-18T03:34:14.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 69, 2020-07-18T03:34:14.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:34:14.883 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:34:14.884 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:34:14.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:14.886 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:34:14.887 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:14.902 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:34:14.902 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:34:14.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:14.903 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:14.974 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.1 && exec sudo ls subdir/childfile) 2020-07-18T03:34:15.048 INFO:teuthology.orchestra.run.smithi047.stdout:subdir/childfile 2020-07-18T03:34:15.050 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:34:15.050 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:34:15.112 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:34:15.163 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:34:15.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.164 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-07-18T03:34:15.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:34:15.183 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:34:15.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.183 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:34:15.236 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:15.236 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:15.236 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:15.237 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.237 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:15.237 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.237 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:15.238 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.238 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:15.238 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.238 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:15.238 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42336sec preferred_lft 42336sec 2020-07-18T03:34:15.239 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:34:15.239 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.239 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:15.239 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.239 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:15.239 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.240 INFO:teuthology.orchestra.run.smithi047.stdout:22: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:34:15.240 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:1c:1a:33:d6:ad brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.240 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:34:15.240 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.240 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::3095:3eff:feff:ead6/64 scope link 2020-07-18T03:34:15.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.241 INFO:teuthology.orchestra.run.smithi047.stdout:23: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:34:15.241 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:1c:1a:33:d6:ad brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:34:15.241 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::481c:1aff:fe33:d6ad/64 scope link tentative 2020-07-18T03:34:15.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.242 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:34:15.306 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.307 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:15.307 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.307 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.307 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.307 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.308 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.308 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.308 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.308 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.308 INFO:teuthology.orchestra.run.smithi047.stdout:22: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.309 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:1c:1a:33:d6:ad brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.309 INFO:teuthology.orchestra.run.smithi047.stdout:23: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.309 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 4a:1c:1a:33:d6:ad brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2020-07-18T03:34:15.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.311 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:15.374 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:34:15.375 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:34:15.376 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:15.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[50097]: fuse finished with error 0 and tester_r 0 2020-07-18T03:34:15.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.459 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:15.512 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:34:15.513 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:34:15.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.514 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:34:15.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.618 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.0' 2020-07-18T03:34:15.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.717 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:34:15.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.758 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:34:15.819 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:15.819 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:15.819 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:15.820 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.820 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:15.820 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.820 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:15.820 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.821 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:15.821 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.821 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:15.821 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42335sec preferred_lft 42335sec 2020-07-18T03:34:15.822 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:34:15.822 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.822 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:15.822 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.822 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:15.823 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.823 INFO:teuthology.orchestra.run.smithi047.stdout:22: ceph-brx: mtu 1500 qdisc noqueue state DOWN group default qlen 1000 2020-07-18T03:34:15.823 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.823 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:34:15.823 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.823 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::3095:3eff:feff:ead6/64 scope link 2020-07-18T03:34:15.824 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:15.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.825 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:34:15.889 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.890 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:15.890 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.890 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.891 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.891 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.891 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.891 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.891 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.892 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.892 INFO:teuthology.orchestra.run.smithi047.stdout:22: ceph-brx: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:15.892 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:15.893 INFO:tasks.cephfs.mount:Removing the 'ceph-brx' 2020-07-18T03:34:15.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.894 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx down' 2020-07-18T03:34:15.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:15.966 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete ceph-brx' 2020-07-18T03:34:16.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.058 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:34:16.075 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:34:16.075 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:34:16.075 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:34:16.076 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:34:16.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.079 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:34:16.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.161 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -D FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:34:16.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.200 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -D POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:34:16.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.282 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:16.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.337 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:34:16.356 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2020-07-18T03:34:16.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:34:16.357 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-07-18T03:34:16.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.358 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:34:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:16.414 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:16.414 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:16.414 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:16.414 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:16.415 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:16.415 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:16.415 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.416 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:16.416 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42334sec preferred_lft 42334sec 2020-07-18T03:34:16.416 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:34:16.416 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:16.417 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:16.417 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.418 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.2) 2020-07-18T03:34:16.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.472 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:34:16.528 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2020-07-18T03:34:16.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:34:16.529 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-07-18T03:34:16.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.530 INFO:teuthology.orchestra.run.smithi018:> ip addr 2020-07-18T03:34:16.600 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:16.601 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:16.601 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:16.601 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:16.601 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:16.602 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:16.602 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:16.602 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.602 INFO:teuthology.orchestra.run.smithi018.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:16.602 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.603 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:16.603 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42334sec preferred_lft 42334sec 2020-07-18T03:34:16.603 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link 2020-07-18T03:34:16.603 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:16.603 INFO:teuthology.orchestra.run.smithi018.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:16.604 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.605 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:16.606 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:16.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:16.607 INFO:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.3) 2020-07-18T03:34:16.658 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync' 2020-07-18T03:34:16.776 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.775+0000 7f9a951bb700 1 Processor -- start 2020-07-18T03:34:16.777 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.775+0000 7f9a951bb700 1 -- start start 2020-07-18T03:34:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a951bb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a900b5370 0x7f9a9005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a951bb700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f9a900b41c0 0x7f9a900ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a951bb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900b3ac0 con 0x7f9a900ad7c0 2020-07-18T03:34:16.779 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a951bb700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900b5a10 con 0x7f9a900b41c0 2020-07-18T03:34:16.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a951bb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900b0040 con 0x7f9a900b5370 2020-07-18T03:34:16.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a900b5370 0x7f9a9005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:16.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a900b5370 0x7f9a9005fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35566/0 (socket says 172.21.15.47:35566) 2020-07-18T03:34:16.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 -- 172.21.15.47:0/239626236 learned_addr learned my addr 172.21.15.47:0/239626236 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:16.780 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8e59c700 1 --2- 172.21.15.47:0/239626236 >> v2:172.21.15.18:3300/0 conn(0x7f9a900b41c0 0x7f9a900ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:16.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 -- 172.21.15.47:0/239626236 >> v2:172.21.15.18:3300/0 conn(0x7f9a900b41c0 msgr2=0x7f9a900ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:16.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 --2- 172.21.15.47:0/239626236 >> v2:172.21.15.18:3300/0 conn(0x7f9a900b41c0 0x7f9a900ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:16.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 -- 172.21.15.47:0/239626236 >> v1:172.21.15.18:6789/0 conn(0x7f9a900ad7c0 legacy=0x7f9a9005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:16.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8ed9d700 1 -- 172.21.15.47:0/239626236 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900b3d20 con 0x7f9a900b5370 2020-07-18T03:34:16.781 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.776+0000 7f9a8e59c700 1 --2- 172.21.15.47:0/239626236 >> v2:172.21.15.18:3300/0 conn(0x7f9a900b41c0 0x7f9a900ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:16.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a8ed9d700 1 --2- 172.21.15.47:0/239626236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b5370 0x7f9a9005fff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f9a90067280 tx=0x7f9a7c006e90).ready entity=mon.1 client_cookie=6b3072cac6c0843a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:16.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a8dd9b700 1 -- 172.21.15.47:0/239626236 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7c0098f0 con 0x7f9a900b5370 2020-07-18T03:34:16.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a8dd9b700 1 -- 172.21.15.47:0/239626236 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c009a50 con 0x7f9a900b5370 2020-07-18T03:34:16.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a8dd9b700 1 -- 172.21.15.47:0/239626236 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7c009df0 con 0x7f9a900b5370 2020-07-18T03:34:16.782 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 -- 172.21.15.47:0/239626236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b5370 msgr2=0x7f9a9005fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 --2- 172.21.15.47:0/239626236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b5370 0x7f9a9005fff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f9a90067280 tx=0x7f9a7c006e90).stop 2020-07-18T03:34:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 -- 172.21.15.47:0/239626236 shutdown_connections 2020-07-18T03:34:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 --2- 172.21.15.47:0/239626236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b5370 0x7f9a9005fff0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 --2- 172.21.15.47:0/239626236 >> v2:172.21.15.18:3300/0 conn(0x7f9a900b41c0 0x7f9a900ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 -- 172.21.15.47:0/239626236 >> 172.21.15.47:0/239626236 conn(0x7f9a900b9340 msgr2=0x7f9a900b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 -- 172.21.15.47:0/239626236 shutdown_connections 2020-07-18T03:34:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.777+0000 7f9a951bb700 1 -- 172.21.15.47:0/239626236 wait complete. 2020-07-18T03:34:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.778+0000 7f9a951bb700 1 Processor -- start 2020-07-18T03:34:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.778+0000 7f9a951bb700 1 -- start start 2020-07-18T03:34:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a951bb700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a900ad7c0 0x7f9a90061b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a951bb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a900b41c0 0x7f9a90155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a951bb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a90156370 con 0x7f9a9013f620 2020-07-18T03:34:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a951bb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a90066750 con 0x7f9a900b41c0 2020-07-18T03:34:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a951bb700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90156610 con 0x7f9a900ad7c0 2020-07-18T03:34:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a8e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a900b41c0 0x7f9a90155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a8f59e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9a9013f620 0x7f9a9013bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44896/0 (socket says 172.21.15.47:44896) 2020-07-18T03:34:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.779+0000 7f9a8f59e700 1 -- 172.21.15.47:0/3753699261 learned_addr learned my addr 172.21.15.47:0/3753699261 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8ed9d700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a900ad7c0 0x7f9a90061b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8e59c700 1 -- 172.21.15.47:0/3753699261 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a900ad7c0 msgr2=0x7f9a90061b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8e59c700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a900ad7c0 0x7f9a90061b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8e59c700 1 -- 172.21.15.47:0/3753699261 >> v1:172.21.15.47:6789/0 conn(0x7f9a9013f620 legacy=0x7f9a9013bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8e59c700 1 -- 172.21.15.47:0/3753699261 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900663b0 con 0x7f9a900b41c0 2020-07-18T03:34:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8ed9d700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a900ad7c0 0x7f9a90061b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a8e59c700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b41c0 0x7f9a90155b10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f9a88002450 tx=0x7f9a80008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a8000ae00 con 0x7f9a900b41c0 2020-07-18T03:34:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a80007690 con 0x7f9a900b41c0 2020-07-18T03:34:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a90156000 con 0x7f9a900b41c0 2020-07-18T03:34:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a800079f0 con 0x7f9a900b41c0 2020-07-18T03:34:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.780+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9013ec20 con 0x7f9a900b41c0 2020-07-18T03:34:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.781+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a90057450 con 0x7f9a900b41c0 2020-07-18T03:34:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.781+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f9a80007c20 con 0x7f9a900b41c0 2020-07-18T03:34:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.781+0000 7f9a86ffd700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a640500d0 0x7f9a64052590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.781+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f9a80067ce0 con 0x7f9a900b41c0 2020-07-18T03:34:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.782+0000 7f9a8ed9d700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a640500d0 0x7f9a64052590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.782+0000 7f9a8ed9d700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a640500d0 0x7f9a64052590 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9a7c017000 tx=0x7f9a7c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:16.785+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f9a800403f0 con 0x7f9a900b41c0 2020-07-18T03:34:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.001+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]} v 0) v1 -- 0x7f9a900b01a0 con 0x7f9a900b41c0 2020-07-18T03:34:17.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.331+0000 7f9a86ffd700 1 -- 172.21.15.47:0/3753699261 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]}]=0 v368) v1 ==== 149+0+0 (secure 0 0 0) 0x7f9a80043a10 con 0x7f9a900b41c0 2020-07-18T03:34:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a640500d0 msgr2=0x7f9a64052590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a640500d0 0x7f9a64052590 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9a7c017000 tx=0x7f9a7c015000).stop 2020-07-18T03:34:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b41c0 msgr2=0x7f9a90155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b41c0 0x7f9a90155b10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f9a88002450 tx=0x7f9a80008040).stop 2020-07-18T03:34:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 shutdown_connections 2020-07-18T03:34:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f9a640500d0 0x7f9a64052590 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f9a900ad7c0 0x7f9a90061b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 --2- 172.21.15.47:0/3753699261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a900b41c0 0x7f9a90155b10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 >> 172.21.15.47:0/3753699261 conn(0x7f9a900b9340 msgr2=0x7f9a9005fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 shutdown_connections 2020-07-18T03:34:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.333+0000 7f9a951bb700 1 -- 172.21.15.47:0/3753699261 wait complete. 2020-07-18T03:34:17.345 INFO:tasks.cephfs_test_runner:test_create_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2020-07-18T03:34:17.346 INFO:tasks.cephfs_test_runner:Starting test: test_dir_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) 2020-07-18T03:34:17.346 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync' 2020-07-18T03:34:17.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.461+0000 7f508a190700 1 Processor -- start 2020-07-18T03:34:17.462 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.461+0000 7f508a190700 1 -- start start 2020-07-18T03:34:17.464 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f508a190700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50840ad100 0x7f50840b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:17.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f508a190700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5084147a00 0x7f5084147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:17.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f508a190700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5084147330 con 0x7f508405f5a0 2020-07-18T03:34:17.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f508a190700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840ae530 con 0x7f5084147a00 2020-07-18T03:34:17.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f508a190700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840b57e0 con 0x7f50840ad100 2020-07-18T03:34:17.465 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50837fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50840ad100 0x7f50840b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f5083fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f508405f5a0 0x7f508405f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44904/0 (socket says 172.21.15.47:44904) 2020-07-18T03:34:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f5083fff700 1 -- 172.21.15.47:0/814585633 learned_addr learned my addr 172.21.15.47:0/814585633 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50837fe700 1 -- 172.21.15.47:0/814585633 >> v2:172.21.15.18:3300/0 conn(0x7f5084147a00 msgr2=0x7f5084147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50837fe700 1 --2- 172.21.15.47:0/814585633 >> v2:172.21.15.18:3300/0 conn(0x7f5084147a00 0x7f5084147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:34:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50837fe700 1 -- 172.21.15.47:0/814585633 >> v1:172.21.15.47:6789/0 conn(0x7f508405f5a0 legacy=0x7f508405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:17.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50837fe700 1 -- 172.21.15.47:0/814585633 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840b5a80 con 0x7f50840ad100 2020-07-18T03:34:17.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50827fc700 1 -- 172.21.15.47:0/814585633 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751415722 0 0) 0x7f5084147330 con 0x7f508405f5a0 2020-07-18T03:34:17.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50837fe700 1 --2- 172.21.15.47:0/814585633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50840ad100 0x7f50840b4c10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f5074002f30 tx=0x7f5074006dd0).ready entity=mon.1 client_cookie=7a4cb7769c10b318 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:17.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.462+0000 7f50827fc700 1 -- 172.21.15.47:0/814585633 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50740097f0 con 0x7f50840ad100 2020-07-18T03:34:17.467 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f50827fc700 1 -- 172.21.15.47:0/814585633 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074009950 con 0x7f50840ad100 2020-07-18T03:34:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f50827fc700 1 -- 172.21.15.47:0/814585633 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5074009cb0 con 0x7f50840ad100 2020-07-18T03:34:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 -- 172.21.15.47:0/814585633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50840ad100 msgr2=0x7f50840b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 --2- 172.21.15.47:0/814585633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50840ad100 0x7f50840b4c10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f5074002f30 tx=0x7f5074006dd0).stop 2020-07-18T03:34:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 -- 172.21.15.47:0/814585633 shutdown_connections 2020-07-18T03:34:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 --2- 172.21.15.47:0/814585633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50840ad100 0x7f50840b4c10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 --2- 172.21.15.47:0/814585633 >> v2:172.21.15.18:3300/0 conn(0x7f5084147a00 0x7f5084147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-07-18T03:34:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 -- 172.21.15.47:0/814585633 >> 172.21.15.47:0/814585633 conn(0x7f50840b9340 msgr2=0x7f50840b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 -- 172.21.15.47:0/814585633 shutdown_connections 2020-07-18T03:34:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.463+0000 7f508a190700 1 -- 172.21.15.47:0/814585633 wait complete. 2020-07-18T03:34:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.464+0000 7f508a190700 1 Processor -- start 2020-07-18T03:34:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.464+0000 7f508a190700 1 -- start start 2020-07-18T03:34:17.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f508a190700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5084158870 0x7f508415fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:17.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f508a190700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508413e3c0 con 0x7f5084147a00 2020-07-18T03:34:17.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f508a190700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5084158df0 con 0x7f508405f5a0 2020-07-18T03:34:17.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f508a190700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5084137730 con 0x7f5084158870 2020-07-18T03:34:17.473 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f50837fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f508405f5a0 0x7f50841402d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44906/0 (socket says 172.21.15.47:44906) 2020-07-18T03:34:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f50837fe700 1 -- 172.21.15.47:0/4032893098 learned_addr learned my addr 172.21.15.47:0/4032893098 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f5083fff700 1 --2- 172.21.15.47:0/4032893098 >> v2:172.21.15.18:3300/0 conn(0x7f5084158870 0x7f508415fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687564982 0 0) 0x7f5084158df0 con 0x7f508405f5a0 2020-07-18T03:34:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505c003240 con 0x7f508405f5a0 2020-07-18T03:34:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.465+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2772867073 0 0) 0x7f505c003240 con 0x7f508405f5a0 2020-07-18T03:34:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f5083fff700 1 -- 172.21.15.47:0/4032893098 >> v1:172.21.15.47:6789/0 conn(0x7f508405f5a0 legacy=0x7f50841402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f5083fff700 1 -- 172.21.15.47:0/4032893098 >> v1:172.21.15.18:6789/0 conn(0x7f5084147a00 legacy=0x7f5084065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f5083fff700 1 -- 172.21.15.47:0/4032893098 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084159090 con 0x7f5084158870 2020-07-18T03:34:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f5083fff700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5084158870 0x7f508415fa00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f5078000f70 tx=0x7f507c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50740097f0 con 0x7f5084158870 2020-07-18T03:34:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5084159330 con 0x7f5084158870 2020-07-18T03:34:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c002060 con 0x7f5084158870 2020-07-18T03:34:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507c005b80 con 0x7f5084158870 2020-07-18T03:34:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.466+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508413f7c0 con 0x7f5084158870 2020-07-18T03:34:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.467+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084057450 con 0x7f5084158870 2020-07-18T03:34:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.467+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f507c01a060 con 0x7f5084158870 2020-07-18T03:34:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.467+0000 7f50637fe700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f505c050230 0x7f505c0526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.467+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f507c0671e0 con 0x7f5084158870 2020-07-18T03:34:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.468+0000 7f50837fe700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f505c050230 0x7f505c0526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.468+0000 7f50837fe700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f505c050230 0x7f505c0526f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5074006b40 tx=0x7f5074015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:17.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.472+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f507c03f7d0 con 0x7f5084158870 2020-07-18T03:34:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:17.690+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync"]} v 0) v1 -- 0x7f50841594c0 con 0x7f5084158870 2020-07-18T03:34:18.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.335+0000 7f50637fe700 1 -- 172.21.15.47:0/4032893098 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync"]}]=0 v369) v1 ==== 149+0+0 (secure 0 0 0) 0x7f507c042df0 con 0x7f5084158870 2020-07-18T03:34:18.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f505c050230 msgr2=0x7f505c0526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f505c050230 0x7f505c0526f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5074006b40 tx=0x7f5074015000).stop 2020-07-18T03:34:18.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5084158870 msgr2=0x7f508415fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5084158870 0x7f508415fa00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f5078000f70 tx=0x7f507c007c40).stop 2020-07-18T03:34:18.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 shutdown_connections 2020-07-18T03:34:18.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5084158870 0x7f508415fa00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:18.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 --2- 172.21.15.47:0/4032893098 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f505c050230 0x7f505c0526f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:18.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 >> 172.21.15.47:0/4032893098 conn(0x7f50840b9340 msgr2=0x7f50841469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:18.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 shutdown_connections 2020-07-18T03:34:18.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.337+0000 7f508a190700 1 -- 172.21.15.47:0/4032893098 wait complete. 2020-07-18T03:34:18.350 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-07-18T03:34:18.351 INFO:teuthology.orchestra.run.smithi018:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:34:18.408 INFO:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-07-18T03:34:18.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-07-18T03:34:18.466 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-07-18T03:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.584+0000 7fb1fe1a4700 1 Processor -- start 2020-07-18T03:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.584+0000 7fb1fe1a4700 1 -- start start 2020-07-18T03:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1f80ad100 0x7fb1f80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f8147330 con 0x7fb1f8147a00 2020-07-18T03:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f80603c0 con 0x7fb1f81443f0 2020-07-18T03:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80b57e0 con 0x7fb1f80ad100 2020-07-18T03:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1f80ad100 0x7fb1f80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1f81443f0 0x7fb1f805fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44918/0 (socket says 172.21.15.47:44918) 2020-07-18T03:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f7fff700 1 -- 172.21.15.47:0/2530989879 learned_addr learned my addr 172.21.15.47:0/2530989879 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1f80ad100 0x7fb1f80b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35594/0 (socket says 172.21.15.47:35594) 2020-07-18T03:34:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f77fe700 1 -- 172.21.15.47:0/2530989879 >> v1:172.21.15.47:6789/0 conn(0x7fb1f81443f0 legacy=0x7fb1f805fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f77fe700 1 -- 172.21.15.47:0/2530989879 >> v1:172.21.15.18:6789/0 conn(0x7fb1f8147a00 legacy=0x7fb1f805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f77fe700 1 -- 172.21.15.47:0/2530989879 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80b5a80 con 0x7fb1f80ad100 2020-07-18T03:34:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f77fe700 1 --2- 172.21.15.47:0/2530989879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f80ad100 0x7fb1f80b4c10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fb1e8002f30 tx=0x7fb1e8006dd0).ready entity=mon.1 client_cookie=73d57475b64dcb2c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f67fc700 1 -- 172.21.15.47:0/2530989879 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e80097f0 con 0x7fb1f80ad100 2020-07-18T03:34:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f67fc700 1 -- 172.21.15.47:0/2530989879 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e8009950 con 0x7fb1f80ad100 2020-07-18T03:34:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1f67fc700 1 -- 172.21.15.47:0/2530989879 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e8009cb0 con 0x7fb1f80ad100 2020-07-18T03:34:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/2530989879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f80ad100 msgr2=0x7fb1f80b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/2530989879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f80ad100 0x7fb1f80b4c10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fb1e8002f30 tx=0x7fb1e8006dd0).stop 2020-07-18T03:34:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/2530989879 shutdown_connections 2020-07-18T03:34:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/2530989879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f80ad100 0x7fb1f80b4c10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.585+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/2530989879 >> 172.21.15.47:0/2530989879 conn(0x7fb1f80b9340 msgr2=0x7fb1f80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.586+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/2530989879 shutdown_connections 2020-07-18T03:34:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.586+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/2530989879 wait complete. 2020-07-18T03:34:18.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.587+0000 7fb1fe1a4700 1 Processor -- start 2020-07-18T03:34:18.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.587+0000 7fb1fe1a4700 1 -- start start 2020-07-18T03:34:18.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1fe1a4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb1f8147a00 0x7fb1f813a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1fe1a4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1f8136c90 0x7fb1f8135cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1fe1a4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f80b16f0 con 0x7fb1f81443f0 2020-07-18T03:34:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1fe1a4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80b14e0 con 0x7fb1f8147a00 2020-07-18T03:34:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1fe1a4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f81555a0 con 0x7fb1f8136c90 2020-07-18T03:34:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1f81443f0 0x7fb1f81389e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44920/0 (socket says 172.21.15.47:44920) 2020-07-18T03:34:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f77fe700 1 -- 172.21.15.47:0/988983288 learned_addr learned my addr 172.21.15.47:0/988983288 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f7fff700 1 --2- 172.21.15.47:0/988983288 >> v2:172.21.15.47:3300/0 conn(0x7fb1f8136c90 0x7fb1f8135cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f6ffd700 1 --2- 172.21.15.47:0/988983288 >> v2:172.21.15.18:3300/0 conn(0x7fb1f8147a00 0x7fb1f813a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524110053 0 0) 0x7fb1f80b16f0 con 0x7fb1f81443f0 2020-07-18T03:34:18.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003240 con 0x7fb1f81443f0 2020-07-18T03:34:18.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f7fff700 1 -- 172.21.15.47:0/988983288 >> v2:172.21.15.18:3300/0 conn(0x7fb1f8147a00 msgr2=0x7fb1f813a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f7fff700 1 --2- 172.21.15.47:0/988983288 >> v2:172.21.15.18:3300/0 conn(0x7fb1f8147a00 0x7fb1f813a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:18.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f7fff700 1 -- 172.21.15.47:0/988983288 >> v1:172.21.15.47:6789/0 conn(0x7fb1f81443f0 legacy=0x7fb1f81389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:18.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f7fff700 1 -- 172.21.15.47:0/988983288 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80b2ba0 con 0x7fb1f8136c90 2020-07-18T03:34:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.588+0000 7fb1f7fff700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f8136c90 0x7fb1f8135cb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fb1f8147160 tx=0x7fb1f0007de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.589+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1f0005970 con 0x7fb1f8136c90 2020-07-18T03:34:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.589+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0005ad0 con 0x7fb1f8136c90 2020-07-18T03:34:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.589+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f813a850 con 0x7fb1f8136c90 2020-07-18T03:34:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.589+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1f0005e30 con 0x7fb1f8136c90 2020-07-18T03:34:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.589+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f8136360 con 0x7fb1f8136c90 2020-07-18T03:34:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.590+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f8057450 con 0x7fb1f8136c90 2020-07-18T03:34:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.590+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb1f0010030 con 0x7fb1f8136c90 2020-07-18T03:34:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.590+0000 7fb1d77fe700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb1d0050330 0x7fb1d00527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.590+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb1f0067a90 con 0x7fb1f8136c90 2020-07-18T03:34:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.590+0000 7fb1f77fe700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb1d0050330 0x7fb1d00527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.591+0000 7fb1f77fe700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb1d0050330 0x7fb1d00527f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fb1e8006a40 tx=0x7fb1e8015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.594+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb1f0040120 con 0x7fb1f8136c90 2020-07-18T03:34:18.755 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.755+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb1f806a100 con 0x7fb1f8136c90 2020-07-18T03:34:18.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.756+0000 7fb1d77fe700 1 -- 172.21.15.47:0/988983288 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v69) v1 ==== 81+0+34268 (secure 0 0 0) 0x7fb1f0043740 con 0x7fb1f8136c90 2020-07-18T03:34:18.757 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:18.758 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:34:18.758 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 69, 2020-07-18T03:34:18.758 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", 2020-07-18T03:34:18.759 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2020-07-18T03:25:24.387359+0000", 2020-07-18T03:34:18.759 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2020-07-18T03:34:07.302099+0000", 2020-07-18T03:34:18.759 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:34:18.759 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2020-07-18T03:25:33.482553+0000", 2020-07-18T03:34:18.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-07-18T03:34:18.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2020-07-18T03:34:18.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2020-07-18T03:34:18.760 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2020-07-18T03:34:18.760 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2020-07-18T03:34:18.761 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2020-07-18T03:34:18.761 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2020-07-18T03:34:18.761 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:34:18.761 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2020-07-18T03:34:18.761 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2020-07-18T03:34:18.762 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-07-18T03:34:18.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2020-07-18T03:34:18.762 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2020-07-18T03:34:18.762 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 12, 2020-07-18T03:34:18.763 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2020-07-18T03:34:18.763 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "luminous", 2020-07-18T03:34:18.763 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2020-07-18T03:34:18.763 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "pacific", 2020-07-18T03:34:18.763 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2020-07-18T03:34:18.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.764 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2020-07-18T03:34:18.764 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2020-07-18T03:34:18.764 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:27.725131+0000", 2020-07-18T03:34:18.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:34:18.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:34:18.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:34:18.765 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:34:18.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:34:18.766 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:34:18.766 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:34:18.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:34:18.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2020-07-18T03:34:18.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2020-07-18T03:34:18.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2020-07-18T03:34:18.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2020-07-18T03:34:18.767 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2020-07-18T03:34:18.767 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:34:18.767 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:34:18.768 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:34:18.768 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:34:18.768 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:34:18.768 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:34:18.768 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:34:18.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "15", 2020-07-18T03:34:18.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:34:18.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:34:18.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:34:18.771 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:34:18.772 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:34:18.772 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:34:18.772 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:34:18.772 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:34:18.772 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:34:18.773 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:34:18.773 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:34:18.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:34:18.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:34:18.774 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:34:18.774 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:34:18.774 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:34:18.774 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:34:18.774 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:34:18.775 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:34:18.775 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:34:18.775 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:34:18.775 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:34:18.775 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:34:18.776 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:34:18.776 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:34:18.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:34:18.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.776 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:34:18.777 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:34:18.777 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:34:18.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:34:18.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:34:18.778 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:34:18.778 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:34:18.778 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:34:18.778 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:34:18.778 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:34:18.779 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:34:18.779 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:34:18.779 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2020-07-18T03:34:18.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.780 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:34:18.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2020-07-18T03:34:18.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.781 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2020-07-18T03:34:18.781 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2020-07-18T03:34:18.782 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:25:37.896092+0000", 2020-07-18T03:34:18.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:34:18.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:34:18.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:34:18.782 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:34:18.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:34:18.783 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:34:18.783 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:34:18.783 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:34:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:34:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:34:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:34:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:34:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:34:18.785 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:34:18.785 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:34:18.785 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:34:18.785 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:34:18.786 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:34:18.786 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:34:18.786 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:34:18.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.786 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "18", 2020-07-18T03:34:18.787 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:34:18.787 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:34:18.787 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:34:18.787 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:34:18.788 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:34:18.788 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:34:18.788 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:34:18.788 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:34:18.788 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:34:18.789 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:34:18.789 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:34:18.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:34:18.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:34:18.789 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:34:18.790 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:34:18.790 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:34:18.790 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:34:18.790 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:34:18.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:34:18.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:34:18.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:34:18.792 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:34:18.792 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:34:18.792 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:34:18.792 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:34:18.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:34:18.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.793 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:34:18.793 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:34:18.793 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:34:18.794 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:34:18.794 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:34:18.794 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:34:18.795 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:34:18.795 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:34:18.795 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:34:18.795 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:34:18.796 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:34:18.796 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:34:18.796 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:34:18.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2020-07-18T03:34:18.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.797 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 11, 2020-07-18T03:34:18.798 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2020-07-18T03:34:18.798 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:32:57.493054+0000", 2020-07-18T03:34:18.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:34:18.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:34:18.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:34:18.799 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:34:18.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:34:18.799 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:34:18.799 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:34:18.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:34:18.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:34:18.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:34:18.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:34:18.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:34:18.801 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:34:18.801 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:34:18.801 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:34:18.801 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:34:18.801 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:34:18.802 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:34:18.802 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:34:18.802 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:34:18.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.803 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "64", 2020-07-18T03:34:18.803 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:34:18.803 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:34:18.803 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:34:18.803 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:34:18.804 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:34:18.804 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:34:18.804 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:34:18.805 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:34:18.805 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:34:18.805 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:34:18.808 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:34:18.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:34:18.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:34:18.809 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:34:18.809 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:34:18.809 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:34:18.810 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:34:18.810 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:34:18.810 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:34:18.810 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:34:18.810 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:34:18.811 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:34:18.811 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:34:18.811 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:34:18.811 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:34:18.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:34:18.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.812 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:34:18.812 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:34:18.812 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:34:18.812 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:34:18.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:34:18.813 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:34:18.813 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:34:18.813 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:34:18.814 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:34:18.814 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:34:18.814 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:34:18.814 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2020-07-18T03:34:18.815 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2020-07-18T03:34:18.815 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2020-07-18T03:34:18.815 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2020-07-18T03:34:18.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.815 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:34:18.816 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:34:18.816 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2020-07-18T03:34:18.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.817 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 12, 2020-07-18T03:34:18.817 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2020-07-18T03:34:18.817 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2020-07-18T03:32:58.764476+0000", 2020-07-18T03:34:18.818 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2020-07-18T03:34:18.818 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2020-07-18T03:34:18.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2020-07-18T03:34:18.818 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2020-07-18T03:34:18.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2020-07-18T03:34:18.819 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2020-07-18T03:34:18.819 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2020-07-18T03:34:18.819 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2020-07-18T03:34:18.819 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2020-07-18T03:34:18.820 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2020-07-18T03:34:18.820 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2020-07-18T03:34:18.820 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2020-07-18T03:34:18.820 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2020-07-18T03:34:18.820 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2020-07-18T03:34:18.821 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2020-07-18T03:34:18.821 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2020-07-18T03:34:18.821 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2020-07-18T03:34:18.821 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2020-07-18T03:34:18.821 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2020-07-18T03:34:18.822 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2020-07-18T03:34:18.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.822 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "66", 2020-07-18T03:34:18.822 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2020-07-18T03:34:18.823 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2020-07-18T03:34:18.823 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2020-07-18T03:34:18.823 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2020-07-18T03:34:18.823 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2020-07-18T03:34:18.823 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2020-07-18T03:34:18.824 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2020-07-18T03:34:18.824 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2020-07-18T03:34:18.824 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2020-07-18T03:34:18.824 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2020-07-18T03:34:18.825 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2020-07-18T03:34:18.825 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2020-07-18T03:34:18.825 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2020-07-18T03:34:18.825 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2020-07-18T03:34:18.825 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2020-07-18T03:34:18.826 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2020-07-18T03:34:18.826 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2020-07-18T03:34:18.826 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2020-07-18T03:34:18.826 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-07-18T03:34:18.826 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-07-18T03:34:18.827 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2020-07-18T03:34:18.827 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2020-07-18T03:34:18.827 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2020-07-18T03:34:18.827 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2020-07-18T03:34:18.828 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2020-07-18T03:34:18.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2020-07-18T03:34:18.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.828 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2020-07-18T03:34:18.828 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2020-07-18T03:34:18.829 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2020-07-18T03:34:18.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2020-07-18T03:34:18.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2020-07-18T03:34:18.829 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2020-07-18T03:34:18.829 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2020-07-18T03:34:18.830 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2020-07-18T03:34:18.830 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2020-07-18T03:34:18.830 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2020-07-18T03:34:18.830 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2020-07-18T03:34:18.830 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2020-07-18T03:34:18.831 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2020-07-18T03:34:18.831 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2020-07-18T03:34:18.831 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2020-07-18T03:34:18.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.832 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.832 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:34:18.832 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2020-07-18T03:34:18.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.833 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:34:18.833 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e", 2020-07-18T03:34:18.833 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.833 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.834 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.834 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.834 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.834 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.834 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.835 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 64, 2020-07-18T03:34:18.835 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.835 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.835 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2020-07-18T03:34:18.836 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2020-07-18T03:34:18.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.838 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.838 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2020-07-18T03:34:18.839 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2020-07-18T03:34:18.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.841 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.841 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2020-07-18T03:34:18.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2020-07-18T03:34:18.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.844 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.844 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2020-07-18T03:34:18.845 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.846 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2020-07-18T03:34:18.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23865 2020-07-18T03:34:18.847 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.856 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6821/23865", 2020-07-18T03:34:18.857 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6825/23865", 2020-07-18T03:34:18.857 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6832/23865", 2020-07-18T03:34:18.857 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6829/23865", 2020-07-18T03:34:18.857 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.858 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.858 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.858 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.859 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:34:18.859 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "ec8353f5-7ad4-45bf-b8e5-99115dd53e24", 2020-07-18T03:34:18.859 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.860 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.860 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.860 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.861 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.861 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 64, 2020-07-18T03:34:18.861 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.861 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.862 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2020-07-18T03:34:18.863 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.863 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2020-07-18T03:34:18.864 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.864 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.864 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.864 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.865 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.865 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.865 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2020-07-18T03:34:18.866 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2020-07-18T03:34:18.867 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.867 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.868 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.868 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.868 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2020-07-18T03:34:18.868 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2020-07-18T03:34:18.870 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.870 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.870 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.871 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.872 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2020-07-18T03:34:18.872 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2020-07-18T03:34:18.873 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23853 2020-07-18T03:34:18.873 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.873 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.874 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/23853", 2020-07-18T03:34:18.874 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/23853", 2020-07-18T03:34:18.874 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/23853", 2020-07-18T03:34:18.874 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/23853", 2020-07-18T03:34:18.874 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.875 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.875 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.875 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.875 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.876 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:34:18.876 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "dc0b1a97-efca-483d-839c-37c286699283", 2020-07-18T03:34:18.876 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.876 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.876 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.877 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.877 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.877 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 63, 2020-07-18T03:34:18.878 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.878 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.878 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.878 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.878 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2020-07-18T03:34:18.879 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.880 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2020-07-18T03:34:18.880 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.880 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.881 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.882 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.882 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.882 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2020-07-18T03:34:18.883 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.883 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.883 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2020-07-18T03:34:18.884 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.884 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.884 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.884 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.885 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2020-07-18T03:34:18.886 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.886 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.886 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2020-07-18T03:34:18.887 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.887 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.887 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.887 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.888 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.888 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2020-07-18T03:34:18.888 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.889 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.889 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2020-07-18T03:34:18.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23872 2020-07-18T03:34:18.890 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.890 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.890 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/23872", 2020-07-18T03:34:18.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6824/23872", 2020-07-18T03:34:18.891 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/23872", 2020-07-18T03:34:18.891 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6828/23872", 2020-07-18T03:34:18.891 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.892 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.892 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.892 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.893 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:34:18.893 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3f949e90-b2a0-4ef4-a63b-90f457a32c86", 2020-07-18T03:34:18.893 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.893 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.894 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.894 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.894 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.894 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.894 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.895 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 61, 2020-07-18T03:34:18.895 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.895 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2020-07-18T03:34:18.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2020-07-18T03:34:18.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.899 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.899 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2020-07-18T03:34:18.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2020-07-18T03:34:18.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.901 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2020-07-18T03:34:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2020-07-18T03:34:18.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.904 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.905 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2020-07-18T03:34:18.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2020-07-18T03:34:18.907 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 23854 2020-07-18T03:34:18.907 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.908 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.908 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/23854", 2020-07-18T03:34:18.908 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/23854", 2020-07-18T03:34:18.909 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/23854", 2020-07-18T03:34:18.909 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/23854", 2020-07-18T03:34:18.909 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.909 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.909 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.910 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.910 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:34:18.910 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "83234bc7-02d5-405a-af0d-2c30c2f80681", 2020-07-18T03:34:18.911 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.911 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.911 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.911 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.911 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.912 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.912 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.912 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 63, 2020-07-18T03:34:18.912 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.912 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.913 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6808", 2020-07-18T03:34:18.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6809", 2020-07-18T03:34:18.915 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.916 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.916 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.917 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.918 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6810", 2020-07-18T03:34:18.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.919 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6811", 2020-07-18T03:34:18.920 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.920 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.920 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.921 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.921 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6814", 2020-07-18T03:34:18.922 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6815", 2020-07-18T03:34:18.923 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.923 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.923 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6812", 2020-07-18T03:34:18.924 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6813", 2020-07-18T03:34:18.925 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24092 2020-07-18T03:34:18.925 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.926 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.926 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6809/24092", 2020-07-18T03:34:18.926 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6811/24092", 2020-07-18T03:34:18.926 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6815/24092", 2020-07-18T03:34:18.927 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6813/24092", 2020-07-18T03:34:18.927 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.927 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.927 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.928 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.928 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.928 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:34:18.928 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a8bc62ad-b199-4f39-bbd5-10a71cd953c8", 2020-07-18T03:34:18.929 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.929 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.929 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.929 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.930 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.930 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.930 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.930 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 62, 2020-07-18T03:34:18.930 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.931 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.931 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6800", 2020-07-18T03:34:18.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6801", 2020-07-18T03:34:18.933 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.934 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6802", 2020-07-18T03:34:18.935 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6803", 2020-07-18T03:34:18.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.936 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.936 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.937 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6806", 2020-07-18T03:34:18.938 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.938 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6807", 2020-07-18T03:34:18.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.939 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.939 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.940 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.941 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6804", 2020-07-18T03:34:18.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6805", 2020-07-18T03:34:18.942 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24091 2020-07-18T03:34:18.942 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.942 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.943 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6801/24091", 2020-07-18T03:34:18.943 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6803/24091", 2020-07-18T03:34:18.943 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6807/24091", 2020-07-18T03:34:18.944 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6805/24091", 2020-07-18T03:34:18.944 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.944 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.944 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.944 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:34:18.945 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3561e1c4-c30d-462b-a364-ae0be63149c9", 2020-07-18T03:34:18.945 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.946 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.946 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.946 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.946 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.947 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 63, 2020-07-18T03:34:18.947 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.947 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.948 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6816", 2020-07-18T03:34:18.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6817", 2020-07-18T03:34:18.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.950 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.951 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6819", 2020-07-18T03:34:18.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6821", 2020-07-18T03:34:18.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.953 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.953 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6828", 2020-07-18T03:34:18.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6830", 2020-07-18T03:34:18.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.958 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6823", 2020-07-18T03:34:18.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6826", 2020-07-18T03:34:18.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24093 2020-07-18T03:34:18.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.961 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6817/24093", 2020-07-18T03:34:18.961 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6821/24093", 2020-07-18T03:34:18.962 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6830/24093", 2020-07-18T03:34:18.962 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6826/24093", 2020-07-18T03:34:18.962 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.962 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.963 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.963 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.963 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:34:18.964 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "cb66e979-d9f3-40eb-946a-20ccb8c8002b", 2020-07-18T03:34:18.964 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2020-07-18T03:34:18.964 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2020-07-18T03:34:18.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2020-07-18T03:34:18.964 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2020-07-18T03:34:18.965 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2020-07-18T03:34:18.965 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2020-07-18T03:34:18.965 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 13, 2020-07-18T03:34:18.965 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 63, 2020-07-18T03:34:18.965 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2020-07-18T03:34:18.966 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2020-07-18T03:34:18.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2020-07-18T03:34:18.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6818", 2020-07-18T03:34:18.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6820", 2020-07-18T03:34:18.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.970 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2020-07-18T03:34:18.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6822", 2020-07-18T03:34:18.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6824", 2020-07-18T03:34:18.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.974 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2020-07-18T03:34:18.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6829", 2020-07-18T03:34:18.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6831", 2020-07-18T03:34:18.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.976 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2020-07-18T03:34:18.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2020-07-18T03:34:18.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2020-07-18T03:34:18.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6825", 2020-07-18T03:34:18.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:18.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.18:6827", 2020-07-18T03:34:18.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 24094 2020-07-18T03:34:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.18:6820/24094", 2020-07-18T03:34:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.18:6824/24094", 2020-07-18T03:34:18.979 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.18:6831/24094", 2020-07-18T03:34:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.18:6827/24094", 2020-07-18T03:34:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2020-07-18T03:34:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2020-07-18T03:34:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2020-07-18T03:34:18.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:18.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:34:18.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2020-07-18T03:34:18.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2020-07-18T03:34:18.982 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.982 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.982 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.982 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.982 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.982 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.933644+0000", 2020-07-18T03:34:18.983 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.983 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2020-07-18T03:34:18.983 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.984 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.984 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.984 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.984 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.929073+0000", 2020-07-18T03:34:18.985 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.985 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2020-07-18T03:34:18.985 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.985 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.986 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.986 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.986 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.986 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.974285+0000", 2020-07-18T03:34:18.986 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.987 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2020-07-18T03:34:18.987 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.988 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.988 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.988 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.988 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.988 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.909674+0000", 2020-07-18T03:34:18.989 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.989 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2020-07-18T03:34:18.989 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.989 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.990 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.990 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.990 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.990 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.935590+0000", 2020-07-18T03:34:18.990 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.991 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2020-07-18T03:34:18.991 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.991 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.992 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.992 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.992 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.916457+0000", 2020-07-18T03:34:18.992 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.993 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2020-07-18T03:34:18.993 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.993 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.993 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.994 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.994 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.994 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.990672+0000", 2020-07-18T03:34:18.994 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:18.995 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2020-07-18T03:34:18.995 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2020-07-18T03:34:18.995 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2020-07-18T03:34:18.995 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2020-07-18T03:34:18.995 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292837744639, 2020-07-18T03:34:18.996 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2020-07-18T03:34:18.996 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2020-07-18T03:25:31.948777+0000", 2020-07-18T03:34:18.996 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2020-07-18T03:34:18.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:18.996 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2020-07-18T03:34:18.997 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2020-07-18T03:34:18.997 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2020-07-18T03:34:18.997 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2020-07-18T03:34:18.997 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2020-07-18T03:34:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2020-07-18T03:34:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: "192.168.0.1:0/1325450171": "2020-07-18T04:34:06.628279+0000", 2020-07-18T03:34:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6835/2273595707": "2020-07-19T03:33:14.874762+0000", 2020-07-18T03:34:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6834/2273595707": "2020-07-19T03:33:14.874762+0000" 2020-07-18T03:34:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:18.999 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2020-07-18T03:34:18.999 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2020-07-18T03:34:18.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2020-07-18T03:34:18.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:34:18.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2020-07-18T03:34:18.999 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2020-07-18T03:34:19.000 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:34:19.000 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2020-07-18T03:34:19.000 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:34:19.000 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:34:19.000 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2020-07-18T03:34:19.001 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2020-07-18T03:34:19.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:19.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2020-07-18T03:34:19.001 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2020-07-18T03:34:19.001 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2020-07-18T03:34:19.001 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2020-07-18T03:34:19.002 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2020-07-18T03:34:19.002 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2020-07-18T03:34:19.002 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2020-07-18T03:34:19.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:19.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:19.003 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2020-07-18T03:34:19.003 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2020-07-18T03:34:19.003 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2020-07-18T03:34:19.003 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2020-07-18T03:34:19.003 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2020-07-18T03:34:19.003 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:34:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb1d0050330 msgr2=0x7fb1d00527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb1d0050330 0x7fb1d00527f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fb1e8006a40 tx=0x7fb1e8015000).stop 2020-07-18T03:34:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f8136c90 msgr2=0x7fb1f8135cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f8136c90 0x7fb1f8135cb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fb1f8147160 tx=0x7fb1f0007de0).stop 2020-07-18T03:34:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 shutdown_connections 2020-07-18T03:34:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f8136c90 0x7fb1f8135cb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/988983288 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb1d0050330 0x7fb1d00527f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 --2- 172.21.15.47:0/988983288 >> v2:172.21.15.18:3300/0 conn(0x7fb1f8147a00 0x7fb1f813a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 >> 172.21.15.47:0/988983288 conn(0x7fb1f80b9340 msgr2=0x7fb1f8150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 shutdown_connections 2020-07-18T03:34:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:18.757+0000 7fb1fe1a4700 1 -- 172.21.15.47:0/988983288 wait complete. 2020-07-18T03:34:19.008 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.123+0000 7fb82025b700 1 Processor -- start 2020-07-18T03:34:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.123+0000 7fb82025b700 1 -- start start 2020-07-18T03:34:19.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb82025b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb818147e40 0x7fb818143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb82025b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb818147330 con 0x7fb818147a00 2020-07-18T03:34:19.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb82025b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8180b57e0 con 0x7fb8180ad100 2020-07-18T03:34:19.127 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb82025b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8180ae530 con 0x7fb818147e40 2020-07-18T03:34:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81dff7700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8180ad100 0x7fb8180b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44928/0 (socket says 172.21.15.47:44928) 2020-07-18T03:34:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81dff7700 1 -- 172.21.15.47:0/3651585167 learned_addr learned my addr 172.21.15.47:0/3651585167 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81e7f8700 1 --2- 172.21.15.47:0/3651585167 >> v2:172.21.15.18:3300/0 conn(0x7fb818147e40 0x7fb818143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81cff5700 1 -- 172.21.15.47:0/3651585167 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502319941 0 0) 0x7fb8180b57e0 con 0x7fb8180ad100 2020-07-18T03:34:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81cff5700 1 -- 172.21.15.47:0/3651585167 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb800003240 con 0x7fb8180ad100 2020-07-18T03:34:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81e7f8700 1 -- 172.21.15.47:0/3651585167 >> v1:172.21.15.47:6789/0 conn(0x7fb8180ad100 legacy=0x7fb8180b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81e7f8700 1 -- 172.21.15.47:0/3651585167 >> v1:172.21.15.18:6789/0 conn(0x7fb818147a00 legacy=0x7fb8181433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81e7f8700 1 -- 172.21.15.47:0/3651585167 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8180b5a80 con 0x7fb818147e40 2020-07-18T03:34:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.124+0000 7fb81e7f8700 1 --2- 172.21.15.47:0/3651585167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb818147e40 0x7fb818143af0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fb8100032b0 tx=0x7fb8100091d0).ready entity=mon.0 client_cookie=ed304cf8c75d2ec3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb81cff5700 1 -- 172.21.15.47:0/3651585167 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb81000b9a0 con 0x7fb818147e40 2020-07-18T03:34:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb81cff5700 1 -- 172.21.15.47:0/3651585167 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000bb00 con 0x7fb818147e40 2020-07-18T03:34:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb81cff5700 1 -- 172.21.15.47:0/3651585167 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb81000bc70 con 0x7fb818147e40 2020-07-18T03:34:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 -- 172.21.15.47:0/3651585167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb818147e40 msgr2=0x7fb818143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 --2- 172.21.15.47:0/3651585167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb818147e40 0x7fb818143af0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fb8100032b0 tx=0x7fb8100091d0).stop 2020-07-18T03:34:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 -- 172.21.15.47:0/3651585167 shutdown_connections 2020-07-18T03:34:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 --2- 172.21.15.47:0/3651585167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb818147e40 0x7fb818143af0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 -- 172.21.15.47:0/3651585167 >> 172.21.15.47:0/3651585167 conn(0x7fb8180b9340 msgr2=0x7fb8180b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 -- 172.21.15.47:0/3651585167 shutdown_connections 2020-07-18T03:34:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.125+0000 7fb82025b700 1 -- 172.21.15.47:0/3651585167 wait complete. 2020-07-18T03:34:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.126+0000 7fb82025b700 1 Processor -- start 2020-07-18T03:34:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.126+0000 7fb82025b700 1 -- start start 2020-07-18T03:34:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb82025b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8180ad100 0x7fb818155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb82025b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fb818149870 0x7fb81814a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb82025b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8181561c0 con 0x7fb818147a00 2020-07-18T03:34:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb82025b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb818148c90 con 0x7fb818149870 2020-07-18T03:34:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb82025b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb818148e90 con 0x7fb8180ad100 2020-07-18T03:34:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb81d7f6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb818147a00 0x7fb818148580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44932/0 (socket says 172.21.15.47:44932) 2020-07-18T03:34:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb81d7f6700 1 -- 172.21.15.47:0/1264431153 learned_addr learned my addr 172.21.15.47:0/1264431153 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb81dff7700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8180ad100 0x7fb818155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb81e7f8700 1 --2- 172.21.15.47:0/1264431153 >> v2:172.21.15.18:3300/0 conn(0x7fb818149870 0x7fb81814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340936285 0 0) 0x7fb8181561c0 con 0x7fb818147a00 2020-07-18T03:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.127+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f0003240 con 0x7fb818147a00 2020-07-18T03:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1691542310 0 0) 0x7fb7f0003240 con 0x7fb818147a00 2020-07-18T03:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb8181561c0 con 0x7fb818147a00 2020-07-18T03:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fb8080019c0 con 0x7fb818147a00 2020-07-18T03:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb81dff7700 1 -- 172.21.15.47:0/1264431153 >> v2:172.21.15.18:3300/0 conn(0x7fb818149870 msgr2=0x7fb81814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb81dff7700 1 --2- 172.21.15.47:0/1264431153 >> v2:172.21.15.18:3300/0 conn(0x7fb818149870 0x7fb81814a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb81dff7700 1 -- 172.21.15.47:0/1264431153 >> v1:172.21.15.47:6789/0 conn(0x7fb818147a00 legacy=0x7fb818148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb81dff7700 1 -- 172.21.15.47:0/1264431153 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818149130 con 0x7fb8180ad100 2020-07-18T03:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb81dff7700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8180ad100 0x7fb818155cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb8140010d0 tx=0x7fb814007940).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8140070d0 con 0x7fb8180ad100 2020-07-18T03:34:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814007230 con 0x7fb8180ad100 2020-07-18T03:34:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8140055f0 con 0x7fb8180ad100 2020-07-18T03:34:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8181493d0 con 0x7fb8180ad100 2020-07-18T03:34:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.128+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81814a850 con 0x7fb8180ad100 2020-07-18T03:34:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.129+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb818149560 con 0x7fb8180ad100 2020-07-18T03:34:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.129+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fb814010070 con 0x7fb8180ad100 2020-07-18T03:34:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.130+0000 7fb805ffb700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb7f0050670 0x7fb7f0052b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.130+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb814066f80 con 0x7fb8180ad100 2020-07-18T03:34:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.130+0000 7fb81d7f6700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb7f0050670 0x7fb7f0052b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.130+0000 7fb81d7f6700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb7f0050670 0x7fb7f0052b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fb8080016d0 tx=0x7fb808007660).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.134+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fb81403f610 con 0x7fb8180ad100 2020-07-18T03:34:19.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.345+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb818057450 con 0x7fb8180ad100 2020-07-18T03:34:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.346+0000 7fb805ffb700 1 -- 172.21.15.47:0/1264431153 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2138 (secure 0 0 0) 0x7fb814042c30 con 0x7fb8180ad100 2020-07-18T03:34:19.347 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:19.347 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:34:09.311823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6228":{"gid":6228,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.47:6835/1654366847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1654366847},{"type":"v1","addr":"172.21.15.47:6835","nonce":1654366847}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:19.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb7f0050670 msgr2=0x7fb7f0052b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb7f0050670 0x7fb7f0052b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fb8080016d0 tx=0x7fb808007660).stop 2020-07-18T03:34:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8180ad100 msgr2=0x7fb818155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8180ad100 0x7fb818155cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb8140010d0 tx=0x7fb814007940).stop 2020-07-18T03:34:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 shutdown_connections 2020-07-18T03:34:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 --2- 172.21.15.47:0/1264431153 >> v2:172.21.15.18:3300/0 conn(0x7fb818149870 0x7fb81814a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fb8180ad100 0x7fb818155cd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 --2- 172.21.15.47:0/1264431153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fb7f0050670 0x7fb7f0052b30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 >> 172.21.15.47:0/1264431153 conn(0x7fb8180b9340 msgr2=0x7fb8181469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 shutdown_connections 2020-07-18T03:34:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.347+0000 7fb82025b700 1 -- 172.21.15.47:0/1264431153 wait complete. 2020-07-18T03:34:19.351 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 36 2020-07-18T03:34:19.359 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.474+0000 7f138e250700 1 Processor -- start 2020-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.474+0000 7f138e250700 1 -- start start 2020-07-18T03:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.474+0000 7f138e250700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 0x7f13880625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.474+0000 7f138e250700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880b5830 con 0x7f13880aea60 2020-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.474+0000 7f138e250700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1388146da0 con 0x7f13881471d0 2020-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.474+0000 7f138e250700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1388147d70 con 0x7f138805ffb0 2020-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f1387fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13881471d0 0x7f1388147660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44940/0 (socket says 172.21.15.47:44940) 2020-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f1387fff700 1 -- 172.21.15.47:0/1444591942 learned_addr learned my addr 172.21.15.47:0/1444591942 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13877fe700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 0x7f13880625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544194921 0 0) 0x7f1388146da0 con 0x7f13881471d0 2020-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1368003240 con 0x7f13881471d0 2020-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1088300870 0 0) 0x7f1368003240 con 0x7f13881471d0 2020-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 msgr2=0x7f13880625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 0x7f13880625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 >> v1:172.21.15.18:6789/0 conn(0x7f13880aea60 legacy=0x7f1388142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880b5a60 con 0x7f13881471d0 2020-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f1378001710 con 0x7f13881471d0 2020-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 >> v1:172.21.15.47:6789/0 conn(0x7f13881471d0 legacy=0x7f1388147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1368003730 0x7f1368003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 0x7f13680044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13680049d0 0x7f1368004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388147d70 con 0x7f1368003730 2020-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13680041d0 con 0x7f13680049d0 2020-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136800b730 con 0x7f138805ffb0 2020-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f1387fff700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13680049d0 0x7f1368004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f1386ffd700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 0x7f13680044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.475+0000 7f13877fe700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1368003730 0x7f1368003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 -- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 msgr2=0x7f13680044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f138805ffb0 0x7f13680044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 -- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1368003730 msgr2=0x7f1368003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1368003730 0x7f1368003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 -- 172.21.15.47:0/1444591942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1368003240 con 0x7f13680049d0 2020-07-18T03:34:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13680049d0 0x7f1368004e50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f1378001420 tx=0x7f13780076d0).ready entity=mon.1 client_cookie=8a0154eb92d967ad server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13780070d0 con 0x7f13680049d0 2020-07-18T03:34:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378007230 con 0x7f13680049d0 2020-07-18T03:34:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f13867fc700 1 -- 172.21.15.47:0/1444591942 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13780055f0 con 0x7f13680049d0 2020-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f138e250700 1 -- 172.21.15.47:0/1444591942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13680049d0 msgr2=0x7f1368004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f138e250700 1 --2- 172.21.15.47:0/1444591942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13680049d0 0x7f1368004e50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f1378001420 tx=0x7f13780076d0).stop 2020-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f1387fff700 1 -- 172.21.15.47:0/1444591942 reap_dead start 2020-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f138e250700 1 -- 172.21.15.47:0/1444591942 shutdown_connections 2020-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f138e250700 1 -- 172.21.15.47:0/1444591942 >> 172.21.15.47:0/1444591942 conn(0x7f13880b93c0 msgr2=0x7f13880b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:19.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f138e250700 1 -- 172.21.15.47:0/1444591942 shutdown_connections 2020-07-18T03:34:19.487 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.476+0000 7f138e250700 1 -- 172.21.15.47:0/1444591942 wait complete. 2020-07-18T03:34:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f138e250700 1 Processor -- start 2020-07-18T03:34:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f138e250700 1 -- start start 2020-07-18T03:34:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f138e250700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1388145920 0x7f1388151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f138e250700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1388144f40 con 0x7f1388136580 2020-07-18T03:34:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f138e250700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13881451e0 con 0x7f13881361a0 2020-07-18T03:34:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f138e250700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1388145480 con 0x7f1388145920 2020-07-18T03:34:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f13877fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13881361a0 0x7f1388143910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44950/0 (socket says 172.21.15.47:44950) 2020-07-18T03:34:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.478+0000 7f13877fe700 1 -- 172.21.15.47:0/2570074598 learned_addr learned my addr 172.21.15.47:0/2570074598 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f1387fff700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1388145920 0x7f1388151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146425385 0 0) 0x7f13881451e0 con 0x7f13881361a0 2020-07-18T03:34:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1364003240 con 0x7f13881361a0 2020-07-18T03:34:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3785889499 0 0) 0x7f1364003240 con 0x7f13881361a0 2020-07-18T03:34:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13881451e0 con 0x7f13881361a0 2020-07-18T03:34:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f1380009bc0 con 0x7f13881361a0 2020-07-18T03:34:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1388145920 msgr2=0x7f1388151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1388145920 0x7f1388151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 >> v1:172.21.15.47:6789/0 conn(0x7f13881361a0 legacy=0x7f1388143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 >> v1:172.21.15.18:6789/0 conn(0x7f1388136580 legacy=0x7f13880b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1364003800 0x7f1364003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13640044e0 0x7f1364004940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1364004e30 0x7f136400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388145480 con 0x7f1364004e30 2020-07-18T03:34:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364004200 con 0x7f13640044e0 2020-07-18T03:34:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136400bc20 con 0x7f1364003800 2020-07-18T03:34:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f1387fff700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1388145920 0x7f1388151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f1386ffd700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13640044e0 0x7f1364004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f13877fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1364003800 0x7f1364003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.479+0000 7f1387fff700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1364004e30 0x7f136400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f1386ffd700 1 -- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1364003800 msgr2=0x7f1364003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f1386ffd700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1364003800 0x7f1364003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f1386ffd700 1 -- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1364004e30 msgr2=0x7f136400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f1386ffd700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1364004e30 0x7f136400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f1386ffd700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880b36b0 con 0x7f13640044e0 2020-07-18T03:34:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1388144b70 con 0x7f13640044e0 2020-07-18T03:34:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1388151200 con 0x7f13640044e0 2020-07-18T03:34:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f1386ffd700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13640044e0 0x7f1364004940 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f138805c960 tx=0x7f1370007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f137000e8f0 con 0x7f13640044e0 2020-07-18T03:34:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137000ea50 con 0x7f13640044e0 2020-07-18T03:34:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.480+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f137000ebc0 con 0x7f13640044e0 2020-07-18T03:34:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.481+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1388145d30 con 0x7f13640044e0 2020-07-18T03:34:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.481+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1370009d70 con 0x7f13640044e0 2020-07-18T03:34:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.481+0000 7f137f7fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1364058760 0x7f136405ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.481+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f1370067990 con 0x7f13640044e0 2020-07-18T03:34:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.481+0000 7f13877fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1364058760 0x7f136405ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.481+0000 7f13877fe700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1364058760 0x7f136405ac20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f1380012000 tx=0x7f13800091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.485+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1388145d30 con 0x7f13640044e0 2020-07-18T03:34:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:19.697+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1388145d30 con 0x7f13640044e0 2020-07-18T03:34:20.335 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:34:20.334+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:34:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.338+0000 7f137f7fe700 1 -- 172.21.15.47:0/2570074598 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1388145d30 con 0x7f13640044e0 2020-07-18T03:34:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.340+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1364058760 msgr2=0x7f136405ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.340+0000 7f138e250700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1364058760 0x7f136405ac20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f1380012000 tx=0x7f13800091d0).stop 2020-07-18T03:34:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.340+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13640044e0 msgr2=0x7f1364004940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.340+0000 7f138e250700 1 --2- 172.21.15.47:0/2570074598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13640044e0 0x7f1364004940 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f138805c960 tx=0x7f1370007b70).stop 2020-07-18T03:34:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.340+0000 7f1387fff700 1 -- 172.21.15.47:0/2570074598 reap_dead start 2020-07-18T03:34:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.340+0000 7f1387fff700 1 -- 172.21.15.47:0/2570074598 reap_dead start 2020-07-18T03:34:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.341+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 shutdown_connections 2020-07-18T03:34:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.341+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 >> 172.21.15.47:0/2570074598 conn(0x7f13880b93c0 msgr2=0x7f13881422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.341+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 shutdown_connections 2020-07-18T03:34:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.341+0000 7f138e250700 1 -- 172.21.15.47:0/2570074598 wait complete. 2020-07-18T03:34:20.344 INFO:tasks.ceph.mds.a.smithi047.stderr: -961> 2020-07-18T03:34:12.452+0000 7f0a5434b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-07-18T03:34:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-07-18T03:34:20.355 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:20.364 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:34:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.476+0000 7f38aae2b700 1 Processor -- start 2020-07-18T03:34:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.476+0000 7f38aae2b700 1 -- start start 2020-07-18T03:34:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.476+0000 7f38aae2b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f38a4148360 0x7f38a41487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.476+0000 7f38aae2b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a4066530 con 0x7f38a4148ce0 2020-07-18T03:34:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.476+0000 7f38aae2b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a4066790 con 0x7f38a40b60d0 2020-07-18T03:34:20.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.476+0000 7f38aae2b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40b64b0 con 0x7f38a4148360 2020-07-18T03:34:20.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a8bc7700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38a40b60d0 0x7f38a40ba4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44982/0 (socket says 172.21.15.47:44982) 2020-07-18T03:34:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a8bc7700 1 -- 172.21.15.47:0/1225510260 learned_addr learned my addr 172.21.15.47:0/1225510260 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a3fff700 1 --2- 172.21.15.47:0/1225510260 >> v2:172.21.15.18:3300/0 conn(0x7f38a4148360 0x7f38a41487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018914681 0 0) 0x7f38a4066790 con 0x7f38a40b60d0 2020-07-18T03:34:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3884003240 con 0x7f38a40b60d0 2020-07-18T03:34:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3217547204 0 0) 0x7f3884003240 con 0x7f38a40b60d0 2020-07-18T03:34:20.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 >> v2:172.21.15.18:3300/0 conn(0x7f38a4148360 msgr2=0x7f38a41487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:20.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 --2- 172.21.15.47:0/1225510260 >> v2:172.21.15.18:3300/0 conn(0x7f38a4148360 0x7f38a41487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 >> v1:172.21.15.18:6789/0 conn(0x7f38a4148ce0 legacy=0x7f38a4144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4062de0 con 0x7f38a40b60d0 2020-07-18T03:34:20.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f388c0016d0 con 0x7f38a40b60d0 2020-07-18T03:34:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 >> v1:172.21.15.47:6789/0 conn(0x7f38a40b60d0 legacy=0x7f38a40ba4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3884003730 0x7f3884003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38840044e0 0x7f3884004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3884004e50 0x7f388400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40b64b0 con 0x7f3884003730 2020-07-18T03:34:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840042c0 con 0x7f3884004e50 2020-07-18T03:34:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388400bc20 con 0x7f38840044e0 2020-07-18T03:34:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.477+0000 7f38a93c8700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3884004e50 0x7f388400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a8bc7700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3884003730 0x7f3884003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a3fff700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38840044e0 0x7f3884004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 -- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38840044e0 msgr2=0x7f3884004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f38840044e0 0x7f3884004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 -- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3884003730 msgr2=0x7f3884003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3884003730 0x7f3884003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 -- 172.21.15.47:0/1225510260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884003240 con 0x7f3884004e50 2020-07-18T03:34:20.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 -- 172.21.15.47:0/1225510260 reap_dead start 2020-07-18T03:34:20.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a93c8700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3884004e50 0x7f388400b730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f389c001360 tx=0x7f389c007e40).ready entity=mon.1 client_cookie=15134696f4b245a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:20.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f389c00ca50 con 0x7f3884004e50 2020-07-18T03:34:20.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00cbb0 con 0x7f3884004e50 2020-07-18T03:34:20.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38a37fe700 1 -- 172.21.15.47:0/1225510260 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f389c012400 con 0x7f3884004e50 2020-07-18T03:34:20.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 -- 172.21.15.47:0/1225510260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3884004e50 msgr2=0x7f388400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3884004e50 0x7f388400b730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f389c001360 tx=0x7f389c007e40).stop 2020-07-18T03:34:20.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 -- 172.21.15.47:0/1225510260 shutdown_connections 2020-07-18T03:34:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 --2- 172.21.15.47:0/1225510260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3884004e50 0x7f388400b730 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 -- 172.21.15.47:0/1225510260 >> 172.21.15.47:0/1225510260 conn(0x7f38a40b7100 msgr2=0x7f38a40b9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 -- 172.21.15.47:0/1225510260 shutdown_connections 2020-07-18T03:34:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.478+0000 7f38aae2b700 1 -- 172.21.15.47:0/1225510260 wait complete. 2020-07-18T03:34:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.480+0000 7f38aae2b700 1 Processor -- start 2020-07-18T03:34:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.480+0000 7f38aae2b700 1 -- start start 2020-07-18T03:34:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38aae2b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f38a40afb40 0x7f38a40b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38aae2b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a413ac00 con 0x7f38a40b3110 2020-07-18T03:34:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38aae2b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a40b3630 con 0x7f38a40b4010 2020-07-18T03:34:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38aae2b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40b38d0 con 0x7f38a40afb40 2020-07-18T03:34:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38a40b4010 0x7f38a4139de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44996/0 (socket says 172.21.15.47:44996) 2020-07-18T03:34:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a3fff700 1 -- 172.21.15.47:0/3822584497 learned_addr learned my addr 172.21.15.47:0/3822584497 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a8bc7700 1 --2- 172.21.15.47:0/3822584497 >> v2:172.21.15.18:3300/0 conn(0x7f38a40afb40 0x7f38a40b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614108724 0 0) 0x7f38a40b3630 con 0x7f38a40b4010 2020-07-18T03:34:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3880003240 con 0x7f38a40b4010 2020-07-18T03:34:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a8bc7700 1 -- 172.21.15.47:0/3822584497 >> v1:172.21.15.47:6789/0 conn(0x7f38a40b4010 legacy=0x7f38a4139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81091243 0 0) 0x7f38a413ac00 con 0x7f38a40b3110 2020-07-18T03:34:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a8bc7700 1 -- 172.21.15.47:0/3822584497 >> v1:172.21.15.18:6789/0 conn(0x7f38a40b3110 legacy=0x7f38a413a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a8bc7700 1 -- 172.21.15.47:0/3822584497 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40b3b70 con 0x7f38a40afb40 2020-07-18T03:34:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a8bc7700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38a40afb40 0x7f38a40b46e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f388c00d000 tx=0x7f38a4066920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.481+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388c011940 con 0x7f38a40afb40 2020-07-18T03:34:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.482+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c011aa0 con 0x7f38a40afb40 2020-07-18T03:34:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.482+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a40ae2c0 con 0x7f38a40afb40 2020-07-18T03:34:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.482+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f388c011e00 con 0x7f38a40afb40 2020-07-18T03:34:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.482+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a40b4bd0 con 0x7f38a40afb40 2020-07-18T03:34:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.482+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a40b4eb0 con 0x7f38a40afb40 2020-07-18T03:34:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.483+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f388c01b030 con 0x7f38a40afb40 2020-07-18T03:34:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.483+0000 7f38a0ff9700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f38800502e0 0x7f38800527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.483+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6635+0+0 (secure 0 0 0) 0x7f388c067520 con 0x7f38a40afb40 2020-07-18T03:34:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.483+0000 7f38a3fff700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f38800502e0 0x7f38800527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.483+0000 7f38a3fff700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f38800502e0 0x7f38800527a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f389400a000 tx=0x7f3894008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.487+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f388c03fbc0 con 0x7f38a40afb40 2020-07-18T03:34:20.559 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:34:20.563 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:34:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.702+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a40b3d00 con 0x7f38a40afb40 2020-07-18T03:34:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.703+0000 7f38a0ff9700 1 -- 172.21.15.47:0/3822584497 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1775 (secure 0 0 0) 0x7f388c019020 con 0x7f38a40afb40 2020-07-18T03:34:20.704 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:20.705 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.18:6833/1870665522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":1870665522},{"type":"v1","addr":"172.21.15.18:6833","nonce":1870665522}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:32:59.940158+0000","modified":"2020-07-18T03:34:20.335315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-07-18T03:34:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f38800502e0 msgr2=0x7f38800527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f38800502e0 0x7f38800527a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f389400a000 tx=0x7f3894008040).stop 2020-07-18T03:34:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38a40afb40 msgr2=0x7f38a40b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38a40afb40 0x7f38a40b46e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f388c00d000 tx=0x7f38a4066920).stop 2020-07-18T03:34:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 shutdown_connections 2020-07-18T03:34:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38a40afb40 0x7f38a40b46e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 --2- 172.21.15.47:0/3822584497 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f38800502e0 0x7f38800527a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 >> 172.21.15.47:0/3822584497 conn(0x7f38a40b7100 msgr2=0x7f38a40b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 shutdown_connections 2020-07-18T03:34:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.704+0000 7f38aae2b700 1 -- 172.21.15.47:0/3822584497 wait complete. 2020-07-18T03:34:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 37 2020-07-18T03:34:20.716 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-07-18T03:34:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.832+0000 7f3558bb4700 1 Processor -- start 2020-07-18T03:34:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.832+0000 7f3558bb4700 1 -- start start 2020-07-18T03:34:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3558bb4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3554144e70 0x7f35540b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3558bb4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3554142430 0x7f3554148b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3558bb4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3554061db0 con 0x7f3554141fb0 2020-07-18T03:34:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3558bb4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35540b14c0 con 0x7f3554142430 2020-07-18T03:34:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3558bb4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35541428c0 con 0x7f3554144e70 2020-07-18T03:34:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3554144e70 0x7f35540b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3551d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3554141fb0 0x7f3554148440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45006/0 (socket says 172.21.15.47:45006) 2020-07-18T03:34:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3551d9b700 1 -- 172.21.15.47:0/9655469 learned_addr learned my addr 172.21.15.47:0/9655469 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3554144e70 0x7f35540b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35682/0 (socket says 172.21.15.47:35682) 2020-07-18T03:34:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f3552d9d700 1 --2- 172.21.15.47:0/9655469 >> v2:172.21.15.18:3300/0 conn(0x7f3554142430 0x7f3554148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 -- 172.21.15.47:0/9655469 >> v2:172.21.15.18:3300/0 conn(0x7f3554142430 msgr2=0x7f3554148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 --2- 172.21.15.47:0/9655469 >> v2:172.21.15.18:3300/0 conn(0x7f3554142430 0x7f3554148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 -- 172.21.15.47:0/9655469 >> v1:172.21.15.47:6789/0 conn(0x7f3554141fb0 legacy=0x7f3554148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355159a700 1 -- 172.21.15.47:0/9655469 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704249383 0 0) 0x7f3554061db0 con 0x7f3554141fb0 2020-07-18T03:34:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 -- 172.21.15.47:0/9655469 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554062050 con 0x7f3554144e70 2020-07-18T03:34:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.833+0000 7f355259c700 1 --2- 172.21.15.47:0/9655469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554144e70 0x7f35540b5f60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f3548002f30 tx=0x7f3548006dd0).ready entity=mon.1 client_cookie=6586399b8848e9d5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f355159a700 1 -- 172.21.15.47:0/9655469 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35480097f0 con 0x7f3554144e70 2020-07-18T03:34:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f355159a700 1 -- 172.21.15.47:0/9655469 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548009950 con 0x7f3554144e70 2020-07-18T03:34:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f355159a700 1 -- 172.21.15.47:0/9655469 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3548009cb0 con 0x7f3554144e70 2020-07-18T03:34:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 -- 172.21.15.47:0/9655469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554144e70 msgr2=0x7f35540b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 --2- 172.21.15.47:0/9655469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554144e70 0x7f35540b5f60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f3548002f30 tx=0x7f3548006dd0).stop 2020-07-18T03:34:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 -- 172.21.15.47:0/9655469 shutdown_connections 2020-07-18T03:34:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 --2- 172.21.15.47:0/9655469 >> v2:172.21.15.18:3300/0 conn(0x7f3554142430 0x7f3554148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 --2- 172.21.15.47:0/9655469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554144e70 0x7f35540b5f60 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 -- 172.21.15.47:0/9655469 >> 172.21.15.47:0/9655469 conn(0x7f355405f5a0 msgr2=0x7f355405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 -- 172.21.15.47:0/9655469 shutdown_connections 2020-07-18T03:34:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.834+0000 7f3558bb4700 1 -- 172.21.15.47:0/9655469 wait complete. 2020-07-18T03:34:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.835+0000 7f3558bb4700 1 Processor -- start 2020-07-18T03:34:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.835+0000 7f3558bb4700 1 -- start start 2020-07-18T03:34:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3558bb4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f35540669f0 0x7f3554159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3558bb4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35541588b0 con 0x7f3554141fb0 2020-07-18T03:34:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3558bb4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3554158b50 con 0x7f3554142430 2020-07-18T03:34:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3558bb4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3554158df0 con 0x7f35540669f0 2020-07-18T03:34:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3551d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3554142430 0x7f35540b9ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45010/0 (socket says 172.21.15.47:45010) 2020-07-18T03:34:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3551d9b700 1 -- 172.21.15.47:0/4218175983 learned_addr learned my addr 172.21.15.47:0/4218175983 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f3552d9d700 1 --2- 172.21.15.47:0/4218175983 >> v2:172.21.15.18:3300/0 conn(0x7f35540669f0 0x7f3554159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161605465 0 0) 0x7f3554158b50 con 0x7f3554142430 2020-07-18T03:34:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.836+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3528003240 con 0x7f3554142430 2020-07-18T03:34:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f3552d9d700 1 -- 172.21.15.47:0/4218175983 >> v1:172.21.15.47:6789/0 conn(0x7f3554142430 legacy=0x7f35540b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f3552d9d700 1 -- 172.21.15.47:0/4218175983 >> v1:172.21.15.18:6789/0 conn(0x7f3554141fb0 legacy=0x7f3554155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870253337 0 0) 0x7f35541588b0 con 0x7f3554141fb0 2020-07-18T03:34:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f3552d9d700 1 -- 172.21.15.47:0/4218175983 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35541547e0 con 0x7f35540669f0 2020-07-18T03:34:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f3552d9d700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35540669f0 0x7f3554159290 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f354c00a000 tx=0x7f354c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f354c009980 con 0x7f35540669f0 2020-07-18T03:34:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c009ae0 con 0x7f35540669f0 2020-07-18T03:34:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3554154a30 con 0x7f35540669f0 2020-07-18T03:34:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f354c009e40 con 0x7f35540669f0 2020-07-18T03:34:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.837+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3554154f00 con 0x7f35540669f0 2020-07-18T03:34:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.838+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3554155120 con 0x7f35540669f0 2020-07-18T03:34:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.838+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f354c013480 con 0x7f35540669f0 2020-07-18T03:34:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.838+0000 7f35427fc700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35280502b0 0x7f3528052770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.839+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6635+0+0 (secure 0 0 0) 0x7f354c068500 con 0x7f35540669f0 2020-07-18T03:34:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.839+0000 7f355259c700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35280502b0 0x7f3528052770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.839+0000 7f355259c700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35280502b0 0x7f3528052770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f3548002ef0 tx=0x7f3548015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:20.842+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f354c040ae0 con 0x7f35540669f0 2020-07-18T03:34:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.052+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f35540b1600 con 0x7f35540669f0 2020-07-18T03:34:21.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.343+0000 7f35427fc700 1 -- 172.21.15.47:0/4218175983 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f354c044100 con 0x7f35540669f0 2020-07-18T03:34:21.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35280502b0 msgr2=0x7f3528052770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:21.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35280502b0 0x7f3528052770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f3548002ef0 tx=0x7f3548015000).stop 2020-07-18T03:34:21.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35540669f0 msgr2=0x7f3554159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35540669f0 0x7f3554159290 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f354c00a000 tx=0x7f354c008040).stop 2020-07-18T03:34:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 shutdown_connections 2020-07-18T03:34:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35540669f0 0x7f3554159290 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 --2- 172.21.15.47:0/4218175983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f35280502b0 0x7f3528052770 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0x7f3548002ef0 tx=0x7f3548015000).stop 2020-07-18T03:34:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.345+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 >> 172.21.15.47:0/4218175983 conn(0x7f355405f5a0 msgr2=0x7f35540b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:21.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.346+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 shutdown_connections 2020-07-18T03:34:21.348 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.346+0000 7f3558bb4700 1 -- 172.21.15.47:0/4218175983 wait complete. 2020-07-18T03:34:21.358 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-07-18T03:34:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.477+0000 7f32d7d77700 1 Processor -- start 2020-07-18T03:34:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.477+0000 7f32d7d77700 1 -- start start 2020-07-18T03:34:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d7d77700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d7d77700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32d0141fb0 0x7f32d0142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d7d77700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d0061db0 con 0x7f32d0148440 2020-07-18T03:34:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d7d77700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d0145250 con 0x7f32d0141fb0 2020-07-18T03:34:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d7d77700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00b14c0 con 0x7f32d0144e70 2020-07-18T03:34:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32d0141fb0 0x7f32d0142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32d0141fb0 0x7f32d0142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35700/0 (socket says 172.21.15.47:35700) 2020-07-18T03:34:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 -- 172.21.15.47:0/3435018040 learned_addr learned my addr 172.21.15.47:0/3435018040 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5b13700 1 --2- 172.21.15.47:0/3435018040 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 -- 172.21.15.47:0/3435018040 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 msgr2=0x7f32d00b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 --2- 172.21.15.47:0/3435018040 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 -- 172.21.15.47:0/3435018040 >> v1:172.21.15.18:6789/0 conn(0x7f32d0148440 legacy=0x7f32d01488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 -- 172.21.15.47:0/3435018040 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0062050 con 0x7f32d0141fb0 2020-07-18T03:34:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d5312700 1 --2- 172.21.15.47:0/3435018040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0141fb0 0x7f32d0142430 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f32c4002f30 tx=0x7f32c4006e90).ready entity=mon.1 client_cookie=fcbaa5d410d3824d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.478+0000 7f32d4b11700 1 -- 172.21.15.47:0/3435018040 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c4009850 con 0x7f32d0141fb0 2020-07-18T03:34:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d4b11700 1 -- 172.21.15.47:0/3435018040 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c40099b0 con 0x7f32d0141fb0 2020-07-18T03:34:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d4b11700 1 -- 172.21.15.47:0/3435018040 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c4009d10 con 0x7f32d0141fb0 2020-07-18T03:34:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 -- 172.21.15.47:0/3435018040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0141fb0 msgr2=0x7f32d0142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 --2- 172.21.15.47:0/3435018040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0141fb0 0x7f32d0142430 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f32c4002f30 tx=0x7f32c4006e90).stop 2020-07-18T03:34:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 -- 172.21.15.47:0/3435018040 shutdown_connections 2020-07-18T03:34:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 --2- 172.21.15.47:0/3435018040 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 --2- 172.21.15.47:0/3435018040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0141fb0 0x7f32d0142430 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 -- 172.21.15.47:0/3435018040 >> 172.21.15.47:0/3435018040 conn(0x7f32d005f5a0 msgr2=0x7f32d005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 -- 172.21.15.47:0/3435018040 shutdown_connections 2020-07-18T03:34:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.479+0000 7f32d7d77700 1 -- 172.21.15.47:0/3435018040 wait complete. 2020-07-18T03:34:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.480+0000 7f32d7d77700 1 Processor -- start 2020-07-18T03:34:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.480+0000 7f32d7d77700 1 -- start start 2020-07-18T03:34:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d7d77700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d7d77700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32d0148440 0x7f32d0065490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d7d77700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f32d00b8c00 0x7f32d00ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d7d77700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d0153aa0 con 0x7f32d00b8c00 2020-07-18T03:34:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d7d77700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d0153ca0 con 0x7f32d0148440 2020-07-18T03:34:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d7d77700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0153f40 con 0x7f32d0144e70 2020-07-18T03:34:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32d0148440 0x7f32d0065490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32d0148440 0x7f32d0065490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35706/0 (socket says 172.21.15.47:35706) 2020-07-18T03:34:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 -- 172.21.15.47:0/1129300880 learned_addr learned my addr 172.21.15.47:0/1129300880 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5b13700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d6314700 1 --2- 172.21.15.47:0/1129300880 >> v2:172.21.15.18:3300/0 conn(0x7f32d00b8c00 0x7f32d00ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 -- 172.21.15.47:0/1129300880 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 msgr2=0x7f32d00b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 -- 172.21.15.47:0/1129300880 >> v2:172.21.15.18:3300/0 conn(0x7f32d00b8c00 msgr2=0x7f32d00ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 --2- 172.21.15.47:0/1129300880 >> v2:172.21.15.18:3300/0 conn(0x7f32d00b8c00 0x7f32d00ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 -- 172.21.15.47:0/1129300880 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00b7180 con 0x7f32d0148440 2020-07-18T03:34:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.481+0000 7f32d5312700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0148440 0x7f32d0065490 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f32c4002f30 tx=0x7f32c4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.482+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c401e460 con 0x7f32d0148440 2020-07-18T03:34:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.482+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c401e5c0 con 0x7f32d0148440 2020-07-18T03:34:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.482+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c401e920 con 0x7f32d0148440 2020-07-18T03:34:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.482+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d0065980 con 0x7f32d0148440 2020-07-18T03:34:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.482+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0065e70 con 0x7f32d0148440 2020-07-18T03:34:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.483+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d00b7310 con 0x7f32d0148440 2020-07-18T03:34:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.483+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f32c4025030 con 0x7f32d0148440 2020-07-18T03:34:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.483+0000 7f32cdffb700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f32a8050060 0x7f32a8052520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.483+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6635+0+0 (secure 0 0 0) 0x7f32c40693f0 con 0x7f32d0148440 2020-07-18T03:34:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.483+0000 7f32d5b13700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f32a8050060 0x7f32a8052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.484+0000 7f32d5b13700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f32a8050060 0x7f32a8052520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f32c800b000 tx=0x7f32c80091d0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.487+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f32c4041ab0 con 0x7f32d0148440 2020-07-18T03:34:21.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:21.644+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32d005b120 con 0x7f32d0148440 2020-07-18T03:34:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.351+0000 7f32cdffb700 1 -- 172.21.15.47:0/1129300880 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) v1 ==== 185+0+0 (secure 0 0 0) 0x7f32c40230e0 con 0x7f32d0148440 2020-07-18T03:34:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f32a8050060 msgr2=0x7f32a8052520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f32a8050060 0x7f32a8052520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f32c800b000 tx=0x7f32c80091d0).stop 2020-07-18T03:34:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0148440 msgr2=0x7f32d0065490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0148440 0x7f32d0065490 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f32c4002f30 tx=0x7f32c4015000).stop 2020-07-18T03:34:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 shutdown_connections 2020-07-18T03:34:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f32a8050060 0x7f32a8052520 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 --2- 172.21.15.47:0/1129300880 >> v2:172.21.15.18:3300/0 conn(0x7f32d00b8c00 0x7f32d00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f32d0144e70 0x7f32d00b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 --2- 172.21.15.47:0/1129300880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d0148440 0x7f32d0065490 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 >> 172.21.15.47:0/1129300880 conn(0x7f32d005f5a0 msgr2=0x7f32d00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 shutdown_connections 2020-07-18T03:34:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.354+0000 7f32d7d77700 1 -- 172.21.15.47:0/1129300880 wait complete. 2020-07-18T03:34:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2020-07-18T03:34:22.367 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-07-18T03:34:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.490+0000 7fc6bcd7b700 1 Processor -- start 2020-07-18T03:34:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.490+0000 7fc6bcd7b700 1 -- start start 2020-07-18T03:34:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6bcd7b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 0x7fc6b80620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6bcd7b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 0x7fc6b80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6bcd7b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80af4b0 con 0x7fc6b8142a20 2020-07-18T03:34:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6bcd7b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b80b6d80 con 0x7fc6b8061c60 2020-07-18T03:34:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6bcd7b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80af710 con 0x7fc6b80b3af0 2020-07-18T03:34:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6b6d9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 0x7fc6b80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6b5d9b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 0x7fc6b80620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6b5d9b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 0x7fc6b80620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:49624/0 (socket says 172.21.15.47:49624) 2020-07-18T03:34:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6b6d9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 0x7fc6b80b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56546/0 (socket says 172.21.15.47:56546) 2020-07-18T03:34:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6b5d9b700 1 -- 172.21.15.47:0/3263571943 learned_addr learned my addr 172.21.15.47:0/3263571943 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.491+0000 7fc6b659c700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fc6b8142a20 0x7fc6b8142e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35566/0 (socket says 172.21.15.47:35566) 2020-07-18T03:34:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b6d9d700 1 -- 172.21.15.47:0/3263571943 >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 msgr2=0x7fc6b80620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b6d9d700 1 --2- 172.21.15.47:0/3263571943 >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 0x7fc6b80620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b6d9d700 1 -- 172.21.15.47:0/3263571943 >> v1:172.21.15.18:6789/0 conn(0x7fc6b8142a20 legacy=0x7fc6b8142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b559a700 1 -- 172.21.15.47:0/3263571943 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485821387 0 0) 0x7fc6b80af4b0 con 0x7fc6b8142a20 2020-07-18T03:34:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b6d9d700 1 -- 172.21.15.47:0/3263571943 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80af9b0 con 0x7fc6b80b3af0 2020-07-18T03:34:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b5d9b700 1 --2- 172.21.15.47:0/3263571943 >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 0x7fc6b80620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b6d9d700 1 --2- 172.21.15.47:0/3263571943 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 0x7fc6b80b3f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc6b00032b0 tx=0x7fc6b00091d0).ready entity=mon.2 client_cookie=31ba81e75063b9ba server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b559a700 1 -- 172.21.15.47:0/3263571943 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6b000b9a0 con 0x7fc6b80b3af0 2020-07-18T03:34:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b559a700 1 -- 172.21.15.47:0/3263571943 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b000bb00 con 0x7fc6b80b3af0 2020-07-18T03:34:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6b559a700 1 -- 172.21.15.47:0/3263571943 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6b000bc70 con 0x7fc6b80b3af0 2020-07-18T03:34:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/3263571943 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 msgr2=0x7fc6b80b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/3263571943 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 0x7fc6b80b3f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc6b00032b0 tx=0x7fc6b00091d0).stop 2020-07-18T03:34:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/3263571943 shutdown_connections 2020-07-18T03:34:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/3263571943 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc6b80b3af0 0x7fc6b80b3f60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/3263571943 >> v2:172.21.15.18:3300/0 conn(0x7fc6b8061c60 0x7fc6b80620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/3263571943 >> 172.21.15.47:0/3263571943 conn(0x7fc6b805f5a0 msgr2=0x7fc6b805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/3263571943 shutdown_connections 2020-07-18T03:34:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.492+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/3263571943 wait complete. 2020-07-18T03:34:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6bcd7b700 1 Processor -- start 2020-07-18T03:34:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6bcd7b700 1 -- start start 2020-07-18T03:34:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6bcd7b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6b8142a20 0x7fc6b80b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6bcd7b700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80b8890 con 0x7fc6b8061c60 2020-07-18T03:34:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6bcd7b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80b8b30 con 0x7fc6b8146110 2020-07-18T03:34:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6bcd7b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b80b8dd0 con 0x7fc6b8142a20 2020-07-18T03:34:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.494+0000 7fc6b5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6b8142a20 0x7fc6b80b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6b8142a20 0x7fc6b80b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35720/0 (socket says 172.21.15.47:35720) 2020-07-18T03:34:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b5d9b700 1 -- 172.21.15.47:0/2545900006 learned_addr learned my addr 172.21.15.47:0/2545900006 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b6d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc6b8146110 0x7fc6b8126c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45042/0 (socket says 172.21.15.47:45042) 2020-07-18T03:34:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031742563 0 0) 0x7fc6b80b8890 con 0x7fc6b8061c60 2020-07-18T03:34:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b5d9b700 1 -- 172.21.15.47:0/2545900006 >> v1:172.21.15.47:6789/0 conn(0x7fc6b8146110 legacy=0x7fc6b8126c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b5d9b700 1 -- 172.21.15.47:0/2545900006 >> v1:172.21.15.18:6789/0 conn(0x7fc6b8061c60 legacy=0x7fc6b8143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b5d9b700 1 -- 172.21.15.47:0/2545900006 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b8127320 con 0x7fc6b8142a20 2020-07-18T03:34:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6b5d9b700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b8142a20 0x7fc6b80b8020 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fc6a800c000 tx=0x7fc6a8006e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6a800ee70 con 0x7fc6b8142a20 2020-07-18T03:34:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b80b8510 con 0x7fc6b8142a20 2020-07-18T03:34:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a800cbb0 con 0x7fc6b8142a20 2020-07-18T03:34:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6a80123f0 con 0x7fc6b8142a20 2020-07-18T03:34:22.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.495+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b80b2bc0 con 0x7fc6b8142a20 2020-07-18T03:34:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.496+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b80ad230 con 0x7fc6b8142a20 2020-07-18T03:34:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.496+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc6a80125f0 con 0x7fc6b8142a20 2020-07-18T03:34:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.496+0000 7fc6a67fc700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc68c04fdc0 0x7fc68c052280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.497+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fc6a8068430 con 0x7fc6b8142a20 2020-07-18T03:34:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.497+0000 7fc6b659c700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc68c04fdc0 0x7fc68c052280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.497+0000 7fc6b659c700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc68c04fdc0 0x7fc68c052280 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fc6ac0010f0 tx=0x7fc6ac007820).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:22.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.500+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc6a8040bc0 con 0x7fc6b8142a20 2020-07-18T03:34:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:22.661+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6b805b120 con 0x7fc6b8142a20 2020-07-18T03:34:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.352+0000 7fc6a67fc700 1 -- 172.21.15.47:0/2545900006 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc6a80441e0 con 0x7fc6b8142a20 2020-07-18T03:34:23.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc68c04fdc0 msgr2=0x7fc68c052280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.356 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc68c04fdc0 0x7fc68c052280 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fc6ac0010f0 tx=0x7fc6ac007820).stop 2020-07-18T03:34:23.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b8142a20 msgr2=0x7fc6b80b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b8142a20 0x7fc6b80b8020 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fc6a800c000 tx=0x7fc6a8006e90).stop 2020-07-18T03:34:23.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 shutdown_connections 2020-07-18T03:34:23.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc68c04fdc0 0x7fc68c052280 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 --2- 172.21.15.47:0/2545900006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b8142a20 0x7fc6b80b8020 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 >> 172.21.15.47:0/2545900006 conn(0x7fc6b805f5a0 msgr2=0x7fc6b80b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 shutdown_connections 2020-07-18T03:34:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.355+0000 7fc6bcd7b700 1 -- 172.21.15.47:0/2545900006 wait complete. 2020-07-18T03:34:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2020-07-18T03:34:23.369 INFO:tasks.ceph.mds.a:Restarting daemon 2020-07-18T03:34:23.370 INFO:tasks.ceph.mds.a:Stopping old one... 2020-07-18T03:34:23.371 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:34:23.371 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:34:23.371 INFO:tasks.ceph.mds.b:Restarting daemon 2020-07-18T03:34:23.372 INFO:tasks.ceph.mds.b:Stopping old one... 2020-07-18T03:34:23.373 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:34:23.373 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:34:23.381 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:34:23.381 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-07-18T03:34:23.383 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:34:23.384 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-07-18T03:34:23.424 INFO:tasks.ceph.mds.a:Started 2020-07-18T03:34:23.425 DEBUG:teuthology.parallel:result is None 2020-07-18T03:34:23.429 INFO:tasks.ceph.mds.b:Started 2020-07-18T03:34:23.429 DEBUG:teuthology.parallel:result is None 2020-07-18T03:34:23.430 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-07-18T03:34:23.510 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2020-07-18T03:34:23.520 INFO:tasks.ceph.mds.b.smithi018.stdout:starting mds.b at 2020-07-18T03:34:23.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.554+0000 7fea4d246700 1 Processor -- start 2020-07-18T03:34:23.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.554+0000 7fea4d246700 1 -- start start 2020-07-18T03:34:23.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea4d246700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea4814f720 0x7fea480ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:23.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea4d246700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480b3a60 con 0x7fea4814b280 2020-07-18T03:34:23.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea4d246700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480b3d00 con 0x7fea48061c60 2020-07-18T03:34:23.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea4d246700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480b3fa0 con 0x7fea4814f720 2020-07-18T03:34:23.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea4659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fea48061c60 0x7fea4814ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45060/0 (socket says 172.21.15.47:45060) 2020-07-18T03:34:23.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea4659c700 1 -- 172.21.15.47:0/4195702442 learned_addr learned my addr 172.21.15.47:0/4195702442 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:23.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea46d9d700 1 --2- 172.21.15.47:0/4195702442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea4814f720 0x7fea480ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:23.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708968161 0 0) 0x7fea480b3d00 con 0x7fea48061c60 2020-07-18T03:34:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.555+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea34003240 con 0x7fea48061c60 2020-07-18T03:34:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330422245 0 0) 0x7fea480b3a60 con 0x7fea4814b280 2020-07-18T03:34:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea480b3d00 con 0x7fea4814b280 2020-07-18T03:34:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea46d9d700 1 -- 172.21.15.47:0/4195702442 >> v1:172.21.15.47:6789/0 conn(0x7fea48061c60 legacy=0x7fea4814ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea46d9d700 1 -- 172.21.15.47:0/4195702442 >> v1:172.21.15.18:6789/0 conn(0x7fea4814b280 legacy=0x7fea480b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea46d9d700 1 -- 172.21.15.47:0/4195702442 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480ae690 con 0x7fea4814f720 2020-07-18T03:34:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea46d9d700 1 --2- 172.21.15.47:0/4195702442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea4814f720 0x7fea480ae1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fea3c002f30 tx=0x7fea3c006dd0).ready entity=mon.2 client_cookie=26dbf71fcb0b38c3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea3c0096e0 con 0x7fea4814f720 2020-07-18T03:34:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c009840 con 0x7fea4814f720 2020-07-18T03:34:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea45d9b700 1 -- 172.21.15.47:0/4195702442 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea3c009ba0 con 0x7fea4814f720 2020-07-18T03:34:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 -- 172.21.15.47:0/4195702442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea4814f720 msgr2=0x7fea480ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 --2- 172.21.15.47:0/4195702442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea4814f720 0x7fea480ae1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fea3c002f30 tx=0x7fea3c006dd0).stop 2020-07-18T03:34:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 -- 172.21.15.47:0/4195702442 shutdown_connections 2020-07-18T03:34:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 --2- 172.21.15.47:0/4195702442 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea4814f720 0x7fea480ae1a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 -- 172.21.15.47:0/4195702442 >> 172.21.15.47:0/4195702442 conn(0x7fea480b93c0 msgr2=0x7fea480b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 -- 172.21.15.47:0/4195702442 shutdown_connections 2020-07-18T03:34:23.563 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.556+0000 7fea4d246700 1 -- 172.21.15.47:0/4195702442 wait complete. 2020-07-18T03:34:23.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 Processor -- start 2020-07-18T03:34:23.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 -- start start 2020-07-18T03:34:23.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea48061c60 0x7fea48149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:23.567 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fea4814dd60 0x7fea48159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea48147fb0 con 0x7fea4814b280 2020-07-18T03:34:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea481479f0 con 0x7fea4814dd60 2020-07-18T03:34:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.558+0000 7fea4d246700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea481481e0 con 0x7fea48061c60 2020-07-18T03:34:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea46d9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea48061c60 0x7fea48149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4759e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fea4814dd60 0x7fea48159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea46d9d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea48061c60 0x7fea48149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56574/0 (socket says 172.21.15.47:56574) 2020-07-18T03:34:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea46d9d700 1 -- 172.21.15.47:0/451485176 learned_addr learned my addr 172.21.15.47:0/451485176 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4759e700 1 -- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea48061c60 msgr2=0x7fea48149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4759e700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea48061c60 0x7fea48149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963365382 0 0) 0x7fea48147fb0 con 0x7fea4814b280 2020-07-18T03:34:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4759e700 1 -- 172.21.15.47:0/451485176 >> v1:172.21.15.18:6789/0 conn(0x7fea4814b280 legacy=0x7fea48062eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4759e700 1 -- 172.21.15.47:0/451485176 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48147670 con 0x7fea4814dd60 2020-07-18T03:34:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4759e700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea4814dd60 0x7fea48159010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fea40001450 tx=0x7fea40007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4000a5a0 con 0x7fea4814dd60 2020-07-18T03:34:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4000a700 con 0x7fea4814dd60 2020-07-18T03:34:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4000a870 con 0x7fea4814dd60 2020-07-18T03:34:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4814b7f0 con 0x7fea4814dd60 2020-07-18T03:34:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.559+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea48159740 con 0x7fea4814dd60 2020-07-18T03:34:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.560+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48147860 con 0x7fea4814dd60 2020-07-18T03:34:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.560+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fea40010070 con 0x7fea4814dd60 2020-07-18T03:34:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.561+0000 7fea2effd700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea2404fea0 0x7fea24052360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.561+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5815+0+0 (secure 0 0 0) 0x7fea400674c0 con 0x7fea4814dd60 2020-07-18T03:34:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.561+0000 7fea46d9d700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea2404fea0 0x7fea24052360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.561+0000 7fea46d9d700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea2404fea0 0x7fea24052360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fea3c019000 tx=0x7fea3c016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.565+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fea4003feb0 con 0x7fea4814dd60 2020-07-18T03:34:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:23.739+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fea48147b30 con 0x7fea4814dd60 2020-07-18T03:34:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.532+0000 7fea2effd700 1 -- 172.21.15.47:0/451485176 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7fea400434d0 con 0x7fea4814dd60 2020-07-18T03:34:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.534+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea2404fea0 msgr2=0x7fea24052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.534+0000 7fea4d246700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea2404fea0 0x7fea24052360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fea3c019000 tx=0x7fea3c016000).stop 2020-07-18T03:34:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.534+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea4814dd60 msgr2=0x7fea48159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.534+0000 7fea4d246700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea4814dd60 0x7fea48159010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fea40001450 tx=0x7fea40007c90).stop 2020-07-18T03:34:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.534+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 shutdown_connections 2020-07-18T03:34:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.534+0000 7fea4d246700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fea2404fea0 0x7fea24052360 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.535+0000 7fea4d246700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea4814dd60 0x7fea48159010 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.535+0000 7fea4d246700 1 --2- 172.21.15.47:0/451485176 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fea48061c60 0x7fea48149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.535+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 >> 172.21.15.47:0/451485176 conn(0x7fea480b93c0 msgr2=0x7fea4814ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.535+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 shutdown_connections 2020-07-18T03:34:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.535+0000 7fea4d246700 1 -- 172.21.15.47:0/451485176 wait complete. 2020-07-18T03:34:24.538 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2020-07-18T03:34:24.548 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2020-07-18T03:34:24.549 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-07-18T03:34:24.672 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.671+0000 7fec88c43700 1 Processor -- start 2020-07-18T03:34:24.673 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.671+0000 7fec88c43700 1 -- start start 2020-07-18T03:34:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec88c43700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 0x7fec8405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec88c43700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 0x7fec840ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec88c43700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840b3ac0 con 0x7fec840b4620 2020-07-18T03:34:24.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec88c43700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840b5a10 con 0x7fec840b5370 2020-07-18T03:34:24.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec88c43700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec840b0040 con 0x7fec840b41c0 2020-07-18T03:34:24.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec81d9b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 0x7fec840ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:24.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.672+0000 7fec8259c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 0x7fec8405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:24.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 0x7fec840ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56586/0 (socket says 172.21.15.47:56586) 2020-07-18T03:34:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec8259c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 0x7fec8405fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:49666/0 (socket says 172.21.15.47:49666) 2020-07-18T03:34:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 -- 172.21.15.47:0/682026132 learned_addr learned my addr 172.21.15.47:0/682026132 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 -- 172.21.15.47:0/682026132 >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 msgr2=0x7fec8405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 --2- 172.21.15.47:0/682026132 >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 0x7fec8405fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 -- 172.21.15.47:0/682026132 >> v1:172.21.15.18:6789/0 conn(0x7fec840b4620 legacy=0x7fec841428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 -- 172.21.15.47:0/682026132 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840b3d20 con 0x7fec840b41c0 2020-07-18T03:34:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec8259c700 1 --2- 172.21.15.47:0/682026132 >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 0x7fec8405fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec81d9b700 1 --2- 172.21.15.47:0/682026132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 0x7fec840ad2d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fec74002f30 tx=0x7fec74006e90).ready entity=mon.2 client_cookie=4a9fdbb2dfa36de3 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec8159a700 1 -- 172.21.15.47:0/682026132 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec74009850 con 0x7fec840b41c0 2020-07-18T03:34:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec8159a700 1 -- 172.21.15.47:0/682026132 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec740099b0 con 0x7fec840b41c0 2020-07-18T03:34:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec8159a700 1 -- 172.21.15.47:0/682026132 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec74009d10 con 0x7fec840b41c0 2020-07-18T03:34:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 -- 172.21.15.47:0/682026132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 msgr2=0x7fec840ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 --2- 172.21.15.47:0/682026132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 0x7fec840ad2d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fec74002f30 tx=0x7fec74006e90).stop 2020-07-18T03:34:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 -- 172.21.15.47:0/682026132 shutdown_connections 2020-07-18T03:34:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 --2- 172.21.15.47:0/682026132 >> v2:172.21.15.18:3300/0 conn(0x7fec840b5370 0x7fec8405fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 --2- 172.21.15.47:0/682026132 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fec840b41c0 0x7fec840ad2d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 -- 172.21.15.47:0/682026132 >> 172.21.15.47:0/682026132 conn(0x7fec840b9340 msgr2=0x7fec840b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:24.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.673+0000 7fec88c43700 1 -- 172.21.15.47:0/682026132 shutdown_connections 2020-07-18T03:34:24.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.674+0000 7fec88c43700 1 -- 172.21.15.47:0/682026132 wait complete. 2020-07-18T03:34:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.675+0000 7fec88c43700 1 Processor -- start 2020-07-18T03:34:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.675+0000 7fec88c43700 1 -- start start 2020-07-18T03:34:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec88c43700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fec8413bc20 0x7fec84159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:24.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec88c43700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8413c4c0 con 0x7fec840b5370 2020-07-18T03:34:24.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec88c43700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec84158cc0 con 0x7fec841598f0 2020-07-18T03:34:24.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec88c43700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec84158f60 con 0x7fec8413bc20 2020-07-18T03:34:24.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec82d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fec841598f0 0x7fec84137c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45084/0 (socket says 172.21.15.47:45084) 2020-07-18T03:34:24.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec82d9d700 1 -- 172.21.15.47:0/842722141 learned_addr learned my addr 172.21.15.47:0/842722141 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec81d9b700 1 --2- 172.21.15.47:0/842722141 >> v2:172.21.15.18:3300/0 conn(0x7fec8413bc20 0x7fec84159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475137877 0 0) 0x7fec84158cc0 con 0x7fec841598f0 2020-07-18T03:34:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec58003240 con 0x7fec841598f0 2020-07-18T03:34:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249786044 0 0) 0x7fec8413c4c0 con 0x7fec840b5370 2020-07-18T03:34:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec81d9b700 1 -- 172.21.15.47:0/842722141 >> v1:172.21.15.47:6789/0 conn(0x7fec841598f0 legacy=0x7fec84137c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec81d9b700 1 -- 172.21.15.47:0/842722141 >> v1:172.21.15.18:6789/0 conn(0x7fec840b5370 legacy=0x7fec84065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec81d9b700 1 -- 172.21.15.47:0/842722141 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84138350 con 0x7fec8413bc20 2020-07-18T03:34:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec81d9b700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec8413bc20 0x7fec84159400 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fec74002f30 tx=0x7fec74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec74009850 con 0x7fec8413bc20 2020-07-18T03:34:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec74011e80 con 0x7fec8413bc20 2020-07-18T03:34:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.676+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8414f5a0 con 0x7fec8413bc20 2020-07-18T03:34:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.677+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec7401d7c0 con 0x7fec8413bc20 2020-07-18T03:34:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.677+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8413c250 con 0x7fec8413bc20 2020-07-18T03:34:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.677+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec84060540 con 0x7fec8413bc20 2020-07-18T03:34:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.678+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fec74025030 con 0x7fec8413bc20 2020-07-18T03:34:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.678+0000 7fec727fc700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec580502c0 0x7fec58052780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.678+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fec74068d50 con 0x7fec8413bc20 2020-07-18T03:34:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.678+0000 7fec8259c700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec580502c0 0x7fec58052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.678+0000 7fec8259c700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec580502c0 0x7fec58052780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fec78002f30 tx=0x7fec78006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.682+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fec740417a0 con 0x7fec8413bc20 2020-07-18T03:34:24.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.896+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fec8414f730 con 0x7fec8413bc20 2020-07-18T03:34:24.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.897+0000 7fec727fc700 1 -- 172.21.15.47:0/842722141 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+4513 (secure 0 0 0) 0x7fec74023020 con 0x7fec8413bc20 2020-07-18T03:34:24.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec580502c0 msgr2=0x7fec58052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec580502c0 0x7fec58052780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fec78002f30 tx=0x7fec78006dd0).stop 2020-07-18T03:34:24.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec8413bc20 msgr2=0x7fec84159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:24.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec8413bc20 0x7fec84159400 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fec74002f30 tx=0x7fec74015000).stop 2020-07-18T03:34:24.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 shutdown_connections 2020-07-18T03:34:24.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fec580502c0 0x7fec58052780 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 --2- 172.21.15.47:0/842722141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec8413bc20 0x7fec84159400 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:24.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 >> 172.21.15.47:0/842722141 conn(0x7fec840b9340 msgr2=0x7fec840ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:24.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 shutdown_connections 2020-07-18T03:34:24.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:24.898+0000 7fec88c43700 1 -- 172.21.15.47:0/842722141 wait complete. 2020-07-18T03:34:24.911 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:24.912 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:34:24.912 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2020-07-18T03:34:24.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.913 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2020-07-18T03:34:24.913 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfJWBHORAAOi+GptCwVrcf99HN+SUw7g==", 2020-07-18T03:34:24.913 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:34:24.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.915 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.916 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2020-07-18T03:34:24.917 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfEwJmAxAALHEGDSLeTC3zHdPXQk2uDQ==", 2020-07-18T03:34:24.917 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:34:24.918 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.918 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.919 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.919 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.920 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2020-07-18T03:34:24.921 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCEaxJfbccZIhAAA0laGXXGNHEG6i8lhBnjoA==", 2020-07-18T03:34:24.921 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.921 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.921 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.921 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.922 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.922 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2020-07-18T03:34:24.922 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCHaxJfkPKlAhAAskPa62bFux7rJRJQBMx0RQ==", 2020-07-18T03:34:24.922 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.923 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.923 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.923 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.924 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2020-07-18T03:34:24.924 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCJaxJfPNocHxAA+BaZ/Ma3ASG85IQHhTjpWA==", 2020-07-18T03:34:24.924 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.924 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.925 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.925 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.925 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.926 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.926 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2020-07-18T03:34:24.926 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCMaxJftFt2ABAAGWP7ohp4lSD1BHMpqYaqNQ==", 2020-07-18T03:34:24.926 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.927 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.927 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.927 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.927 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.928 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.928 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2020-07-18T03:34:24.928 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCPaxJfC4/CIRAA7WGx9xFmVBS1VX+B4Tc3QQ==", 2020-07-18T03:34:24.929 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.929 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.929 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.929 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.931 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.932 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2020-07-18T03:34:24.932 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCSaxJf1QrKAhAAhQdyfol+8e5yNrcj6b+NHA==", 2020-07-18T03:34:24.932 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.932 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.933 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.933 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.934 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2020-07-18T03:34:24.934 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCUaxJfssjbHhAArXw2DFBdIMgNXv10DLjCGg==", 2020-07-18T03:34:24.935 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.935 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.935 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.936 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.937 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2020-07-18T03:34:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCWaxJf3n0DOxAAy1tMI1rDFIv2/KUVxaa3rg==", 2020-07-18T03:34:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.939 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.939 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.941 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2020-07-18T03:34:24.941 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJffNYiCBAA0LAcpPo3BjyH46ibnAmhuw==", 2020-07-18T03:34:24.941 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:34:24.942 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:34:24.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:34:24.942 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.943 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2020-07-18T03:34:24.944 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfNSD6CxAA0WfQ0mzXpa8Q3BQAOOnb+g==", 2020-07-18T03:34:24.944 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:34:24.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:34:24.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:34:24.945 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.945 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2020-07-18T03:34:24.946 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfL3zUDxAAwjIhteo69aAMKiC63jgS0w==", 2020-07-18T03:34:24.946 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.947 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:34:24.947 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:34:24.947 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:34:24.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.948 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2020-07-18T03:34:24.948 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCDaxJfy7mQExAARVMrxtFnhCA5EItp7UZnog==", 2020-07-18T03:34:24.948 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2020-07-18T03:34:24.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2020-07-18T03:34:24.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2020-07-18T03:34:24.949 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.950 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2020-07-18T03:34:24.951 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJfi9GyDxAACUH4y/92DOhKqZyh9ejEnA==", 2020-07-18T03:34:24.951 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:34:24.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2020-07-18T03:34:24.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2020-07-18T03:34:24.952 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.953 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2020-07-18T03:34:24.953 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJf8GxYNRAA4j6QmmTZ1YbcSBmnsdUtyQ==", 2020-07-18T03:34:24.954 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.954 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:34:24.954 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:34:24.954 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:24.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:24.955 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2020-07-18T03:34:24.955 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQCCaxJftjguMBAAinfWzDwZhEmLhlF0nQV9rw==", 2020-07-18T03:34:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2020-07-18T03:34:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2020-07-18T03:34:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2020-07-18T03:34:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2020-07-18T03:34:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2020-07-18T03:34:24.957 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:34:24.959 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:25.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.072+0000 7fbde009f700 1 Processor -- start 2020-07-18T03:34:25.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.073+0000 7fbde009f700 1 -- start start 2020-07-18T03:34:25.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.073+0000 7fbde009f700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbdd80ad100 0x7fbdd80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.073+0000 7fbde009f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd81443f0 0x7fbdd8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.073+0000 7fbde009f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd8147330 con 0x7fbdd8147a00 2020-07-18T03:34:25.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.073+0000 7fbde009f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80ae530 con 0x7fbdd81443f0 2020-07-18T03:34:25.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.073+0000 7fbde009f700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b57e0 con 0x7fbdd80ad100 2020-07-18T03:34:25.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbdde63c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd81443f0 0x7fbdd8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddd63a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbdd8147a00 0x7fbdd805f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45088/0 (socket says 172.21.15.47:45088) 2020-07-18T03:34:25.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddd63a700 1 -- 172.21.15.47:0/3480777929 learned_addr learned my addr 172.21.15.47:0/3480777929 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:25.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddde3b700 1 --2- 172.21.15.47:0/3480777929 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbdd80ad100 0x7fbdd80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbdde63c700 1 -- 172.21.15.47:0/3480777929 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbdd80ad100 msgr2=0x7fbdd80b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbdde63c700 1 --2- 172.21.15.47:0/3480777929 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbdd80ad100 0x7fbdd80b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbdde63c700 1 -- 172.21.15.47:0/3480777929 >> v1:172.21.15.47:6789/0 conn(0x7fbdd8147a00 legacy=0x7fbdd805f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbdde63c700 1 -- 172.21.15.47:0/3480777929 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80b5a80 con 0x7fbdd81443f0 2020-07-18T03:34:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddce39700 1 -- 172.21.15.47:0/3480777929 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234226043 0 0) 0x7fbdd8147330 con 0x7fbdd8147a00 2020-07-18T03:34:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbdde63c700 1 --2- 172.21.15.47:0/3480777929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd81443f0 0x7fbdd8147e40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fbdd00032b0 tx=0x7fbdd00091d0).ready entity=mon.1 client_cookie=e476c19a4f6980ff server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddce39700 1 -- 172.21.15.47:0/3480777929 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd000b9a0 con 0x7fbdd81443f0 2020-07-18T03:34:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddce39700 1 -- 172.21.15.47:0/3480777929 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd000bb00 con 0x7fbdd81443f0 2020-07-18T03:34:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbddce39700 1 -- 172.21.15.47:0/3480777929 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd000bc70 con 0x7fbdd81443f0 2020-07-18T03:34:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 -- 172.21.15.47:0/3480777929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd81443f0 msgr2=0x7fbdd8147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 --2- 172.21.15.47:0/3480777929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd81443f0 0x7fbdd8147e40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fbdd00032b0 tx=0x7fbdd00091d0).stop 2020-07-18T03:34:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 -- 172.21.15.47:0/3480777929 shutdown_connections 2020-07-18T03:34:25.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 --2- 172.21.15.47:0/3480777929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd81443f0 0x7fbdd8147e40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 --2- 172.21.15.47:0/3480777929 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fbdd80ad100 0x7fbdd80b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 -- 172.21.15.47:0/3480777929 >> 172.21.15.47:0/3480777929 conn(0x7fbdd80b9340 msgr2=0x7fbdd80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 -- 172.21.15.47:0/3480777929 shutdown_connections 2020-07-18T03:34:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.074+0000 7fbde009f700 1 -- 172.21.15.47:0/3480777929 wait complete. 2020-07-18T03:34:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.076+0000 7fbde009f700 1 Processor -- start 2020-07-18T03:34:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.076+0000 7fbde009f700 1 -- start start 2020-07-18T03:34:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.076+0000 7fbde009f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd80ad100 0x7fbdd8142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbde009f700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fbdd813d3a0 0x7fbdd813d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbde009f700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd8159720 con 0x7fbdd8147a00 2020-07-18T03:34:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbde009f700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd8142610 con 0x7fbdd813d3a0 2020-07-18T03:34:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbde009f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd81591e0 con 0x7fbdd80ad100 2020-07-18T03:34:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd80ad100 0x7fbdd8142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd80ad100 0x7fbdd8142120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35774/0 (socket says 172.21.15.47:35774) 2020-07-18T03:34:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 -- 172.21.15.47:0/423753633 learned_addr learned my addr 172.21.15.47:0/423753633 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbdde63c700 1 --2- 172.21.15.47:0/423753633 >> v2:172.21.15.18:3300/0 conn(0x7fbdd813d3a0 0x7fbdd813d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 -- 172.21.15.47:0/423753633 >> v2:172.21.15.18:3300/0 conn(0x7fbdd813d3a0 msgr2=0x7fbdd813d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 --2- 172.21.15.47:0/423753633 >> v2:172.21.15.18:3300/0 conn(0x7fbdd813d3a0 0x7fbdd813d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 -- 172.21.15.47:0/423753633 >> v1:172.21.15.18:6789/0 conn(0x7fbdd8147a00 legacy=0x7fbdd80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 -- 172.21.15.47:0/423753633 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8159480 con 0x7fbdd80ad100 2020-07-18T03:34:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbdde63c700 1 --2- 172.21.15.47:0/423753633 >> v2:172.21.15.18:3300/0 conn(0x7fbdd813d3a0 0x7fbdd813d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbddde3b700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd80ad100 0x7fbdd8142120 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7fbdcc00c000 tx=0x7fbdcc006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdcc00cae0 con 0x7fbdd80ad100 2020-07-18T03:34:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc00cc40 con 0x7fbdd80ad100 2020-07-18T03:34:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd8155380 con 0x7fbdd80ad100 2020-07-18T03:34:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdcc0123f0 con 0x7fbdd80ad100 2020-07-18T03:34:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.077+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd8139590 con 0x7fbdd80ad100 2020-07-18T03:34:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.078+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd81551c0 con 0x7fbdd80ad100 2020-07-18T03:34:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.078+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fbdcc012550 con 0x7fbdd80ad100 2020-07-18T03:34:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.079+0000 7fbdc9ffb700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdb0050010 0x7fbdb00524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.079+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fbdcc067f90 con 0x7fbdd80ad100 2020-07-18T03:34:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.079+0000 7fbddd63a700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdb0050010 0x7fbdb00524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.079+0000 7fbddd63a700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdb0050010 0x7fbdb00524d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fbdd80b8550 tx=0x7fbdd4007720).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.082+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fbdcc040a00 con 0x7fbdd80ad100 2020-07-18T03:34:25.246 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.245+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdd8057450 con 0x7fbdd80ad100 2020-07-18T03:34:25.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.246+0000 7fbdc9ffb700 1 -- 172.21.15.47:0/423753633 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13418 (secure 0 0 0) 0x7fbdcc044020 con 0x7fbdd80ad100 2020-07-18T03:34:25.247 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:25.247 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":75,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:24.527812+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:25.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdb0050010 msgr2=0x7fbdb00524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdb0050010 0x7fbdb00524d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fbdd80b8550 tx=0x7fbdd4007720).stop 2020-07-18T03:34:25.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd80ad100 msgr2=0x7fbdd8142120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd80ad100 0x7fbdd8142120 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7fbdcc00c000 tx=0x7fbdcc006dd0).stop 2020-07-18T03:34:25.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 shutdown_connections 2020-07-18T03:34:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fbdb0050010 0x7fbdb00524d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 --2- 172.21.15.47:0/423753633 >> v2:172.21.15.18:3300/0 conn(0x7fbdd813d3a0 0x7fbdd813d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 --2- 172.21.15.47:0/423753633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd80ad100 0x7fbdd8142120 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 >> 172.21.15.47:0/423753633 conn(0x7fbdd80b9340 msgr2=0x7fbdd8060100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 shutdown_connections 2020-07-18T03:34:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.247+0000 7fbde009f700 1 -- 172.21.15.47:0/423753633 wait complete. 2020-07-18T03:34:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-07-18T03:34:25.259 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:34:25.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.376+0000 7f3def994700 1 Processor -- start 2020-07-18T03:34:25.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.376+0000 7f3def994700 1 -- start start 2020-07-18T03:34:25.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3de8142a20 0x7f3de8142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3de80b3af0 0x7f3de80b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de80af4b0 con 0x7f3de8061c60 2020-07-18T03:34:25.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de80b6d80 con 0x7f3de8142a20 2020-07-18T03:34:25.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de80af710 con 0x7f3de80b3af0 2020-07-18T03:34:25.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3dedf31700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3de80b3af0 0x7f3de80b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3decf2f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3de8061c60 0x7f3de80620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45102/0 (socket says 172.21.15.47:45102) 2020-07-18T03:34:25.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3decf2f700 1 -- 172.21.15.47:0/221275622 learned_addr learned my addr 172.21.15.47:0/221275622 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:25.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3ded730700 1 --2- 172.21.15.47:0/221275622 >> v2:172.21.15.18:3300/0 conn(0x7f3de8142a20 0x7f3de8142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3dedf31700 1 -- 172.21.15.47:0/221275622 >> v2:172.21.15.18:3300/0 conn(0x7f3de8142a20 msgr2=0x7f3de8142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:25.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3dedf31700 1 --2- 172.21.15.47:0/221275622 >> v2:172.21.15.18:3300/0 conn(0x7f3de8142a20 0x7f3de8142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3dedf31700 1 -- 172.21.15.47:0/221275622 >> v1:172.21.15.47:6789/0 conn(0x7f3de8061c60 legacy=0x7f3de80620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3de7fff700 1 -- 172.21.15.47:0/221275622 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936347282 0 0) 0x7f3de80af4b0 con 0x7f3de8061c60 2020-07-18T03:34:25.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3dedf31700 1 -- 172.21.15.47:0/221275622 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de80af9b0 con 0x7f3de80b3af0 2020-07-18T03:34:25.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3dedf31700 1 --2- 172.21.15.47:0/221275622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de80b3af0 0x7f3de80b3f10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f3dd8002f30 tx=0x7f3dd8006dd0).ready entity=mon.1 client_cookie=b058cb5de9bea540 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3de7fff700 1 -- 172.21.15.47:0/221275622 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd8014070 con 0x7f3de80b3af0 2020-07-18T03:34:25.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3de7fff700 1 -- 172.21.15.47:0/221275622 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd80099e0 con 0x7f3de80b3af0 2020-07-18T03:34:25.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3de7fff700 1 -- 172.21.15.47:0/221275622 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd8009d40 con 0x7f3de80b3af0 2020-07-18T03:34:25.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 -- 172.21.15.47:0/221275622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de80b3af0 msgr2=0x7f3de80b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 --2- 172.21.15.47:0/221275622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de80b3af0 0x7f3de80b3f10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f3dd8002f30 tx=0x7f3dd8006dd0).stop 2020-07-18T03:34:25.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 -- 172.21.15.47:0/221275622 shutdown_connections 2020-07-18T03:34:25.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 --2- 172.21.15.47:0/221275622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de80b3af0 0x7f3de80b3f10 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 --2- 172.21.15.47:0/221275622 >> v2:172.21.15.18:3300/0 conn(0x7f3de8142a20 0x7f3de8142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.377+0000 7f3def994700 1 -- 172.21.15.47:0/221275622 >> 172.21.15.47:0/221275622 conn(0x7f3de805f5a0 msgr2=0x7f3de805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.378+0000 7f3def994700 1 -- 172.21.15.47:0/221275622 shutdown_connections 2020-07-18T03:34:25.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.378+0000 7f3def994700 1 -- 172.21.15.47:0/221275622 wait complete. 2020-07-18T03:34:25.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.379+0000 7f3def994700 1 Processor -- start 2020-07-18T03:34:25.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.379+0000 7f3def994700 1 -- start start 2020-07-18T03:34:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3def994700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3de8061c60 0x7f3de8154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3def994700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 0x7f3de813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3def994700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de8154f70 con 0x7f3de8142a20 2020-07-18T03:34:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3def994700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de8066810 con 0x7f3de8061c60 2020-07-18T03:34:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3def994700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8066ab0 con 0x7f3de8065040 2020-07-18T03:34:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3dedf31700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 0x7f3de813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3dedf31700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 0x7f3de813ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56622/0 (socket says 172.21.15.47:56622) 2020-07-18T03:34:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3dedf31700 1 -- 172.21.15.47:0/309249978 learned_addr learned my addr 172.21.15.47:0/309249978 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3ded730700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3de8061c60 0x7f3de8154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3ded730700 1 -- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 msgr2=0x7f3de813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3ded730700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 0x7f3de813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3ded730700 1 -- 172.21.15.47:0/309249978 >> v1:172.21.15.18:6789/0 conn(0x7f3de8142a20 legacy=0x7f3de80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.380+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611406241 0 0) 0x7f3de8154f70 con 0x7f3de8142a20 2020-07-18T03:34:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3ded730700 1 -- 172.21.15.47:0/309249978 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de80643e0 con 0x7f3de8061c60 2020-07-18T03:34:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3dedf31700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 0x7f3de813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3ded730700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3de8061c60 0x7f3de8154a80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f3de000a000 tx=0x7f3de0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de0009a30 con 0x7f3de8061c60 2020-07-18T03:34:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de8150b60 con 0x7f3de8061c60 2020-07-18T03:34:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de0009b90 con 0x7f3de8061c60 2020-07-18T03:34:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de0013480 con 0x7f3de8061c60 2020-07-18T03:34:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.381+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de813d3a0 con 0x7f3de8061c60 2020-07-18T03:34:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.382+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de813d610 con 0x7f3de8061c60 2020-07-18T03:34:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.382+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3de0013680 con 0x7f3de8061c60 2020-07-18T03:34:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.382+0000 7f3de57fa700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3dc004fdb0 0x7f3dc0052270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.382+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3de0067e30 con 0x7f3de8061c60 2020-07-18T03:34:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.382+0000 7f3decf2f700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3dc004fdb0 0x7f3dc0052270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.383+0000 7f3decf2f700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3dc004fdb0 0x7f3dc0052270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f3ddc000ee0 tx=0x7f3ddc007830).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.386+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3de0040820 con 0x7f3de8061c60 2020-07-18T03:34:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.545+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3de805b120 con 0x7f3de8061c60 2020-07-18T03:34:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.546+0000 7f3de57fa700 1 -- 172.21.15.47:0/309249978 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v75) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3de0043e40 con 0x7f3de8061c60 2020-07-18T03:34:25.547 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:34:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3dc004fdb0 msgr2=0x7f3dc0052270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3dc004fdb0 0x7f3dc0052270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f3ddc000ee0 tx=0x7f3ddc007830).stop 2020-07-18T03:34:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3de8061c60 msgr2=0x7f3de8154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3de8061c60 0x7f3de8154a80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f3de000a000 tx=0x7f3de0008040).stop 2020-07-18T03:34:25.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 shutdown_connections 2020-07-18T03:34:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3dc004fdb0 0x7f3dc0052270 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3de8065040 0x7f3de813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 --2- 172.21.15.47:0/309249978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3de8061c60 0x7f3de8154a80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 >> 172.21.15.47:0/309249978 conn(0x7f3de805f5a0 msgr2=0x7f3de80b9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 shutdown_connections 2020-07-18T03:34:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.547+0000 7f3def994700 1 -- 172.21.15.47:0/309249978 wait complete. 2020-07-18T03:34:25.559 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:34:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.675+0000 7efef8594700 1 Processor -- start 2020-07-18T03:34:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.675+0000 7efef8594700 1 -- start start 2020-07-18T03:34:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.676+0000 7efef8594700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 0x7efef0142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.676+0000 7efef8594700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef0061db0 con 0x7efef0144e70 2020-07-18T03:34:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.676+0000 7efef8594700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef0062050 con 0x7efef00b04b0 2020-07-18T03:34:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.676+0000 7efef8594700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0145250 con 0x7efef0141fb0 2020-07-18T03:34:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef6b31700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efef00b04b0 0x7efef00b0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45120/0 (socket says 172.21.15.47:45120) 2020-07-18T03:34:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef6b31700 1 -- 172.21.15.47:0/3870859766 learned_addr learned my addr 172.21.15.47:0/3870859766 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef5b2f700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 0x7efef0142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858354528 0 0) 0x7efef0062050 con 0x7efef00b04b0 2020-07-18T03:34:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed8003240 con 0x7efef00b04b0 2020-07-18T03:34:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 730820227 0 0) 0x7efed8003240 con 0x7efef00b04b0 2020-07-18T03:34:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 msgr2=0x7efef0142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 0x7efef0142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 >> v1:172.21.15.18:6789/0 conn(0x7efef0144e70 legacy=0x7efef00b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00622f0 con 0x7efef00b04b0 2020-07-18T03:34:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7efee8001a50 con 0x7efef00b04b0 2020-07-18T03:34:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 >> v1:172.21.15.47:6789/0 conn(0x7efef00b04b0 legacy=0x7efef00b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed8003760 0x7efed8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 0x7efed80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efed80049d0 0x7efed8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0145250 con 0x7efed80049d0 2020-07-18T03:34:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8004170 con 0x7efed8003760 2020-07-18T03:34:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed8004330 con 0x7efef0141fb0 2020-07-18T03:34:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef6330700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed8003760 0x7efed8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef5b2f700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 0x7efed80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.677+0000 7efef6b31700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efed80049d0 0x7efed8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6330700 1 -- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 msgr2=0x7efed80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6330700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0141fb0 0x7efed80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6330700 1 -- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efed80049d0 msgr2=0x7efed8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6330700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efed80049d0 0x7efed8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6330700 1 -- 172.21.15.47:0/3870859766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8003240 con 0x7efed8003760 2020-07-18T03:34:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6330700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed8003760 0x7efed8003b40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7efee4000fe0 tx=0x7efee4007b50).ready entity=mon.1 client_cookie=446e9177dc677fd5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee400c870 con 0x7efed8003760 2020-07-18T03:34:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400c9d0 con 0x7efed8003760 2020-07-18T03:34:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef532e700 1 -- 172.21.15.47:0/3870859766 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee400cb40 con 0x7efed8003760 2020-07-18T03:34:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef8594700 1 -- 172.21.15.47:0/3870859766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed8003760 msgr2=0x7efed8003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef8594700 1 --2- 172.21.15.47:0/3870859766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed8003760 0x7efed8003b40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7efee4000fe0 tx=0x7efee4007b50).stop 2020-07-18T03:34:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef6b31700 1 -- 172.21.15.47:0/3870859766 reap_dead start 2020-07-18T03:34:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef8594700 1 -- 172.21.15.47:0/3870859766 shutdown_connections 2020-07-18T03:34:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef8594700 1 -- 172.21.15.47:0/3870859766 >> 172.21.15.47:0/3870859766 conn(0x7efef005f5a0 msgr2=0x7efef005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef8594700 1 -- 172.21.15.47:0/3870859766 shutdown_connections 2020-07-18T03:34:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.678+0000 7efef8594700 1 -- 172.21.15.47:0/3870859766 wait complete. 2020-07-18T03:34:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.679+0000 7efef8594700 1 Processor -- start 2020-07-18T03:34:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 -- start start 2020-07-18T03:34:25.691 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efef01480a0 0x7efef0065fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0148480 0x7efef012db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7efef0066490 0x7efef012e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef0066970 con 0x7efef0066490 2020-07-18T03:34:25.692 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef0066c10 con 0x7efef01480a0 2020-07-18T03:34:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef8594700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0066eb0 con 0x7efef0148480 2020-07-18T03:34:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef6330700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efef01480a0 0x7efef0065fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef6330700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efef01480a0 0x7efef0065fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35808/0 (socket says 172.21.15.47:35808) 2020-07-18T03:34:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef6330700 1 -- 172.21.15.47:0/3468998587 learned_addr learned my addr 172.21.15.47:0/3468998587 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:25.693 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.680+0000 7efef5b2f700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0148480 0x7efef012db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6b31700 1 --2- 172.21.15.47:0/3468998587 >> v2:172.21.15.18:3300/0 conn(0x7efef0066490 0x7efef012e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6330700 1 -- 172.21.15.47:0/3468998587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0148480 msgr2=0x7efef012db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6330700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0148480 0x7efef012db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6330700 1 -- 172.21.15.47:0/3468998587 >> v2:172.21.15.18:3300/0 conn(0x7efef0066490 msgr2=0x7efef012e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.694 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6330700 1 --2- 172.21.15.47:0/3468998587 >> v2:172.21.15.18:3300/0 conn(0x7efef0066490 0x7efef012e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6330700 1 -- 172.21.15.47:0/3468998587 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef0156470 con 0x7efef01480a0 2020-07-18T03:34:25.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef6330700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef01480a0 0x7efef0065fa0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7efee4000fe0 tx=0x7efee4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee400c870 con 0x7efef01480a0 2020-07-18T03:34:25.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4007020 con 0x7efef01480a0 2020-07-18T03:34:25.695 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee4007380 con 0x7efef01480a0 2020-07-18T03:34:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef0063170 con 0x7efef01480a0 2020-07-18T03:34:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.681+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef00635a0 con 0x7efef01480a0 2020-07-18T03:34:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.682+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0067020 con 0x7efef01480a0 2020-07-18T03:34:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.682+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7efee4010030 con 0x7efef01480a0 2020-07-18T03:34:25.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.682+0000 7efee27fc700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efec8050040 0x7efec8052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.682+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5600+0+0 (secure 0 0 0) 0x7efee4069160 con 0x7efef01480a0 2020-07-18T03:34:25.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.683+0000 7efef5b2f700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efec8050040 0x7efec8052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.683+0000 7efef5b2f700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efec8050040 0x7efec8052500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7efeec008000 tx=0x7efeec008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.697 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.686+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7efef0067020 con 0x7efef01480a0 2020-07-18T03:34:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.843+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efef0067020 con 0x7efef01480a0 2020-07-18T03:34:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.843+0000 7efee27fc700 1 -- 172.21.15.47:0/3468998587 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7efef0067020 con 0x7efef01480a0 2020-07-18T03:34:25.845 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efec8050040 msgr2=0x7efec8052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efec8050040 0x7efec8052500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7efeec008000 tx=0x7efeec008040).stop 2020-07-18T03:34:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef01480a0 msgr2=0x7efef0065fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef01480a0 0x7efef0065fa0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7efee4000fe0 tx=0x7efee4016000).stop 2020-07-18T03:34:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 shutdown_connections 2020-07-18T03:34:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7efec8050040 0x7efec8052500 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 --2- 172.21.15.47:0/3468998587 >> v2:172.21.15.18:3300/0 conn(0x7efef0066490 0x7efef012e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef01480a0 0x7efef0065fa0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 --2- 172.21.15.47:0/3468998587 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7efef0148480 0x7efef012db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 >> 172.21.15.47:0/3468998587 conn(0x7efef005f5a0 msgr2=0x7efef00b8720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 shutdown_connections 2020-07-18T03:34:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.845+0000 7efef8594700 1 -- 172.21.15.47:0/3468998587 wait complete. 2020-07-18T03:34:25.856 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-07-18T03:34:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.970+0000 7fa9c332d700 1 Processor -- start 2020-07-18T03:34:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.970+0000 7fa9c332d700 1 -- start start 2020-07-18T03:34:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c332d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9bc0b5370 0x7fa9bc05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c332d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b4620 0x7fa9bc1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c332d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc0b3ac0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c332d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b5a10 con 0x7fa9bc0b4620 2020-07-18T03:34:25.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c332d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b0040 con 0x7fa9bc0b5370 2020-07-18T03:34:25.976 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c08c8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa9bc0b41c0 0x7fa9bc0ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45136/0 (socket says 172.21.15.47:45136) 2020-07-18T03:34:25.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c08c8700 1 -- 172.21.15.47:0/2203041423 learned_addr learned my addr 172.21.15.47:0/2203041423 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:25.977 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c18ca700 1 --2- 172.21.15.47:0/2203041423 >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b4620 0x7fa9bc1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.978 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.971+0000 7fa9c10c9700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9bc0b5370 0x7fa9bc05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9abfff700 1 -- 172.21.15.47:0/2203041423 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800194463 0 0) 0x7fa9bc0b3ac0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9abfff700 1 -- 172.21.15.47:0/2203041423 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a4003240 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9abfff700 1 -- 172.21.15.47:0/2203041423 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3617252097 0 0) 0x7fa9a4003240 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.980 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c18ca700 1 -- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9bc0b5370 msgr2=0x7fa9bc05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c18ca700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9bc0b5370 0x7fa9bc05fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c18ca700 1 -- 172.21.15.47:0/2203041423 >> v1:172.21.15.47:6789/0 conn(0x7fa9bc0b41c0 legacy=0x7fa9bc0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c18ca700 1 -- 172.21.15.47:0/2203041423 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0b3d20 con 0x7fa9bc0b4620 2020-07-18T03:34:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c10c9700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9bc0b5370 0x7fa9bc05fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c18ca700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9bc0b4620 0x7fa9bc1428f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fa9b8001870 tx=0x7fa9b40091d0).ready entity=mon.0 client_cookie=4be1c748783f08e4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9abfff700 1 -- 172.21.15.47:0/2203041423 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b80016d0 con 0x7fa9bc0b4620 2020-07-18T03:34:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9abfff700 1 -- 172.21.15.47:0/2203041423 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400b800 con 0x7fa9bc0b4620 2020-07-18T03:34:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9abfff700 1 -- 172.21.15.47:0/2203041423 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b400b970 con 0x7fa9bc0b4620 2020-07-18T03:34:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 -- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9bc0b4620 msgr2=0x7fa9bc1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9bc0b4620 0x7fa9bc1428f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fa9b8001870 tx=0x7fa9b40091d0).stop 2020-07-18T03:34:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 -- 172.21.15.47:0/2203041423 shutdown_connections 2020-07-18T03:34:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9bc0b4620 0x7fa9bc1428f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 --2- 172.21.15.47:0/2203041423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa9bc0b5370 0x7fa9bc05fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 -- 172.21.15.47:0/2203041423 >> 172.21.15.47:0/2203041423 conn(0x7fa9bc0b9340 msgr2=0x7fa9bc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:25.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 -- 172.21.15.47:0/2203041423 shutdown_connections 2020-07-18T03:34:25.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.972+0000 7fa9c332d700 1 -- 172.21.15.47:0/2203041423 wait complete. 2020-07-18T03:34:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.974+0000 7fa9c332d700 1 Processor -- start 2020-07-18T03:34:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.974+0000 7fa9c332d700 1 -- start start 2020-07-18T03:34:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9bc0b41c0 0x7fa9bc0b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b5370 0x7fa9bc0628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc063d90 con 0x7fa9bc143680 2020-07-18T03:34:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc064030 con 0x7fa9bc0b5370 2020-07-18T03:34:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc144680 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c10c9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9bc0b41c0 0x7fa9bc0b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c18ca700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa9bc143680 0x7fa9bc062dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45146/0 (socket says 172.21.15.47:45146) 2020-07-18T03:34:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c18ca700 1 -- 172.21.15.47:0/1814141707 learned_addr learned my addr 172.21.15.47:0/1814141707 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c08c8700 1 --2- 172.21.15.47:0/1814141707 >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b5370 0x7fa9bc0628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c10c9700 1 -- 172.21.15.47:0/1814141707 >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b5370 msgr2=0x7fa9bc0628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c10c9700 1 --2- 172.21.15.47:0/1814141707 >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b5370 0x7fa9bc0628e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c10c9700 1 -- 172.21.15.47:0/1814141707 >> v1:172.21.15.47:6789/0 conn(0x7fa9bc143680 legacy=0x7fa9bc062dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c10c9700 1 -- 172.21.15.47:0/1814141707 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc1447c0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275134560 0 0) 0x7fa9bc063d90 con 0x7fa9bc143680 2020-07-18T03:34:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c10c9700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0b41c0 0x7fa9bc0b7700 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7fa9ac00c000 tx=0x7fa9ac006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ac00cb40 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac00cca0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc0634e0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ac0123f0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.975+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc0b7e30 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.976+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc0641d0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.976+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa9ac0125f0 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.977+0000 7fa9a97fa700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa99404fdd0 0x7fa994052290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.977+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa9ac068040 con 0x7fa9bc0b41c0 2020-07-18T03:34:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.977+0000 7fa9c08c8700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa99404fdd0 0x7fa994052290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.977+0000 7fa9c08c8700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa99404fdd0 0x7fa994052290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fa9b800d000 tx=0x7fa9b8001470).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:25.980+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa9ac040a30 con 0x7fa9bc0b41c0 2020-07-18T03:34:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.139+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa9bc057450 con 0x7fa9bc0b41c0 2020-07-18T03:34:26.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.139+0000 7fa9a97fa700 1 -- 172.21.15.47:0/1814141707 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa9ac044050 con 0x7fa9bc0b41c0 2020-07-18T03:34:26.141 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:26.142 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2020-07-18T03:34:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2020-07-18T03:34:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2020-07-18T03:34:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2020-07-18T03:34:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2020-07-18T03:34:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2020-07-18T03:34:26.144 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:26.144 INFO:teuthology.orchestra.run.smithi047.stdout:] 2020-07-18T03:34:26.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.140+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa99404fdd0 msgr2=0x7fa994052290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:26.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.140+0000 7fa9c332d700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa99404fdd0 0x7fa994052290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fa9b800d000 tx=0x7fa9b8001470).stop 2020-07-18T03:34:26.147 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.140+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0b41c0 msgr2=0x7fa9bc0b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:26.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.140+0000 7fa9c332d700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0b41c0 0x7fa9bc0b7700 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7fa9ac00c000 tx=0x7fa9ac006dd0).stop 2020-07-18T03:34:26.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.140+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 shutdown_connections 2020-07-18T03:34:26.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.140+0000 7fa9c332d700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa99404fdd0 0x7fa994052290 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.148 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.141+0000 7fa9c332d700 1 --2- 172.21.15.47:0/1814141707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0b41c0 0x7fa9bc0b7700 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.141+0000 7fa9c332d700 1 --2- 172.21.15.47:0/1814141707 >> v2:172.21.15.18:3300/0 conn(0x7fa9bc0b5370 0x7fa9bc0628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.141+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 >> 172.21.15.47:0/1814141707 conn(0x7fa9bc0b9340 msgr2=0x7fa9bc0b8310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:26.149 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.141+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 shutdown_connections 2020-07-18T03:34:26.150 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.141+0000 7fa9c332d700 1 -- 172.21.15.47:0/1814141707 wait complete. 2020-07-18T03:34:26.152 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-07-18T03:34:26.153 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-07-18T03:34:26.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.269+0000 7fd9f35cc700 1 Processor -- start 2020-07-18T03:34:26.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.269+0000 7fd9f35cc700 1 -- start start 2020-07-18T03:34:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f35cc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd9ec061c60 0x7fd9ec0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f35cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9ec0b3af0 0x7fd9ec0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f35cc700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec0af4b0 con 0x7fd9ec142a20 2020-07-18T03:34:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f35cc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b6d80 con 0x7fd9ec0b3af0 2020-07-18T03:34:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f35cc700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0af710 con 0x7fd9ec061c60 2020-07-18T03:34:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f1b69700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9ec0b3af0 0x7fd9ec0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f1b69700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9ec0b3af0 0x7fd9ec0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35830/0 (socket says 172.21.15.47:35830) 2020-07-18T03:34:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f1b69700 1 -- 172.21.15.47:0/2795392336 learned_addr learned my addr 172.21.15.47:0/2795392336 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f0b67700 1 --2- 172.21.15.47:0/2795392336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd9ec061c60 0x7fd9ec0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f1b69700 1 -- 172.21.15.47:0/2795392336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd9ec061c60 msgr2=0x7fd9ec0620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f1b69700 1 --2- 172.21.15.47:0/2795392336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd9ec061c60 0x7fd9ec0620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.270+0000 7fd9f1b69700 1 -- 172.21.15.47:0/2795392336 >> v1:172.21.15.18:6789/0 conn(0x7fd9ec142a20 legacy=0x7fd9ec142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f1b69700 1 -- 172.21.15.47:0/2795392336 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0af9b0 con 0x7fd9ec0b3af0 2020-07-18T03:34:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f1b69700 1 --2- 172.21.15.47:0/2795392336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b3af0 0x7fd9ec0b3f60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7fd9ec0afd30 tx=0x7fd9e4009270).ready entity=mon.1 client_cookie=617aceae05447488 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9dffff700 1 -- 172.21.15.47:0/2795392336 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e400bbc0 con 0x7fd9ec0b3af0 2020-07-18T03:34:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9dffff700 1 -- 172.21.15.47:0/2795392336 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e400bd20 con 0x7fd9ec0b3af0 2020-07-18T03:34:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9dffff700 1 -- 172.21.15.47:0/2795392336 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e4011400 con 0x7fd9ec0b3af0 2020-07-18T03:34:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 -- 172.21.15.47:0/2795392336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b3af0 msgr2=0x7fd9ec0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/2795392336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b3af0 0x7fd9ec0b3f60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7fd9ec0afd30 tx=0x7fd9e4009270).stop 2020-07-18T03:34:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 -- 172.21.15.47:0/2795392336 shutdown_connections 2020-07-18T03:34:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/2795392336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b3af0 0x7fd9ec0b3f60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/2795392336 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd9ec061c60 0x7fd9ec0620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 -- 172.21.15.47:0/2795392336 >> 172.21.15.47:0/2795392336 conn(0x7fd9ec05f5a0 msgr2=0x7fd9ec05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 -- 172.21.15.47:0/2795392336 shutdown_connections 2020-07-18T03:34:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.271+0000 7fd9f35cc700 1 -- 172.21.15.47:0/2795392336 wait complete. 2020-07-18T03:34:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.272+0000 7fd9f35cc700 1 Processor -- start 2020-07-18T03:34:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.272+0000 7fd9f35cc700 1 -- start start 2020-07-18T03:34:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f35cc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd9ec061c60 0x7fd9ec1460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f35cc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9ec0b2e00 0x7fd9ec13adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f35cc700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec13c4d0 con 0x7fd9ec142a20 2020-07-18T03:34:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f35cc700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b16b0 con 0x7fd9ec061c60 2020-07-18T03:34:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f35cc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec144480 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f1b69700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9ec0b2e00 0x7fd9ec13adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f1b69700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9ec0b2e00 0x7fd9ec13adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35836/0 (socket says 172.21.15.47:35836) 2020-07-18T03:34:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.273+0000 7fd9f1b69700 1 -- 172.21.15.47:0/1579621555 learned_addr learned my addr 172.21.15.47:0/1579621555 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f1368700 1 --2- 172.21.15.47:0/1579621555 >> v2:172.21.15.18:3300/0 conn(0x7fd9ec061c60 0x7fd9ec1460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f1b69700 1 -- 172.21.15.47:0/1579621555 >> v2:172.21.15.18:3300/0 conn(0x7fd9ec061c60 msgr2=0x7fd9ec1460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f1b69700 1 --2- 172.21.15.47:0/1579621555 >> v2:172.21.15.18:3300/0 conn(0x7fd9ec061c60 0x7fd9ec1460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f1b69700 1 -- 172.21.15.47:0/1579621555 >> v1:172.21.15.18:6789/0 conn(0x7fd9ec142a20 legacy=0x7fd9ec13a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f1b69700 1 -- 172.21.15.47:0/1579621555 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec13c1c0 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f1b69700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b2e00 0x7fd9ec13adc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fd9ec143920 tx=0x7fd9e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e4010990 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e4010af0 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec13b2b0 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e4010e50 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.274+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec146590 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.275+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec05b120 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.275+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd9e4028070 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.275+0000 7fd9dd7fa700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd9c404ffc0 0x7fd9c4052480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.275+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd9e4068ba0 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.276+0000 7fd9f1368700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd9c404ffc0 0x7fd9c4052480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.276+0000 7fd9f1368700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd9c404ffc0 0x7fd9c4052480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd9e000b000 tx=0x7fd9e0009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.279+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd9e4041670 con 0x7fd9ec0b2e00 2020-07-18T03:34:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:26.441+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd9ec0690d0 con 0x7fd9ec0b2e00 2020-07-18T03:34:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.370+0000 7fd9dd7fa700 1 -- 172.21.15.47:0/1579621555 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v76) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd9e4015030 con 0x7fd9ec0b2e00 2020-07-18T03:34:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd9c404ffc0 msgr2=0x7fd9c4052480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd9c404ffc0 0x7fd9c4052480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd9e000b000 tx=0x7fd9e0009040).stop 2020-07-18T03:34:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b2e00 msgr2=0x7fd9ec13adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b2e00 0x7fd9ec13adc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fd9ec143920 tx=0x7fd9e4016000).stop 2020-07-18T03:34:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 shutdown_connections 2020-07-18T03:34:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd9c404ffc0 0x7fd9c4052480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/1579621555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9ec0b2e00 0x7fd9ec13adc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 --2- 172.21.15.47:0/1579621555 >> v2:172.21.15.18:3300/0 conn(0x7fd9ec061c60 0x7fd9ec1460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 >> 172.21.15.47:0/1579621555 conn(0x7fd9ec05f5a0 msgr2=0x7fd9ec139d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 shutdown_connections 2020-07-18T03:34:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.373+0000 7fd9f35cc700 1 -- 172.21.15.47:0/1579621555 wait complete. 2020-07-18T03:34:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2020-07-18T03:34:27.387 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-07-18T03:34:27.511 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.510+0000 7f6be3ca1700 1 Processor -- start 2020-07-18T03:34:27.512 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 -- start start 2020-07-18T03:34:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc142a20 0x7f6bdc142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f6bdc061c60 0x7f6bdc0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bdc146ac0 0x7f6bdc146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc0af4b0 con 0x7f6bdc061c60 2020-07-18T03:34:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc0b6d80 con 0x7f6bdc146ac0 2020-07-18T03:34:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.511+0000 7f6be3ca1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc0af710 con 0x7f6bdc142a20 2020-07-18T03:34:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bdc146ac0 0x7f6bdc146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bdc146ac0 0x7f6bdc146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35844/0 (socket says 172.21.15.47:35844) 2020-07-18T03:34:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 -- 172.21.15.47:0/1913325828 learned_addr learned my addr 172.21.15.47:0/1913325828 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be1a3d700 1 --2- 172.21.15.47:0/1913325828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc142a20 0x7f6bdc142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be123c700 1 --2- 172.21.15.47:0/1913325828 >> v2:172.21.15.18:3300/0 conn(0x7f6bdc061c60 0x7f6bdc0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:27.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 -- 172.21.15.47:0/1913325828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc142a20 msgr2=0x7f6bdc142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:27.517 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 --2- 172.21.15.47:0/1913325828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc142a20 0x7f6bdc142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 -- 172.21.15.47:0/1913325828 >> v2:172.21.15.18:3300/0 conn(0x7f6bdc061c60 msgr2=0x7f6bdc0620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:27.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 --2- 172.21.15.47:0/1913325828 >> v2:172.21.15.18:3300/0 conn(0x7f6bdc061c60 0x7f6bdc0620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 -- 172.21.15.47:0/1913325828 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc0af9b0 con 0x7f6bdc146ac0 2020-07-18T03:34:27.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be223e700 1 --2- 172.21.15.47:0/1913325828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc146ac0 0x7f6bdc146f30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f6bd40032b0 tx=0x7f6bd4009270).ready entity=mon.1 client_cookie=afc8fa28281d8910 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:27.518 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be0a3b700 1 -- 172.21.15.47:0/1913325828 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bd400bc00 con 0x7f6bdc146ac0 2020-07-18T03:34:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be0a3b700 1 -- 172.21.15.47:0/1913325828 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd400bd60 con 0x7f6bdc146ac0 2020-07-18T03:34:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be0a3b700 1 -- 172.21.15.47:0/1913325828 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bd4011400 con 0x7f6bdc146ac0 2020-07-18T03:34:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 -- 172.21.15.47:0/1913325828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc146ac0 msgr2=0x7f6bdc146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:27.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/1913325828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc146ac0 0x7f6bdc146f30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f6bd40032b0 tx=0x7f6bd4009270).stop 2020-07-18T03:34:27.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 -- 172.21.15.47:0/1913325828 shutdown_connections 2020-07-18T03:34:27.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/1913325828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc146ac0 0x7f6bdc146f30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/1913325828 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc142a20 0x7f6bdc142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/1913325828 >> v2:172.21.15.18:3300/0 conn(0x7f6bdc061c60 0x7f6bdc0620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 -- 172.21.15.47:0/1913325828 >> 172.21.15.47:0/1913325828 conn(0x7f6bdc05f5a0 msgr2=0x7f6bdc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:27.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 -- 172.21.15.47:0/1913325828 shutdown_connections 2020-07-18T03:34:27.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.512+0000 7f6be3ca1700 1 -- 172.21.15.47:0/1913325828 wait complete. 2020-07-18T03:34:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.514+0000 7f6be3ca1700 1 Processor -- start 2020-07-18T03:34:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.514+0000 7f6be3ca1700 1 -- start start 2020-07-18T03:34:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be3ca1700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc061c60 0x7f6bdc1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:27.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be3ca1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bdc1176f0 0x7f6bdc1395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:27.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be3ca1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc116a70 con 0x7f6bdc142a20 2020-07-18T03:34:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be3ca1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc116d10 con 0x7f6bdc1176f0 2020-07-18T03:34:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be3ca1700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc116fb0 con 0x7f6bdc061c60 2020-07-18T03:34:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bdc1176f0 0x7f6bdc1395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bdc1176f0 0x7f6bdc1395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35850/0 (socket says 172.21.15.47:35850) 2020-07-18T03:34:27.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 -- 172.21.15.47:0/469886793 learned_addr learned my addr 172.21.15.47:0/469886793 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:27.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be1a3d700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc061c60 0x7f6bdc1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:27.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 -- 172.21.15.47:0/469886793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc061c60 msgr2=0x7f6bdc1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:27.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc061c60 0x7f6bdc1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:27.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 -- 172.21.15.47:0/469886793 >> v1:172.21.15.18:6789/0 conn(0x7f6bdc142a20 legacy=0x7f6bdc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:27.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 -- 172.21.15.47:0/469886793 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc117250 con 0x7f6bdc1176f0 2020-07-18T03:34:27.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508110989 0 0) 0x7f6bdc116a70 con 0x7f6bdc142a20 2020-07-18T03:34:27.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be223e700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc1176f0 0x7f6bdc1395a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f6bd40032b0 tx=0x7f6bd4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:27.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bd40108c0 con 0x7f6bdc1176f0 2020-07-18T03:34:27.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4010a20 con 0x7f6bdc1176f0 2020-07-18T03:34:27.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bd4010d80 con 0x7f6bdc1176f0 2020-07-18T03:34:27.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.515+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc117b60 con 0x7f6bdc1176f0 2020-07-18T03:34:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.516+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc065f40 con 0x7f6bdc1176f0 2020-07-18T03:34:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.517+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc1173e0 con 0x7f6bdc1176f0 2020-07-18T03:34:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.517+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f6bd400f070 con 0x7f6bdc1176f0 2020-07-18T03:34:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.517+0000 7f6bc9ffb700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bb404fe60 0x7f6bb4052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:27.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.517+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f6bd40690c0 con 0x7f6bdc1176f0 2020-07-18T03:34:27.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.517+0000 7f6be1a3d700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bb404fe60 0x7f6bb4052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:27.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.517+0000 7f6be1a3d700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bb404fe60 0x7f6bb4052320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f6bcc00c000 tx=0x7f6bcc008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:27.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.520+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f6bd4041a50 con 0x7f6bdc1176f0 2020-07-18T03:34:27.682 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:27.681+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f6bdc05fed0 con 0x7f6bdc1176f0 2020-07-18T03:34:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.376+0000 7f6bc9ffb700 1 -- 172.21.15.47:0/469886793 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v77) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6bd4015020 con 0x7f6bdc1176f0 2020-07-18T03:34:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.381+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bb404fe60 msgr2=0x7f6bb4052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.381+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bb404fe60 0x7f6bb4052320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f6bcc00c000 tx=0x7f6bcc008040).stop 2020-07-18T03:34:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.381+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc1176f0 msgr2=0x7f6bdc1395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.381+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc1176f0 0x7f6bdc1395a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f6bd40032b0 tx=0x7f6bd4016000).stop 2020-07-18T03:34:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.382+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 shutdown_connections 2020-07-18T03:34:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.382+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f6bb404fe60 0x7f6bb4052320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.382+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc1176f0 0x7f6bdc1395a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.382+0000 7f6be3ca1700 1 --2- 172.21.15.47:0/469886793 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f6bdc061c60 0x7f6bdc1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.382+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 >> 172.21.15.47:0/469886793 conn(0x7f6bdc05f5a0 msgr2=0x7f6bdc0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.384+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 shutdown_connections 2020-07-18T03:34:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.384+0000 7f6be3ca1700 1 -- 172.21.15.47:0/469886793 wait complete. 2020-07-18T03:34:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2020-07-18T03:34:28.398 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-07-18T03:34:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.522+0000 7f56eadba700 1 Processor -- start 2020-07-18T03:34:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.522+0000 7f56eadba700 1 -- start start 2020-07-18T03:34:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.522+0000 7f56eadba700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 0x7f56e40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.522+0000 7f56eadba700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e4147de0 con 0x7f56e40b59b0 2020-07-18T03:34:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.522+0000 7f56eadba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40ae000 con 0x7f56e40ad330 2020-07-18T03:34:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.522+0000 7f56eadba700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4148010 con 0x7f56e40b3a10 2020-07-18T03:34:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e8b56700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f56e40ad330 0x7f56e4143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45174/0 (socket says 172.21.15.47:45174) 2020-07-18T03:34:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e8b56700 1 -- 172.21.15.47:0/3523700800 learned_addr learned my addr 172.21.15.47:0/3523700800 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e9357700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 0x7f56e40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550315914 0 0) 0x7f56e40ae000 con 0x7f56e40ad330 2020-07-18T03:34:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d0003240 con 0x7f56e40ad330 2020-07-18T03:34:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86630756 0 0) 0x7f56e4147de0 con 0x7f56e40b59b0 2020-07-18T03:34:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e40ae000 con 0x7f56e40b59b0 2020-07-18T03:34:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 320086302 0 0) 0x7f56d0003240 con 0x7f56e40ad330 2020-07-18T03:34:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 msgr2=0x7f56e40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 0x7f56e40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 >> v1:172.21.15.18:6789/0 conn(0x7f56e40b59b0 legacy=0x7f56e40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40ad710 con 0x7f56e40ad330 2020-07-18T03:34:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f56d40016d0 con 0x7f56e40ad330 2020-07-18T03:34:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 >> v1:172.21.15.47:6789/0 conn(0x7f56e40ad330 legacy=0x7f56e4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56d00038c0 0x7f56d0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 0x7f56d00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d00049d0 0x7f56d0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4148010 con 0x7f56d00038c0 2020-07-18T03:34:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0005340 con 0x7f56d00049d0 2020-07-18T03:34:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.523+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d000b760 con 0x7f56e40b3a10 2020-07-18T03:34:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d00049d0 0x7f56d0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e8b56700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56d00038c0 0x7f56d0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e3fff700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 0x7f56d00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 -- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 msgr2=0x7f56d00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f56e40b3a10 0x7f56d00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 -- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56d00038c0 msgr2=0x7f56d0003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56d00038c0 0x7f56d0003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 -- 172.21.15.47:0/3523700800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d0003240 con 0x7f56d00049d0 2020-07-18T03:34:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d00049d0 0x7f56d0004e50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f56d0004450 tx=0x7f56dc0091d0).ready entity=mon.1 client_cookie=66e560451e93167 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc00ec50 con 0x7f56d00049d0 2020-07-18T03:34:28.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc00edb0 con 0x7f56d00049d0 2020-07-18T03:34:28.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e37fe700 1 -- 172.21.15.47:0/3523700800 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc0125e0 con 0x7f56d00049d0 2020-07-18T03:34:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56eadba700 1 -- 172.21.15.47:0/3523700800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d00049d0 msgr2=0x7f56d0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56eadba700 1 --2- 172.21.15.47:0/3523700800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d00049d0 0x7f56d0004e50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f56d0004450 tx=0x7f56dc0091d0).stop 2020-07-18T03:34:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56e9357700 1 -- 172.21.15.47:0/3523700800 reap_dead start 2020-07-18T03:34:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56eadba700 1 -- 172.21.15.47:0/3523700800 shutdown_connections 2020-07-18T03:34:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56eadba700 1 -- 172.21.15.47:0/3523700800 >> 172.21.15.47:0/3523700800 conn(0x7f56e405f5a0 msgr2=0x7f56e405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56eadba700 1 -- 172.21.15.47:0/3523700800 shutdown_connections 2020-07-18T03:34:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.524+0000 7f56eadba700 1 -- 172.21.15.47:0/3523700800 wait complete. 2020-07-18T03:34:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.526+0000 7f56eadba700 1 Processor -- start 2020-07-18T03:34:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.526+0000 7f56eadba700 1 -- start start 2020-07-18T03:34:28.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.526+0000 7f56eadba700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f56e4140d50 0x7f56e41411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.546 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.526+0000 7f56eadba700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e4155b90 con 0x7f56e4146d20 2020-07-18T03:34:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.526+0000 7f56eadba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e4143e00 con 0x7f56e4142080 2020-07-18T03:34:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.526+0000 7f56eadba700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e41416b0 con 0x7f56e4140d50 2020-07-18T03:34:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f56e4142080 0x7f56e4144540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45188/0 (socket says 172.21.15.47:45188) 2020-07-18T03:34:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e3fff700 1 -- 172.21.15.47:0/2701649894 learned_addr learned my addr 172.21.15.47:0/2701649894 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e9357700 1 --2- 172.21.15.47:0/2701649894 >> v2:172.21.15.18:3300/0 conn(0x7f56e4140d50 0x7f56e41411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609643209 0 0) 0x7f56e4143e00 con 0x7f56e4142080 2020-07-18T03:34:28.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc003240 con 0x7f56e4142080 2020-07-18T03:34:28.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130819546 0 0) 0x7f56e4155b90 con 0x7f56e4146d20 2020-07-18T03:34:28.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e4143e00 con 0x7f56e4146d20 2020-07-18T03:34:28.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2189072060 0 0) 0x7f56bc003240 con 0x7f56e4142080 2020-07-18T03:34:28.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e9357700 1 -- 172.21.15.47:0/2701649894 >> v1:172.21.15.47:6789/0 conn(0x7f56e4142080 legacy=0x7f56e4144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e9357700 1 -- 172.21.15.47:0/2701649894 >> v1:172.21.15.18:6789/0 conn(0x7f56e4146d20 legacy=0x7f56e40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e9357700 1 -- 172.21.15.47:0/2701649894 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e41440a0 con 0x7f56e4140d50 2020-07-18T03:34:28.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e9357700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56e4140d50 0x7f56e41411c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f56d8008000 tx=0x7f56dc01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:28.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc00ec50 con 0x7f56e4140d50 2020-07-18T03:34:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8001110 con 0x7f56e4140d50 2020-07-18T03:34:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56dc006ca0 con 0x7f56e4140d50 2020-07-18T03:34:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.527+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e40b2ee0 con 0x7f56e4140d50 2020-07-18T03:34:28.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.528+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4144c50 con 0x7f56e4140d50 2020-07-18T03:34:28.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.528+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e4144230 con 0x7f56e4140d50 2020-07-18T03:34:28.552 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.529+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f56dc02b020 con 0x7f56e4140d50 2020-07-18T03:34:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.529+0000 7f56e0ff9700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f56bc050450 0x7f56bc052910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.529+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f56dc069720 con 0x7f56e4140d50 2020-07-18T03:34:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.529+0000 7f56e8b56700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f56bc050450 0x7f56bc052910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.529+0000 7f56e8b56700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f56bc050450 0x7f56bc052910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f56d400b000 tx=0x7f56d40076a0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:28.559 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.533+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f56dc041ed0 con 0x7f56e4140d50 2020-07-18T03:34:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.744+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f56e405b120 con 0x7f56e4140d50 2020-07-18T03:34:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.751+0000 7f56e0ff9700 1 -- 172.21.15.47:0/2701649894 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v40) v1 ==== 189+0+0 (secure 0 0 0) 0x7f56dc0454f0 con 0x7f56e4140d50 2020-07-18T03:34:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f56bc050450 msgr2=0x7f56bc052910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f56bc050450 0x7f56bc052910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f56d400b000 tx=0x7f56d40076a0).stop 2020-07-18T03:34:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56e4140d50 msgr2=0x7f56e41411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56e4140d50 0x7f56e41411c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f56d8008000 tx=0x7f56dc01a000).stop 2020-07-18T03:34:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 shutdown_connections 2020-07-18T03:34:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56e4140d50 0x7f56e41411c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 --2- 172.21.15.47:0/2701649894 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f56bc050450 0x7f56bc052910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.755+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 >> 172.21.15.47:0/2701649894 conn(0x7f56e405f5a0 msgr2=0x7f56e4147460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.756+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 shutdown_connections 2020-07-18T03:34:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.757+0000 7f56eadba700 1 -- 172.21.15.47:0/2701649894 wait complete. 2020-07-18T03:34:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 13 and data pool 14 2020-07-18T03:34:28.773 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.887+0000 7fa1bdf57700 1 Processor -- start 2020-07-18T03:34:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.887+0000 7fa1bdf57700 1 -- start start 2020-07-18T03:34:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1bdf57700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1b80b41c0 0x7fa1b80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1bdf57700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa1b80ad7c0 0x7fa1b81428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1bdf57700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80b3ac0 con 0x7fa1b80b5370 2020-07-18T03:34:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1bdf57700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80b5a10 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1bdf57700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80b0040 con 0x7fa1b80b41c0 2020-07-18T03:34:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1b6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1b80b41c0 0x7fa1b80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1b6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1b80b41c0 0x7fa1b80ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35874/0 (socket says 172.21.15.47:35874) 2020-07-18T03:34:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.888+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/2310515660 learned_addr learned my addr 172.21.15.47:0/2310515660 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b7fff700 1 --2- 172.21.15.47:0/2310515660 >> v2:172.21.15.18:3300/0 conn(0x7fa1b80ad7c0 0x7fa1b81428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/2310515660 >> v2:172.21.15.18:3300/0 conn(0x7fa1b80ad7c0 msgr2=0x7fa1b81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b6ffd700 1 --2- 172.21.15.47:0/2310515660 >> v2:172.21.15.18:3300/0 conn(0x7fa1b80ad7c0 0x7fa1b81428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/2310515660 >> v1:172.21.15.18:6789/0 conn(0x7fa1b80b5370 legacy=0x7fa1b805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/2310515660 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80b3d20 con 0x7fa1b80b41c0 2020-07-18T03:34:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b6ffd700 1 --2- 172.21.15.47:0/2310515660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b80b41c0 0x7fa1b80ad2d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fa1b80af270 tx=0x7fa1ac006e90).ready entity=mon.1 client_cookie=86229ffdfde38e8e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b67fc700 1 -- 172.21.15.47:0/2310515660 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1ac0098f0 con 0x7fa1b80b41c0 2020-07-18T03:34:28.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b67fc700 1 -- 172.21.15.47:0/2310515660 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac009a50 con 0x7fa1b80b41c0 2020-07-18T03:34:28.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1b67fc700 1 -- 172.21.15.47:0/2310515660 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1ac009df0 con 0x7fa1b80b41c0 2020-07-18T03:34:28.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 -- 172.21.15.47:0/2310515660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b80b41c0 msgr2=0x7fa1b80ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/2310515660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b80b41c0 0x7fa1b80ad2d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fa1b80af270 tx=0x7fa1ac006e90).stop 2020-07-18T03:34:28.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 -- 172.21.15.47:0/2310515660 shutdown_connections 2020-07-18T03:34:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/2310515660 >> v2:172.21.15.18:3300/0 conn(0x7fa1b80ad7c0 0x7fa1b81428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/2310515660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b80b41c0 0x7fa1b80ad2d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 -- 172.21.15.47:0/2310515660 >> 172.21.15.47:0/2310515660 conn(0x7fa1b80b9340 msgr2=0x7fa1b80b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 -- 172.21.15.47:0/2310515660 shutdown_connections 2020-07-18T03:34:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.889+0000 7fa1bdf57700 1 -- 172.21.15.47:0/2310515660 wait complete. 2020-07-18T03:34:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1bdf57700 1 Processor -- start 2020-07-18T03:34:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1bdf57700 1 -- start start 2020-07-18T03:34:28.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1bdf57700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1b80b5370 0x7fa1b8063b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1bdf57700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b8146940 con 0x7fa1b81461c0 2020-07-18T03:34:28.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1bdf57700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b81466a0 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1bdf57700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81427d0 con 0x7fa1b80b5370 2020-07-18T03:34:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1b77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1b80ad7c0 0x7fa1b80661c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45200/0 (socket says 172.21.15.47:45200) 2020-07-18T03:34:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.891+0000 7fa1b77fe700 1 -- 172.21.15.47:0/3004998144 learned_addr learned my addr 172.21.15.47:0/3004998144 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1b6ffd700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1b80b5370 0x7fa1b8063b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500841521 0 0) 0x7fa1b81466a0 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa190003240 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205759791 0 0) 0x7fa1b8146940 con 0x7fa1b81461c0 2020-07-18T03:34:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1b81466a0 con 0x7fa1b81461c0 2020-07-18T03:34:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3712350164 0 0) 0x7fa190003240 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa1b8146940 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fa1a80029e0 con 0x7fa1b80ad7c0 2020-07-18T03:34:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/3004998144 >> v1:172.21.15.47:6789/0 conn(0x7fa1b80ad7c0 legacy=0x7fa1b80661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/3004998144 >> v1:172.21.15.18:6789/0 conn(0x7fa1b81461c0 legacy=0x7fa1b8141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1b6ffd700 1 -- 172.21.15.47:0/3004998144 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b8149980 con 0x7fa1b80b5370 2020-07-18T03:34:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1b6ffd700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1b80b5370 0x7fa1b8063b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fa1b80af270 tx=0x7fa1ac015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1ac011d10 con 0x7fa1b80b5370 2020-07-18T03:34:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.892+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac011e70 con 0x7fa1b80b5370 2020-07-18T03:34:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.893+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b8142250 con 0x7fa1b80b5370 2020-07-18T03:34:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.893+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1ac01e650 con 0x7fa1b80b5370 2020-07-18T03:34:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.893+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b8148fe0 con 0x7fa1b80b5370 2020-07-18T03:34:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.893+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b8057450 con 0x7fa1b80b5370 2020-07-18T03:34:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.894+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa1ac01a040 con 0x7fa1b80b5370 2020-07-18T03:34:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.894+0000 7fa1977fe700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa190050770 0x7fa190052c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.894+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa1ac068ba0 con 0x7fa1b80b5370 2020-07-18T03:34:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.894+0000 7fa1b77fe700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa190050770 0x7fa190052c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:28.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.894+0000 7fa1b77fe700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa190050770 0x7fa190052c30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fa1a80027b0 tx=0x7fa1a8002e60).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:28.907 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:28.898+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa1ac041360 con 0x7fa1b80b5370 2020-07-18T03:34:29.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.056+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b80b01a0 con 0x7fa1b80b5370 2020-07-18T03:34:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.058+0000 7fa1977fe700 1 -- 172.21.15.47:0/3004998144 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16290 (secure 0 0 0) 0x7fa1ac02ffc0 con 0x7fa1b80b5370 2020-07-18T03:34:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa190050770 msgr2=0x7fa190052c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa190050770 0x7fa190052c30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fa1a80027b0 tx=0x7fa1a8002e60).stop 2020-07-18T03:34:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1b80b5370 msgr2=0x7fa1b8063b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1b80b5370 0x7fa1b8063b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fa1b80af270 tx=0x7fa1ac015000).stop 2020-07-18T03:34:29.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 shutdown_connections 2020-07-18T03:34:29.062 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa190050770 0x7fa190052c30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 --2- 172.21.15.47:0/3004998144 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa1b80b5370 0x7fa1b8063b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 >> 172.21.15.47:0/3004998144 conn(0x7fa1b80b9340 msgr2=0x7fa1b80b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:29.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 shutdown_connections 2020-07-18T03:34:29.063 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.059+0000 7fa1bdf57700 1 -- 172.21.15.47:0/3004998144 wait complete. 2020-07-18T03:34:29.063 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:29.064 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":78,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:28.745703+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:34:26.480589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-07-18T03:34:27.720390+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:29.072 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:29.186 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.185+0000 7f3cb258d700 1 Processor -- start 2020-07-18T03:34:29.187 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.185+0000 7f3cb258d700 1 -- start start 2020-07-18T03:34:29.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cb258d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3cac153b40 0x7f3cac155f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cb258d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac067090 con 0x7f3cac0afe50 2020-07-18T03:34:29.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cb258d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac0af470 con 0x7f3cac0adcb0 2020-07-18T03:34:29.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cb258d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0b0230 con 0x7f3cac153b40 2020-07-18T03:34:29.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cab7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3cac0adcb0 0x7f3cac148600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45210/0 (socket says 172.21.15.47:45210) 2020-07-18T03:34:29.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cab7fe700 1 -- 172.21.15.47:0/1466848699 learned_addr learned my addr 172.21.15.47:0/1466848699 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:29.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3cb0b2a700 1 --2- 172.21.15.47:0/1466848699 >> v2:172.21.15.18:3300/0 conn(0x7f3cac153b40 0x7f3cac155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633117341 0 0) 0x7f3cac0af470 con 0x7f3cac0adcb0 2020-07-18T03:34:29.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.186+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c8c003240 con 0x7f3cac0adcb0 2020-07-18T03:34:29.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1208410468 0 0) 0x7f3c8c003240 con 0x7f3cac0adcb0 2020-07-18T03:34:29.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 >> v2:172.21.15.18:3300/0 conn(0x7f3cac153b40 msgr2=0x7f3cac155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:29.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 --2- 172.21.15.47:0/1466848699 >> v2:172.21.15.18:3300/0 conn(0x7f3cac153b40 0x7f3cac155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 >> v1:172.21.15.18:6789/0 conn(0x7f3cac0afe50 legacy=0x7f3cac05f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0af710 con 0x7f3cac0adcb0 2020-07-18T03:34:29.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f3c940016d0 con 0x7f3cac0adcb0 2020-07-18T03:34:29.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 >> v1:172.21.15.47:6789/0 conn(0x7f3cac0adcb0 legacy=0x7f3cac148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c8c003730 0x7f3c8c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c8c0044e0 0x7f3c8c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8c004e50 0x7f3c8c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b0230 con 0x7f3c8c003730 2020-07-18T03:34:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c004350 con 0x7f3c8c0044e0 2020-07-18T03:34:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c00bc50 con 0x7f3c8c004e50 2020-07-18T03:34:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c8c0044e0 0x7f3c8c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cabfff700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c8c003730 0x7f3c8c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cb0b2a700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8c004e50 0x7f3c8c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 -- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8c004e50 msgr2=0x7f3c8c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:29.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3c8c004e50 0x7f3c8c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 -- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c8c003730 msgr2=0x7f3c8c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:29.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c8c003730 0x7f3c8c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 -- 172.21.15.47:0/1466848699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c003240 con 0x7f3c8c0044e0 2020-07-18T03:34:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cb0b2a700 1 -- 172.21.15.47:0/1466848699 reap_dead start 2020-07-18T03:34:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3cab7fe700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c8c0044e0 0x7f3c8c004960 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f3c940013e0 tx=0x7f3cac067220).ready entity=mon.1 client_cookie=13beb8224d5b3761 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9400ce20 con 0x7f3c8c0044e0 2020-07-18T03:34:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c940070d0 con 0x7f3c8c0044e0 2020-07-18T03:34:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.187+0000 7f3caaffd700 1 -- 172.21.15.47:0/1466848699 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c94007430 con 0x7f3c8c0044e0 2020-07-18T03:34:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 -- 172.21.15.47:0/1466848699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c8c0044e0 msgr2=0x7f3c8c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c8c0044e0 0x7f3c8c004960 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f3c940013e0 tx=0x7f3cac067220).stop 2020-07-18T03:34:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 -- 172.21.15.47:0/1466848699 shutdown_connections 2020-07-18T03:34:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1466848699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c8c0044e0 0x7f3c8c004960 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 -- 172.21.15.47:0/1466848699 >> 172.21.15.47:0/1466848699 conn(0x7f3cac0b9340 msgr2=0x7f3cac0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 -- 172.21.15.47:0/1466848699 shutdown_connections 2020-07-18T03:34:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.188+0000 7f3cb258d700 1 -- 172.21.15.47:0/1466848699 wait complete. 2020-07-18T03:34:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.189+0000 7f3cb258d700 1 Processor -- start 2020-07-18T03:34:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.189+0000 7f3cb258d700 1 -- start start 2020-07-18T03:34:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb258d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cac0b2560 0x7f3cac139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb258d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3cac1431d0 0x7f3cac1419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb258d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac159340 con 0x7f3cac0b2940 2020-07-18T03:34:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb258d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac139d60 con 0x7f3cac0b2560 2020-07-18T03:34:29.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb258d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac13a000 con 0x7f3cac1431d0 2020-07-18T03:34:29.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cabfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cac0b2560 0x7f3cac139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cab7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3cac0b2940 0x7f3cac138ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45222/0 (socket says 172.21.15.47:45222) 2020-07-18T03:34:29.232 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cab7fe700 1 -- 172.21.15.47:0/1138215852 learned_addr learned my addr 172.21.15.47:0/1138215852 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:29.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb0b2a700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3cac1431d0 0x7f3cac1419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cabfff700 1 -- 172.21.15.47:0/1138215852 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3cac1431d0 msgr2=0x7f3cac1419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971809405 0 0) 0x7f3cac159340 con 0x7f3cac0b2940 2020-07-18T03:34:29.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cabfff700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3cac1431d0 0x7f3cac1419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.233 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cabfff700 1 -- 172.21.15.47:0/1138215852 >> v1:172.21.15.47:6789/0 conn(0x7f3cac0b2940 legacy=0x7f3cac138ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cabfff700 1 -- 172.21.15.47:0/1138215852 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac13a2a0 con 0x7f3cac0b2560 2020-07-18T03:34:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cb0b2a700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3cac1431d0 0x7f3cac1419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.190+0000 7f3cabfff700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cac0b2560 0x7f3cac139650 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f3c9c00d000 tx=0x7f3c9c007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.191+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9c0025e0 con 0x7f3cac0b2560 2020-07-18T03:34:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.191+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c002740 con 0x7f3cac0b2560 2020-07-18T03:34:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.191+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac1391d0 con 0x7f3cac0b2560 2020-07-18T03:34:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.191+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9c002aa0 con 0x7f3cac0b2560 2020-07-18T03:34:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.191+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac142060 con 0x7f3cac0b2560 2020-07-18T03:34:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.191+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac057450 con 0x7f3cac0b2560 2020-07-18T03:34:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.192+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f3c9c01b030 con 0x7f3cac0b2560 2020-07-18T03:34:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.192+0000 7f3ca3fff700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c8804fdd0 0x7f3c88052290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.192+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c9c067700 con 0x7f3cac0b2560 2020-07-18T03:34:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.192+0000 7f3cab7fe700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c8804fdd0 0x7f3c88052290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.192+0000 7f3cab7fe700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c8804fdd0 0x7f3c88052290 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f3c9400ca80 tx=0x7f3c94013000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.195+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f3c9c03fec0 con 0x7f3cac0b2560 2020-07-18T03:34:29.357 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.356+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cac06a100 con 0x7f3cac0b2560 2020-07-18T03:34:29.358 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.357+0000 7f3ca3fff700 1 -- 172.21.15.47:0/1138215852 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16290 (secure 0 0 0) 0x7f3c9c026fc0 con 0x7f3cac0b2560 2020-07-18T03:34:29.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:29.359 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":78,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:28.745703+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:34:26.480589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-07-18T03:34:27.720390+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:29.360 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c8804fdd0 msgr2=0x7f3c88052290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c8804fdd0 0x7f3c88052290 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f3c9400ca80 tx=0x7f3c94013000).stop 2020-07-18T03:34:29.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cac0b2560 msgr2=0x7f3cac139650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cac0b2560 0x7f3cac139650 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f3c9c00d000 tx=0x7f3c9c007b70).stop 2020-07-18T03:34:29.361 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 shutdown_connections 2020-07-18T03:34:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f3cac1431d0 0x7f3cac1419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cac0b2560 0x7f3cac139650 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 --2- 172.21.15.47:0/1138215852 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f3c8804fdd0 0x7f3c88052290 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 >> 172.21.15.47:0/1138215852 conn(0x7f3cac0b9340 msgr2=0x7f3cac0b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:29.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 shutdown_connections 2020-07-18T03:34:29.363 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.358+0000 7f3cb258d700 1 -- 172.21.15.47:0/1138215852 wait complete. 2020-07-18T03:34:29.370 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-07-18T03:34:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.488+0000 7f825d35d700 1 Processor -- start 2020-07-18T03:34:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.488+0000 7f825d35d700 1 -- start start 2020-07-18T03:34:29.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.488+0000 7f825d35d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82580b59b0 0x7f82580b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8258147de0 con 0x7f82580b6300 2020-07-18T03:34:29.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82580ae000 con 0x7f82580ad330 2020-07-18T03:34:29.492 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8258148010 con 0x7f82580b59b0 2020-07-18T03:34:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f82567fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82580b59b0 0x7f82580b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f8256ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f82580ad330 0x7f8258143640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45228/0 (socket says 172.21.15.47:45228) 2020-07-18T03:34:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f8256ffd700 1 -- 172.21.15.47:0/3836062375 learned_addr learned my addr 172.21.15.47:0/3836062375 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f82567fc700 1 -- 172.21.15.47:0/3836062375 >> v1:172.21.15.47:6789/0 conn(0x7f82580ad330 legacy=0x7f8258143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f82567fc700 1 -- 172.21.15.47:0/3836062375 >> v1:172.21.15.18:6789/0 conn(0x7f82580b6300 legacy=0x7f82580b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f8255ffb700 1 -- 172.21.15.47:0/3836062375 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939980780 0 0) 0x7f82580ae000 con 0x7f82580ad330 2020-07-18T03:34:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f82567fc700 1 -- 172.21.15.47:0/3836062375 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580ad710 con 0x7f82580b59b0 2020-07-18T03:34:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f82567fc700 1 --2- 172.21.15.47:0/3836062375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82580b59b0 0x7f82580b5e10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f82580ae4e0 tx=0x7f82480091d0).ready entity=mon.1 client_cookie=541edd7c1c43d6ba server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f8255ffb700 1 -- 172.21.15.47:0/3836062375 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f824800bc00 con 0x7f82580b59b0 2020-07-18T03:34:29.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f8255ffb700 1 -- 172.21.15.47:0/3836062375 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824800bd60 con 0x7f82580b59b0 2020-07-18T03:34:29.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f8255ffb700 1 -- 172.21.15.47:0/3836062375 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8248010660 con 0x7f82580b59b0 2020-07-18T03:34:29.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- 172.21.15.47:0/3836062375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82580b59b0 msgr2=0x7f82580b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 --2- 172.21.15.47:0/3836062375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82580b59b0 0x7f82580b5e10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f82580ae4e0 tx=0x7f82480091d0).stop 2020-07-18T03:34:29.496 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- 172.21.15.47:0/3836062375 shutdown_connections 2020-07-18T03:34:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 --2- 172.21.15.47:0/3836062375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82580b59b0 0x7f82580b5e10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- 172.21.15.47:0/3836062375 >> 172.21.15.47:0/3836062375 conn(0x7f825805f5a0 msgr2=0x7f825805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.489+0000 7f825d35d700 1 -- 172.21.15.47:0/3836062375 shutdown_connections 2020-07-18T03:34:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.490+0000 7f825d35d700 1 -- 172.21.15.47:0/3836062375 wait complete. 2020-07-18T03:34:29.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.491+0000 7f825d35d700 1 Processor -- start 2020-07-18T03:34:29.500 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.491+0000 7f825d35d700 1 -- start start 2020-07-18T03:34:29.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f825d35d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 0x7f82580b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f825d35d700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82580b6300 0x7f82581460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f825d35d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825813b640 con 0x7f82581392e0 2020-07-18T03:34:29.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f825d35d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82580633c0 con 0x7f82580ad330 2020-07-18T03:34:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f825d35d700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825813b8e0 con 0x7f82580b6300 2020-07-18T03:34:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f8256ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 0x7f82580b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f82567fc700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82580b6300 0x7f82581460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f8256ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 0x7f82580b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:49826/0 (socket says 172.21.15.47:49826) 2020-07-18T03:34:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f8256ffd700 1 -- 172.21.15.47:0/3243689944 learned_addr learned my addr 172.21.15.47:0/3243689944 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.492+0000 7f82577fe700 1 --1- 172.21.15.47:0/3243689944 >> v1:172.21.15.18:6789/0 conn(0x7f82581392e0 0x7f82580b2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35772/0 (socket says 172.21.15.47:35772) 2020-07-18T03:34:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f82567fc700 1 -- 172.21.15.47:0/3243689944 >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 msgr2=0x7f82580b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f82567fc700 1 --2- 172.21.15.47:0/3243689944 >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 0x7f82580b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f82567fc700 1 -- 172.21.15.47:0/3243689944 >> v1:172.21.15.18:6789/0 conn(0x7f82581392e0 legacy=0x7f82580b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f82567fc700 1 -- 172.21.15.47:0/3243689944 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825813bb80 con 0x7f82580b6300 2020-07-18T03:34:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f8256ffd700 1 --2- 172.21.15.47:0/3243689944 >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 0x7f82580b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f82567fc700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82580b6300 0x7f82581460f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f82580ae4e0 tx=0x7f8248018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82480187e0 con 0x7f82580b6300 2020-07-18T03:34:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8248018940 con 0x7f82580b6300 2020-07-18T03:34:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82580b1810 con 0x7f82580b6300 2020-07-18T03:34:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8248018ca0 con 0x7f82580b6300 2020-07-18T03:34:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.493+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8258063080 con 0x7f82580b6300 2020-07-18T03:34:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.494+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82580adf90 con 0x7f82580b6300 2020-07-18T03:34:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.494+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8248021020 con 0x7f82580b6300 2020-07-18T03:34:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.494+0000 7f823effd700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f823404fea0 0x7f8234052360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.494+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f824806a9b0 con 0x7f82580b6300 2020-07-18T03:34:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.494+0000 7f8256ffd700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f823404fea0 0x7f8234052360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.495+0000 7f8256ffd700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f823404fea0 0x7f8234052360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f824c00d000 tx=0x7f824c009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.498+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f82480430c0 con 0x7f82580b6300 2020-07-18T03:34:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:29.712+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8258148280 con 0x7f82580b6300 2020-07-18T03:34:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.381+0000 7f823effd700 1 -- 172.21.15.47:0/3243689944 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82480466e0 con 0x7f82580b6300 2020-07-18T03:34:30.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f823404fea0 msgr2=0x7f8234052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.384 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f823404fea0 0x7f8234052360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f824c00d000 tx=0x7f824c009040).stop 2020-07-18T03:34:30.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82580b6300 msgr2=0x7f82581460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82580b6300 0x7f82581460f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f82580ae4e0 tx=0x7f8248018000).stop 2020-07-18T03:34:30.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 shutdown_connections 2020-07-18T03:34:30.385 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f823404fea0 0x7f8234052360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 --2- 172.21.15.47:0/3243689944 >> v2:172.21.15.18:3300/0 conn(0x7f82580ad330 0x7f82580b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 --2- 172.21.15.47:0/3243689944 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f82580b6300 0x7f82581460f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 >> 172.21.15.47:0/3243689944 conn(0x7f825805f5a0 msgr2=0x7f825813bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 shutdown_connections 2020-07-18T03:34:30.386 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.383+0000 7f825d35d700 1 -- 172.21.15.47:0/3243689944 wait complete. 2020-07-18T03:34:30.397 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.518+0000 7f195ae79700 1 Processor -- start 2020-07-18T03:34:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.518+0000 7f195ae79700 1 -- start start 2020-07-18T03:34:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f195ae79700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19540b5370 0x7f195405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f195ae79700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f19540b41c0 0x7f19540ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f195ae79700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19540b3ac0 con 0x7f19540ad7c0 2020-07-18T03:34:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f195ae79700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19540b5a10 con 0x7f19540b41c0 2020-07-18T03:34:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f195ae79700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19540b0040 con 0x7f19540b5370 2020-07-18T03:34:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19540b5370 0x7f195405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19540b5370 0x7f195405fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35922/0 (socket says 172.21.15.47:35922) 2020-07-18T03:34:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 -- 172.21.15.47:0/1932060186 learned_addr learned my addr 172.21.15.47:0/1932060186 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1953fff700 1 --2- 172.21.15.47:0/1932060186 >> v2:172.21.15.18:3300/0 conn(0x7f19540b41c0 0x7f19540ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 -- 172.21.15.47:0/1932060186 >> v2:172.21.15.18:3300/0 conn(0x7f19540b41c0 msgr2=0x7f19540ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 --2- 172.21.15.47:0/1932060186 >> v2:172.21.15.18:3300/0 conn(0x7f19540b41c0 0x7f19540ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 -- 172.21.15.47:0/1932060186 >> v1:172.21.15.18:6789/0 conn(0x7f19540ad7c0 legacy=0x7f19541428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 -- 172.21.15.47:0/1932060186 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19540b3d20 con 0x7f19540b5370 2020-07-18T03:34:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f1958c15700 1 --2- 172.21.15.47:0/1932060186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b5370 0x7f195405fff0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f19540afb30 tx=0x7f1948006e90).ready entity=mon.1 client_cookie=718daff454013a4 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.519+0000 7f19537fe700 1 -- 172.21.15.47:0/1932060186 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1948009980 con 0x7f19540b5370 2020-07-18T03:34:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f19537fe700 1 -- 172.21.15.47:0/1932060186 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1948009ae0 con 0x7f19540b5370 2020-07-18T03:34:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f19537fe700 1 -- 172.21.15.47:0/1932060186 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1948009e40 con 0x7f19540b5370 2020-07-18T03:34:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 -- 172.21.15.47:0/1932060186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b5370 msgr2=0x7f195405fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 --2- 172.21.15.47:0/1932060186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b5370 0x7f195405fff0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f19540afb30 tx=0x7f1948006e90).stop 2020-07-18T03:34:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 -- 172.21.15.47:0/1932060186 shutdown_connections 2020-07-18T03:34:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 --2- 172.21.15.47:0/1932060186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b5370 0x7f195405fff0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 --2- 172.21.15.47:0/1932060186 >> v2:172.21.15.18:3300/0 conn(0x7f19540b41c0 0x7f19540ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 -- 172.21.15.47:0/1932060186 >> 172.21.15.47:0/1932060186 conn(0x7f19540b9340 msgr2=0x7f19540b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 -- 172.21.15.47:0/1932060186 shutdown_connections 2020-07-18T03:34:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.520+0000 7f195ae79700 1 -- 172.21.15.47:0/1932060186 wait complete. 2020-07-18T03:34:30.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.521+0000 7f195ae79700 1 Processor -- start 2020-07-18T03:34:30.528 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.521+0000 7f195ae79700 1 -- start start 2020-07-18T03:34:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f195ae79700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f19540ad7c0 0x7f1954138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f195ae79700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19540b41c0 0x7f19541297b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f195ae79700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195412a390 con 0x7f195413db10 2020-07-18T03:34:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f195ae79700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195412a180 con 0x7f19540b41c0 2020-07-18T03:34:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f195ae79700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1954065f20 con 0x7f19540ad7c0 2020-07-18T03:34:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19540b41c0 0x7f19541297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19540b41c0 0x7f19541297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35930/0 (socket says 172.21.15.47:35930) 2020-07-18T03:34:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 -- 172.21.15.47:0/1034205477 learned_addr learned my addr 172.21.15.47:0/1034205477 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1958c15700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f19540ad7c0 0x7f1954138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 -- 172.21.15.47:0/1034205477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f19540ad7c0 msgr2=0x7f1954138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f19540ad7c0 0x7f1954138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 -- 172.21.15.47:0/1034205477 >> v1:172.21.15.18:6789/0 conn(0x7f195413db10 legacy=0x7f1954065810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.522+0000 7f1953fff700 1 -- 172.21.15.47:0/1034205477 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954066460 con 0x7f19540b41c0 2020-07-18T03:34:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.523+0000 7f1953fff700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b41c0 0x7f19541297b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f194000a000 tx=0x7f1940008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.523+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1940009980 con 0x7f19540b41c0 2020-07-18T03:34:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.523+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1940009ae0 con 0x7f19540b41c0 2020-07-18T03:34:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.523+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954129ca0 con 0x7f19540b41c0 2020-07-18T03:34:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.523+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1940009e40 con 0x7f19540b41c0 2020-07-18T03:34:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.523+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195413a630 con 0x7f19540b41c0 2020-07-18T03:34:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.524+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19540604e0 con 0x7f19540b41c0 2020-07-18T03:34:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.524+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1940013480 con 0x7f19540b41c0 2020-07-18T03:34:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.524+0000 7f1950ff9700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f192c050010 0x7f192c0524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.524+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f19400685b0 con 0x7f19540b41c0 2020-07-18T03:34:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.524+0000 7f1958c15700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f192c050010 0x7f192c0524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.525+0000 7f1958c15700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f192c050010 0x7f192c0524d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f194801c000 tx=0x7f1948015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.528+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1940040ce0 con 0x7f19540b41c0 2020-07-18T03:34:30.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.735+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19540666b0 con 0x7f19540b41c0 2020-07-18T03:34:30.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.736+0000 7f1950ff9700 1 -- 172.21.15.47:0/1034205477 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f1940044300 con 0x7f19540b41c0 2020-07-18T03:34:30.737 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:30.738 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6639,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2078214370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2078214370},{"type":"v1","addr":"172.21.15.47:6835","nonce":2078214370}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:34:28.745915+0000","modified":"2020-07-18T03:34:30.375173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6645":{"gid":6645,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/303188627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":303188627},{"type":"v1","addr":"172.21.15.18:6833","nonce":303188627}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-07-18T03:34:30.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f192c050010 msgr2=0x7f192c0524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f192c050010 0x7f192c0524d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f194801c000 tx=0x7f1948015000).stop 2020-07-18T03:34:30.739 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b41c0 msgr2=0x7f19541297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b41c0 0x7f19541297b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f194000a000 tx=0x7f1940008040).stop 2020-07-18T03:34:30.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 shutdown_connections 2020-07-18T03:34:30.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f192c050010 0x7f192c0524d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f19540ad7c0 0x7f1954138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.740 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 --2- 172.21.15.47:0/1034205477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19540b41c0 0x7f19541297b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 >> 172.21.15.47:0/1034205477 conn(0x7f19540b9340 msgr2=0x7f19540b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:30.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 shutdown_connections 2020-07-18T03:34:30.741 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.737+0000 7f195ae79700 1 -- 172.21.15.47:0/1034205477 wait complete. 2020-07-18T03:34:30.741 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2020-07-18T03:34:30.753 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.866+0000 7fa483248700 1 Processor -- start 2020-07-18T03:34:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.866+0000 7fa483248700 1 -- start start 2020-07-18T03:34:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa483248700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa47c0ad100 0x7fa47c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa483248700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c147330 con 0x7fa47c1443f0 2020-07-18T03:34:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa483248700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0603c0 con 0x7fa47c147a00 2020-07-18T03:34:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa483248700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0b57e0 con 0x7fa47c0ad100 2020-07-18T03:34:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa47bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa47c147a00 0x7fa47c05f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45256/0 (socket says 172.21.15.47:45256) 2020-07-18T03:34:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa480fe4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa47c0ad100 0x7fa47c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.867+0000 7fa47bfff700 1 -- 172.21.15.47:0/3016825021 learned_addr learned my addr 172.21.15.47:0/3016825021 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa480fe4700 1 -- 172.21.15.47:0/3016825021 >> v1:172.21.15.47:6789/0 conn(0x7fa47c147a00 legacy=0x7fa47c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa47b7fe700 1 -- 172.21.15.47:0/3016825021 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723440992 0 0) 0x7fa47c147330 con 0x7fa47c1443f0 2020-07-18T03:34:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa480fe4700 1 -- 172.21.15.47:0/3016825021 >> v1:172.21.15.18:6789/0 conn(0x7fa47c1443f0 legacy=0x7fa47c05fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa480fe4700 1 -- 172.21.15.47:0/3016825021 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0b5a80 con 0x7fa47c0ad100 2020-07-18T03:34:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa480fe4700 1 --2- 172.21.15.47:0/3016825021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c0ad100 0x7fa47c0b4c10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa4700010d0 tx=0x7fa46c006dd0).ready entity=mon.1 client_cookie=2dab4b83db29124d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa47b7fe700 1 -- 172.21.15.47:0/3016825021 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c009730 con 0x7fa47c0ad100 2020-07-18T03:34:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa47b7fe700 1 -- 172.21.15.47:0/3016825021 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c009890 con 0x7fa47c0ad100 2020-07-18T03:34:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa47b7fe700 1 -- 172.21.15.47:0/3016825021 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c009bf0 con 0x7fa47c0ad100 2020-07-18T03:34:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 -- 172.21.15.47:0/3016825021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c0ad100 msgr2=0x7fa47c0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 --2- 172.21.15.47:0/3016825021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c0ad100 0x7fa47c0b4c10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa4700010d0 tx=0x7fa46c006dd0).stop 2020-07-18T03:34:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 -- 172.21.15.47:0/3016825021 shutdown_connections 2020-07-18T03:34:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 --2- 172.21.15.47:0/3016825021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c0ad100 0x7fa47c0b4c10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 -- 172.21.15.47:0/3016825021 >> 172.21.15.47:0/3016825021 conn(0x7fa47c0b9340 msgr2=0x7fa47c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 -- 172.21.15.47:0/3016825021 shutdown_connections 2020-07-18T03:34:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.868+0000 7fa483248700 1 -- 172.21.15.47:0/3016825021 wait complete. 2020-07-18T03:34:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 Processor -- start 2020-07-18T03:34:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 -- start start 2020-07-18T03:34:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa47c147a00 0x7fa47c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa47c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c064e00 con 0x7fa47c1443f0 2020-07-18T03:34:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0650a0 con 0x7fa47c147a00 2020-07-18T03:34:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa483248700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0b0530 con 0x7fa47c1424a0 2020-07-18T03:34:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa480fe4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa47c1443f0 0x7fa47c0624a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45262/0 (socket says 172.21.15.47:45262) 2020-07-18T03:34:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.870+0000 7fa480fe4700 1 -- 172.21.15.47:0/716485983 learned_addr learned my addr 172.21.15.47:0/716485983 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666383226 0 0) 0x7fa47c064e00 con 0x7fa47c1443f0 2020-07-18T03:34:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa454003240 con 0x7fa47c1443f0 2020-07-18T03:34:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa4817e5700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa47c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa47bfff700 1 --2- 172.21.15.47:0/716485983 >> v2:172.21.15.18:3300/0 conn(0x7fa47c147a00 0x7fa47c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1988109275 0 0) 0x7fa454003240 con 0x7fa47c1443f0 2020-07-18T03:34:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa47c064e00 con 0x7fa47c1443f0 2020-07-18T03:34:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fa46c009730 con 0x7fa47c1443f0 2020-07-18T03:34:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 msgr2=0x7fa47c156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa47c156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 >> v2:172.21.15.18:3300/0 conn(0x7fa47c147a00 msgr2=0x7fa47c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 --2- 172.21.15.47:0/716485983 >> v2:172.21.15.18:3300/0 conn(0x7fa47c147a00 0x7fa47c156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 >> v1:172.21.15.47:6789/0 conn(0x7fa47c1443f0 legacy=0x7fa47c0624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa454003800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa454003d10 0x7fa4540044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4540049d0 0x7fa454004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0650a0 con 0x7fa454003d10 2020-07-18T03:34:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0b0530 con 0x7fa4540049d0 2020-07-18T03:34:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa454004280 con 0x7fa47c1424a0 2020-07-18T03:34:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa4817e5700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4540049d0 0x7fa454004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa47bfff700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa454003d10 0x7fa4540044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa480fe4700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa454003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa4817e5700 1 -- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 msgr2=0x7fa454003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa4817e5700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa454003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa4817e5700 1 -- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa454003d10 msgr2=0x7fa4540044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.871+0000 7fa4817e5700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa454003d10 0x7fa4540044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa4817e5700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c064a40 con 0x7fa4540049d0 2020-07-18T03:34:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c061aa0 con 0x7fa4540049d0 2020-07-18T03:34:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa480fe4700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fa47c1424a0 0x7fa454003800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c156d70 con 0x7fa4540049d0 2020-07-18T03:34:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa4817e5700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4540049d0 0x7fa454004e50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fa47400d000 tx=0x7fa474007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa47400ca70 con 0x7fa4540049d0 2020-07-18T03:34:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47400cbd0 con 0x7fa4540049d0 2020-07-18T03:34:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa474014480 con 0x7fa4540049d0 2020-07-18T03:34:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.872+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c057450 con 0x7fa4540049d0 2020-07-18T03:34:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.873+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fa4740145e0 con 0x7fa4540049d0 2020-07-18T03:34:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.873+0000 7fa478ff9700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa454058170 0x7fa45405a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.873+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa474067990 con 0x7fa4540049d0 2020-07-18T03:34:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.873+0000 7fa480fe4700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa454058170 0x7fa45405a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.874+0000 7fa480fe4700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa454058170 0x7fa45405a630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fa46c01a000 tx=0x7fa46c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:30.877+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fa474040120 con 0x7fa4540049d0 2020-07-18T03:34:31.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.036+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa47c06a100 con 0x7fa4540049d0 2020-07-18T03:34:31.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.037+0000 7fa478ff9700 1 -- 172.21.15.47:0/716485983 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fa474043740 con 0x7fa4540049d0 2020-07-18T03:34:31.039 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:31.039 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:29.747072+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:34:26.480589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-07-18T03:34:27.720390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:31.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa454058170 msgr2=0x7fa45405a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fa454058170 0x7fa45405a630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fa46c01a000 tx=0x7fa46c015000).stop 2020-07-18T03:34:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4540049d0 msgr2=0x7fa454004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 --2- 172.21.15.47:0/716485983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4540049d0 0x7fa454004e50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fa47400d000 tx=0x7fa474007bf0).stop 2020-07-18T03:34:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa4817e5700 1 -- 172.21.15.47:0/716485983 reap_dead start 2020-07-18T03:34:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa4817e5700 1 -- 172.21.15.47:0/716485983 reap_dead start 2020-07-18T03:34:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 shutdown_connections 2020-07-18T03:34:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 >> 172.21.15.47:0/716485983 conn(0x7fa47c0b9340 msgr2=0x7fa47c0b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 shutdown_connections 2020-07-18T03:34:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.039+0000 7fa483248700 1 -- 172.21.15.47:0/716485983 wait complete. 2020-07-18T03:34:31.051 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:34:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.171+0000 7f2fc159d700 1 Processor -- start 2020-07-18T03:34:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.171+0000 7f2fc159d700 1 -- start start 2020-07-18T03:34:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.171+0000 7f2fc159d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.171+0000 7f2fc159d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc153f80 0x7f2fbc157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.171+0000 7f2fc159d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc066c20 con 0x7f2fbc080510 2020-07-18T03:34:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.171+0000 7f2fc159d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc080bb0 con 0x7f2fbc153ad0 2020-07-18T03:34:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc157140 con 0x7f2fbc153f80 2020-07-18T03:34:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc153f80 0x7f2fbc157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc153f80 0x7f2fbc157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35960/0 (socket says 172.21.15.47:35960) 2020-07-18T03:34:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 -- 172.21.15.47:0/4291773826 learned_addr learned my addr 172.21.15.47:0/4291773826 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbaffd700 1 --2- 172.21.15.47:0/4291773826 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 -- 172.21.15.47:0/4291773826 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 msgr2=0x7f2fbc1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 --2- 172.21.15.47:0/4291773826 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 -- 172.21.15.47:0/4291773826 >> v1:172.21.15.18:6789/0 conn(0x7f2fbc080510 legacy=0x7f2fbc0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 -- 172.21.15.47:0/4291773826 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0808f0 con 0x7f2fbc153f80 2020-07-18T03:34:31.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fbbfff700 1 --2- 172.21.15.47:0/4291773826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc153f80 0x7f2fbc157ad0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f2fb40032b0 tx=0x7f2fb4009270).ready entity=mon.1 client_cookie=6cbbb1fefa58a87d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.181 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fba7fc700 1 -- 172.21.15.47:0/4291773826 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb400bc00 con 0x7f2fbc153f80 2020-07-18T03:34:31.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fba7fc700 1 -- 172.21.15.47:0/4291773826 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb400bd60 con 0x7f2fbc153f80 2020-07-18T03:34:31.183 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fba7fc700 1 -- 172.21.15.47:0/4291773826 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb4011400 con 0x7f2fbc153f80 2020-07-18T03:34:31.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 -- 172.21.15.47:0/4291773826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc153f80 msgr2=0x7f2fbc157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 --2- 172.21.15.47:0/4291773826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc153f80 0x7f2fbc157ad0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f2fb40032b0 tx=0x7f2fb4009270).stop 2020-07-18T03:34:31.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 -- 172.21.15.47:0/4291773826 shutdown_connections 2020-07-18T03:34:31.184 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 --2- 172.21.15.47:0/4291773826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc153f80 0x7f2fbc157ad0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 --2- 172.21.15.47:0/4291773826 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 -- 172.21.15.47:0/4291773826 >> 172.21.15.47:0/4291773826 conn(0x7f2fbc05c620 msgr2=0x7f2fbc0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:31.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.172+0000 7f2fc159d700 1 -- 172.21.15.47:0/4291773826 shutdown_connections 2020-07-18T03:34:31.185 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.173+0000 7f2fc159d700 1 -- 172.21.15.47:0/4291773826 wait complete. 2020-07-18T03:34:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.174+0000 7f2fc159d700 1 Processor -- start 2020-07-18T03:34:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.174+0000 7f2fc159d700 1 -- start start 2020-07-18T03:34:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fc159d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc080510 0x7f2fbc0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fc159d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fc159d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0b4e50 con 0x7f2fbc0b49b0 2020-07-18T03:34:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fc159d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc143810 con 0x7f2fbc153ad0 2020-07-18T03:34:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fc159d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b1b40 con 0x7f2fbc080510 2020-07-18T03:34:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc080510 0x7f2fbc0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc080510 0x7f2fbc0b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35962/0 (socket says 172.21.15.47:35962) 2020-07-18T03:34:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 -- 172.21.15.47:0/534424153 learned_addr learned my addr 172.21.15.47:0/534424153 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbaffd700 1 --2- 172.21.15.47:0/534424153 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 -- 172.21.15.47:0/534424153 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 msgr2=0x7f2fbc0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 --2- 172.21.15.47:0/534424153 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 -- 172.21.15.47:0/534424153 >> v1:172.21.15.18:6789/0 conn(0x7f2fbc0b49b0 legacy=0x7f2fbc0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 -- 172.21.15.47:0/534424153 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc156630 con 0x7f2fbc080510 2020-07-18T03:34:31.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.175+0000 7f2fbb7fe700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc080510 0x7f2fbc0b3240 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f2fbc064e60 tx=0x7f2fac007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.176+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fac00e4a0 con 0x7f2fbc080510 2020-07-18T03:34:31.194 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.176+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00e600 con 0x7f2fbc080510 2020-07-18T03:34:31.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.176+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc0b5eb0 con 0x7f2fbc080510 2020-07-18T03:34:31.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.176+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fac00e960 con 0x7f2fbc080510 2020-07-18T03:34:31.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.176+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc143e70 con 0x7f2fbc080510 2020-07-18T03:34:31.195 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.177+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc0b96c0 con 0x7f2fbc080510 2020-07-18T03:34:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.177+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f2fac019030 con 0x7f2fbc080510 2020-07-18T03:34:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.177+0000 7f2fa2ffd700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9804ff80 0x7f2f98052440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.177+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2fac067120 con 0x7f2fbc080510 2020-07-18T03:34:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.177+0000 7f2fbaffd700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9804ff80 0x7f2f98052440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.178+0000 7f2fbaffd700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9804ff80 0x7f2f98052440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f2fb000c000 tx=0x7f2fb0008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.181+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f2fac03f8b0 con 0x7f2fbc080510 2020-07-18T03:34:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.396+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2fbc061d20 con 0x7f2fbc080510 2020-07-18T03:34:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.400+0000 7f2fa2ffd700 1 -- 172.21.15.47:0/534424153 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2fac0170e0 con 0x7f2fbc080510 2020-07-18T03:34:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.401+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9804ff80 msgr2=0x7f2f98052440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.401+0000 7f2fc159d700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9804ff80 0x7f2f98052440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f2fb000c000 tx=0x7f2fb0008040).stop 2020-07-18T03:34:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.401+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc080510 msgr2=0x7f2fbc0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.401+0000 7f2fc159d700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc080510 0x7f2fbc0b3240 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f2fbc064e60 tx=0x7f2fac007c30).stop 2020-07-18T03:34:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 shutdown_connections 2020-07-18T03:34:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f2f9804ff80 0x7f2f98052440 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 --2- 172.21.15.47:0/534424153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc080510 0x7f2fbc0b3240 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 --2- 172.21.15.47:0/534424153 >> v2:172.21.15.18:3300/0 conn(0x7f2fbc153ad0 0x7f2fbc0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 >> 172.21.15.47:0/534424153 conn(0x7f2fbc05c620 msgr2=0x7f2fbc062440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 shutdown_connections 2020-07-18T03:34:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.402+0000 7f2fc159d700 1 -- 172.21.15.47:0/534424153 wait complete. 2020-07-18T03:34:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2020-07-18T03:34:31.413 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:34:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f779aa32700 1 Processor -- start 2020-07-18T03:34:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f779aa32700 1 -- start start 2020-07-18T03:34:31.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f779aa32700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7794153ad0 0x7f77941575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f779aa32700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7794080bb0 con 0x7f7794153f80 2020-07-18T03:34:31.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f779aa32700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7794157140 con 0x7f7794080510 2020-07-18T03:34:31.538 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f779aa32700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77940808f0 con 0x7f7794153ad0 2020-07-18T03:34:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f7798fcf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7794080510 0x7f77940b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45290/0 (socket says 172.21.15.47:45290) 2020-07-18T03:34:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.534+0000 7f7798fcf700 1 -- 172.21.15.47:0/1889957523 learned_addr learned my addr 172.21.15.47:0/1889957523 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f7793fff700 1 --2- 172.21.15.47:0/1889957523 >> v2:172.21.15.18:3300/0 conn(0x7f7794153ad0 0x7f77941575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f77937fe700 1 -- 172.21.15.47:0/1889957523 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394850582 0 0) 0x7f7794157140 con 0x7f7794080510 2020-07-18T03:34:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f77937fe700 1 -- 172.21.15.47:0/1889957523 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f777c003240 con 0x7f7794080510 2020-07-18T03:34:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f7793fff700 1 -- 172.21.15.47:0/1889957523 >> v1:172.21.15.47:6789/0 conn(0x7f7794080510 legacy=0x7f77940b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f7793fff700 1 -- 172.21.15.47:0/1889957523 >> v1:172.21.15.18:6789/0 conn(0x7f7794153f80 legacy=0x7f7794063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f77937fe700 1 -- 172.21.15.47:0/1889957523 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188511310 0 0) 0x7f7794080bb0 con 0x7f7794153f80 2020-07-18T03:34:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f7793fff700 1 -- 172.21.15.47:0/1889957523 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794080e50 con 0x7f7794153ad0 2020-07-18T03:34:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f7793fff700 1 --2- 172.21.15.47:0/1889957523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7794153ad0 0x7f77941575e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f7788002f30 tx=0x7f7788006dd0).ready entity=mon.0 client_cookie=193d0a4007e4120a server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f77937fe700 1 -- 172.21.15.47:0/1889957523 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7788009800 con 0x7f7794153ad0 2020-07-18T03:34:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f77937fe700 1 -- 172.21.15.47:0/1889957523 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788009960 con 0x7f7794153ad0 2020-07-18T03:34:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f77937fe700 1 -- 172.21.15.47:0/1889957523 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7788009cc0 con 0x7f7794153ad0 2020-07-18T03:34:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f779aa32700 1 -- 172.21.15.47:0/1889957523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7794153ad0 msgr2=0x7f77941575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f779aa32700 1 --2- 172.21.15.47:0/1889957523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7794153ad0 0x7f77941575e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f7788002f30 tx=0x7f7788006dd0).stop 2020-07-18T03:34:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f779aa32700 1 -- 172.21.15.47:0/1889957523 shutdown_connections 2020-07-18T03:34:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f779aa32700 1 --2- 172.21.15.47:0/1889957523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7794153ad0 0x7f77941575e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f779aa32700 1 -- 172.21.15.47:0/1889957523 >> 172.21.15.47:0/1889957523 conn(0x7f779405c620 msgr2=0x7f77940b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.535+0000 7f779aa32700 1 -- 172.21.15.47:0/1889957523 shutdown_connections 2020-07-18T03:34:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.536+0000 7f779aa32700 1 -- 172.21.15.47:0/1889957523 wait complete. 2020-07-18T03:34:31.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.537+0000 7f779aa32700 1 Processor -- start 2020-07-18T03:34:31.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.537+0000 7f779aa32700 1 -- start start 2020-07-18T03:34:31.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f779aa32700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7794080510 0x7f77940aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.553 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f779aa32700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f779415b900 0x7f7794159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f779aa32700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779415c440 con 0x7f7794153f80 2020-07-18T03:34:31.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f779aa32700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77941251e0 con 0x7f7794080510 2020-07-18T03:34:31.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f779aa32700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779415e2b0 con 0x7f779415b900 2020-07-18T03:34:31.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f7798fcf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7794080510 0x7f77940aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.554 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f7798fcf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7794080510 0x7f77940aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35978/0 (socket says 172.21.15.47:35978) 2020-07-18T03:34:31.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f7798fcf700 1 -- 172.21.15.47:0/1170998573 learned_addr learned my addr 172.21.15.47:0/1170998573 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:31.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.538+0000 7f77997d0700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f779415b900 0x7f7794159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7798fcf700 1 -- 172.21.15.47:0/1170998573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f779415b900 msgr2=0x7f7794159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.555 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7798fcf700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f779415b900 0x7f7794159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7798fcf700 1 -- 172.21.15.47:0/1170998573 >> v1:172.21.15.18:6789/0 conn(0x7f7794153f80 legacy=0x7f7794124ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7798fcf700 1 -- 172.21.15.47:0/1170998573 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940b4810 con 0x7f7794080510 2020-07-18T03:34:31.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7798fcf700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794080510 0x7f77940aeae0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f77840010d0 tx=0x7f7784007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77840072e0 con 0x7f7794080510 2020-07-18T03:34:31.556 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7784007440 con 0x7f7794080510 2020-07-18T03:34:31.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77840057e0 con 0x7f7794080510 2020-07-18T03:34:31.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77940b49a0 con 0x7f7794080510 2020-07-18T03:34:31.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.539+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77940b4e70 con 0x7f7794080510 2020-07-18T03:34:31.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.540+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77940b96c0 con 0x7f7794080510 2020-07-18T03:34:31.557 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.540+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7784010070 con 0x7f7794080510 2020-07-18T03:34:31.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.540+0000 7f7773fff700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f776c04ffa0 0x7f776c052460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.541+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7784067310 con 0x7f7794080510 2020-07-18T03:34:31.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.541+0000 7f7793fff700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f776c04ffa0 0x7f776c052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.541+0000 7f7793fff700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f776c04ffa0 0x7f776c052460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f779415ac70 tx=0x7f7788015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.558 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.544+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f778403faa0 con 0x7f7794080510 2020-07-18T03:34:31.761 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.760+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7794062010 con 0x7f7794080510 2020-07-18T03:34:31.765 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.764+0000 7f7773fff700 1 -- 172.21.15.47:0/1170998573 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f77840430c0 con 0x7f7794080510 2020-07-18T03:34:31.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f776c04ffa0 msgr2=0x7f776c052460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f776c04ffa0 0x7f776c052460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f779415ac70 tx=0x7f7788015000).stop 2020-07-18T03:34:31.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794080510 msgr2=0x7f77940aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794080510 0x7f77940aeae0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f77840010d0 tx=0x7f7784007a80).stop 2020-07-18T03:34:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 shutdown_connections 2020-07-18T03:34:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f776c04ffa0 0x7f776c052460 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f779415b900 0x7f7794159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 --2- 172.21.15.47:0/1170998573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794080510 0x7f77940aeae0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 >> 172.21.15.47:0/1170998573 conn(0x7f779405c620 msgr2=0x7f7794155190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 shutdown_connections 2020-07-18T03:34:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.766+0000 7f779aa32700 1 -- 172.21.15.47:0/1170998573 wait complete. 2020-07-18T03:34:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2020-07-18T03:34:31.777 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:34:31.900 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.897+0000 7f5c4a519700 1 Processor -- start 2020-07-18T03:34:31.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.897+0000 7f5c4a519700 1 -- start start 2020-07-18T03:34:31.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4a519700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5c44151110 0x7f5c440bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4a519700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c44157860 con 0x7f5c440b67a0 2020-07-18T03:34:31.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4a519700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c44157ad0 con 0x7f5c440bad40 2020-07-18T03:34:31.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4a519700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c44157d70 con 0x7f5c44151110 2020-07-18T03:34:31.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c49517700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5c440bad40 0x7f5c44150a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45306/0 (socket says 172.21.15.47:45306) 2020-07-18T03:34:31.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4b77b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5c44151110 0x7f5c440bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c49517700 1 -- 172.21.15.47:0/666274850 learned_addr learned my addr 172.21.15.47:0/666274850 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:31.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4b77b700 1 -- 172.21.15.47:0/666274850 >> v1:172.21.15.47:6789/0 conn(0x7f5c440bad40 legacy=0x7f5c44150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4b77b700 1 -- 172.21.15.47:0/666274850 >> v1:172.21.15.18:6789/0 conn(0x7f5c440b67a0 legacy=0x7f5c440b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c48d16700 1 -- 172.21.15.47:0/666274850 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602549419 0 0) 0x7f5c44157ad0 con 0x7f5c440bad40 2020-07-18T03:34:31.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.898+0000 7f5c4b77b700 1 -- 172.21.15.47:0/666274850 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c441539d0 con 0x7f5c44151110 2020-07-18T03:34:31.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4b77b700 1 --2- 172.21.15.47:0/666274850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c44151110 0x7f5c440bb1d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f5c3c0032b0 tx=0x7f5c3c0091d0).ready entity=mon.1 client_cookie=12066fd58116e85b server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c48d16700 1 -- 172.21.15.47:0/666274850 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3c00ba50 con 0x7f5c44151110 2020-07-18T03:34:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c48d16700 1 -- 172.21.15.47:0/666274850 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00bbb0 con 0x7f5c44151110 2020-07-18T03:34:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c48d16700 1 -- 172.21.15.47:0/666274850 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3c00bd20 con 0x7f5c44151110 2020-07-18T03:34:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 -- 172.21.15.47:0/666274850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c44151110 msgr2=0x7f5c440bb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.904 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 --2- 172.21.15.47:0/666274850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c44151110 0x7f5c440bb1d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f5c3c0032b0 tx=0x7f5c3c0091d0).stop 2020-07-18T03:34:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 -- 172.21.15.47:0/666274850 shutdown_connections 2020-07-18T03:34:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 --2- 172.21.15.47:0/666274850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c44151110 0x7f5c440bb1d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 -- 172.21.15.47:0/666274850 >> 172.21.15.47:0/666274850 conn(0x7f5c4406b070 msgr2=0x7f5c440ba630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 -- 172.21.15.47:0/666274850 shutdown_connections 2020-07-18T03:34:31.905 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.899+0000 7f5c4a519700 1 -- 172.21.15.47:0/666274850 wait complete. 2020-07-18T03:34:31.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 Processor -- start 2020-07-18T03:34:31.908 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 -- start start 2020-07-18T03:34:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 0x7f5c441554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5c44062ae0 0x7f5c44135bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c44155bb0 con 0x7f5c440b67a0 2020-07-18T03:34:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c44155990 con 0x7f5c44062ae0 2020-07-18T03:34:31.909 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.901+0000 7f5c4a519700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44135470 con 0x7f5c440bad40 2020-07-18T03:34:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 0x7f5c441554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 0x7f5c441554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3301/0 says I am v2:172.21.15.47:56822/0 (socket says 172.21.15.47:56822) 2020-07-18T03:34:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c4b77b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f5c44062ae0 0x7f5c44135bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 -- 172.21.15.47:0/4089344746 learned_addr learned my addr 172.21.15.47:0/4089344746 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:31.910 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 -- 172.21.15.47:0/4089344746 >> v2:172.21.15.18:3300/0 conn(0x7f5c44062ae0 msgr2=0x7f5c44135bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 --2- 172.21.15.47:0/4089344746 >> v2:172.21.15.18:3300/0 conn(0x7f5c44062ae0 0x7f5c44135bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 -- 172.21.15.47:0/4089344746 >> v1:172.21.15.18:6789/0 conn(0x7f5c440b67a0 legacy=0x7f5c44152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 -- 172.21.15.47:0/4089344746 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44135710 con 0x7f5c440bad40 2020-07-18T03:34:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c49517700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 0x7f5c441554a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5c34000b80 tx=0x7f5c34007a40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c340072e0 con 0x7f5c440bad40 2020-07-18T03:34:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34007440 con 0x7f5c440bad40 2020-07-18T03:34:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c441360a0 con 0x7f5c440bad40 2020-07-18T03:34:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c340057e0 con 0x7f5c440bad40 2020-07-18T03:34:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.902+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c440ad1e0 con 0x7f5c440bad40 2020-07-18T03:34:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.904+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c441358a0 con 0x7f5c440bad40 2020-07-18T03:34:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.904+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5c3401a060 con 0x7f5c440bad40 2020-07-18T03:34:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.904+0000 7f5c397fa700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c1c050020 0x7f5c1c0524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.904+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5c340675f0 con 0x7f5c440bad40 2020-07-18T03:34:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.904+0000 7f5c49d18700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c1c050020 0x7f5c1c0524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.904+0000 7f5c49d18700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c1c050020 0x7f5c1c0524e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f5c440b0230 tx=0x7f5c40007720).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:31.914 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:31.908+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5c3403fd00 con 0x7f5c440bad40 2020-07-18T03:34:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.124+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5c4405b8e0 con 0x7f5c440bad40 2020-07-18T03:34:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.128+0000 7f5c397fa700 1 -- 172.21.15.47:0/4089344746 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5c34018070 con 0x7f5c440bad40 2020-07-18T03:34:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c1c050020 msgr2=0x7f5c1c0524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c1c050020 0x7f5c1c0524e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f5c440b0230 tx=0x7f5c40007720).stop 2020-07-18T03:34:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 msgr2=0x7f5c441554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 0x7f5c441554a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5c34000b80 tx=0x7f5c34007a40).stop 2020-07-18T03:34:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 shutdown_connections 2020-07-18T03:34:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5c1c050020 0x7f5c1c0524e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 --2- 172.21.15.47:0/4089344746 >> v2:172.21.15.18:3300/0 conn(0x7f5c44062ae0 0x7f5c44135bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.129+0000 7f5c4a519700 1 --2- 172.21.15.47:0/4089344746 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5c440bad40 0x7f5c441554a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.130+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 >> 172.21.15.47:0/4089344746 conn(0x7f5c4406b070 msgr2=0x7f5c441560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.130+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 shutdown_connections 2020-07-18T03:34:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.130+0000 7f5c4a519700 1 -- 172.21.15.47:0/4089344746 wait complete. 2020-07-18T03:34:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.2 2020-07-18T03:34:32.141 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-07-18T03:34:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.263+0000 7f8b613d8700 1 Processor -- start 2020-07-18T03:34:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.263+0000 7f8b613d8700 1 -- start start 2020-07-18T03:34:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b613d8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8b5c153ad0 0x7f8b5c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b613d8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 0x7f8b5c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b613d8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5c066c20 con 0x7f8b5c080510 2020-07-18T03:34:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b613d8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c080bb0 con 0x7f8b5c153ad0 2020-07-18T03:34:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b613d8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c157140 con 0x7f8b5c153f80 2020-07-18T03:34:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b5c080510 0x7f8b5c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45318/0 (socket says 172.21.15.47:45318) 2020-07-18T03:34:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5b7fe700 1 -- 172.21.15.47:0/3478095161 learned_addr learned my addr 172.21.15.47:0/3478095161 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 0x7f8b5c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5affd700 1 --2- 172.21.15.47:0/3478095161 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c153ad0 0x7f8b5c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797831420 0 0) 0x7f8b5c066c20 con 0x7f8b5c080510 2020-07-18T03:34:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b44003240 con 0x7f8b5c080510 2020-07-18T03:34:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4154190846 0 0) 0x7f8b44003240 con 0x7f8b5c080510 2020-07-18T03:34:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 msgr2=0x7f8b5c157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 0x7f8b5c157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c153ad0 msgr2=0x7f8b5c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 --2- 172.21.15.47:0/3478095161 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c153ad0 0x7f8b5c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0808f0 con 0x7f8b5c080510 2020-07-18T03:34:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.264+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8b4c0016d0 con 0x7f8b5c080510 2020-07-18T03:34:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 >> v1:172.21.15.47:6789/0 conn(0x7f8b5c080510 legacy=0x7f8b5c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b44003730 0x7f8b44003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 0x7f8b440044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b440049d0 0x7f8b44004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c080bb0 con 0x7f8b44003730 2020-07-18T03:34:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c157140 con 0x7f8b440049d0 2020-07-18T03:34:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44004140 con 0x7f8b5c153f80 2020-07-18T03:34:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b440049d0 0x7f8b44004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5b7fe700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b44003730 0x7f8b44003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5affd700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 0x7f8b440044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 -- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 msgr2=0x7f8b440044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c153f80 0x7f8b440044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 -- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b44003730 msgr2=0x7f8b44003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b44003730 0x7f8b44003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 -- 172.21.15.47:0/3478095161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44003240 con 0x7f8b440049d0 2020-07-18T03:34:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b440049d0 0x7f8b44004e50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f8b5400c000 tx=0x7f8b540091d0).ready entity=mon.1 client_cookie=a264b7d3fb31530c server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5400ec30 con 0x7f8b440049d0 2020-07-18T03:34:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5400ed90 con 0x7f8b440049d0 2020-07-18T03:34:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5a7fc700 1 -- 172.21.15.47:0/3478095161 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b540125e0 con 0x7f8b440049d0 2020-07-18T03:34:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b613d8700 1 -- 172.21.15.47:0/3478095161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b440049d0 msgr2=0x7f8b44004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b613d8700 1 --2- 172.21.15.47:0/3478095161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b440049d0 0x7f8b44004e50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f8b5400c000 tx=0x7f8b540091d0).stop 2020-07-18T03:34:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b5bfff700 1 -- 172.21.15.47:0/3478095161 reap_dead start 2020-07-18T03:34:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b613d8700 1 -- 172.21.15.47:0/3478095161 shutdown_connections 2020-07-18T03:34:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b613d8700 1 -- 172.21.15.47:0/3478095161 >> 172.21.15.47:0/3478095161 conn(0x7f8b5c05c620 msgr2=0x7f8b5c0b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b613d8700 1 -- 172.21.15.47:0/3478095161 shutdown_connections 2020-07-18T03:34:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.265+0000 7f8b613d8700 1 -- 172.21.15.47:0/3478095161 wait complete. 2020-07-18T03:34:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.267+0000 7f8b613d8700 1 Processor -- start 2020-07-18T03:34:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.267+0000 7f8b613d8700 1 -- start start 2020-07-18T03:34:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b613d8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f8b5c0b3cf0 0x7f8b5c124b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b613d8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c130630 0x7f8b5c130aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b613d8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5c12f9b0 con 0x7f8b5c124730 2020-07-18T03:34:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b613d8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c12fc50 con 0x7f8b5c0b3cf0 2020-07-18T03:34:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b613d8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c12fef0 con 0x7f8b5c130630 2020-07-18T03:34:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b5b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b5c124730 0x7f8b5c0b1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45328/0 (socket says 172.21.15.47:45328) 2020-07-18T03:34:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b5b7fe700 1 -- 172.21.15.47:0/2569115756 learned_addr learned my addr 172.21.15.47:0/2569115756 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b5affd700 1 --2- 172.21.15.47:0/2569115756 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c0b3cf0 0x7f8b5c124b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c130630 0x7f8b5c130aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149047144 0 0) 0x7f8b5c12f9b0 con 0x7f8b5c124730 2020-07-18T03:34:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b38003240 con 0x7f8b5c124730 2020-07-18T03:34:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3214187567 0 0) 0x7f8b38003240 con 0x7f8b5c124730 2020-07-18T03:34:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b5c12f9b0 con 0x7f8b5c124730 2020-07-18T03:34:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f8b4c0019c0 con 0x7f8b5c124730 2020-07-18T03:34:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.268+0000 7f8b5bfff700 1 -- 172.21.15.47:0/2569115756 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c0b3cf0 msgr2=0x7f8b5c124b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/2569115756 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c0b3cf0 0x7f8b5c124b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b5bfff700 1 -- 172.21.15.47:0/2569115756 >> v1:172.21.15.47:6789/0 conn(0x7f8b5c124730 legacy=0x7f8b5c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b5bfff700 1 -- 172.21.15.47:0/2569115756 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c130190 con 0x7f8b5c130630 2020-07-18T03:34:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b5bfff700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c130630 0x7f8b5c130aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f8b5c065ca0 tx=0x7f8b5401a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5400ec30 con 0x7f8b5c130630 2020-07-18T03:34:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c1537b0 con 0x7f8b5c130630 2020-07-18T03:34:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54012a10 con 0x7f8b5c130630 2020-07-18T03:34:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b54006ca0 con 0x7f8b5c130630 2020-07-18T03:34:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.269+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1430f0 con 0x7f8b5c130630 2020-07-18T03:34:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.270+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c130320 con 0x7f8b5c130630 2020-07-18T03:34:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.270+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f8b54012d70 con 0x7f8b5c130630 2020-07-18T03:34:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.270+0000 7f8b42ffd700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b38050600 0x7f8b38052ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.270+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b54069c00 con 0x7f8b5c130630 2020-07-18T03:34:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.270+0000 7f8b5b7fe700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b38050600 0x7f8b38052ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.271+0000 7f8b5b7fe700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b38050600 0x7f8b38052ac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f8b4c001b30 tx=0x7f8b4c008b30).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.274+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f8b5406f020 con 0x7f8b5c130630 2020-07-18T03:34:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.494+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8b5c12e750 con 0x7f8b5c130630 2020-07-18T03:34:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.498+0000 7f8b42ffd700 1 -- 172.21.15.47:0/2569115756 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8b54042390 con 0x7f8b5c130630 2020-07-18T03:34:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b38050600 msgr2=0x7f8b38052ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b38050600 0x7f8b38052ac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f8b4c001b30 tx=0x7f8b4c008b30).stop 2020-07-18T03:34:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c130630 msgr2=0x7f8b5c130aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c130630 0x7f8b5c130aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f8b5c065ca0 tx=0x7f8b5401a000).stop 2020-07-18T03:34:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 shutdown_connections 2020-07-18T03:34:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f8b5c130630 0x7f8b5c130aa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 --2- 172.21.15.47:0/2569115756 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f8b38050600 0x7f8b38052ac0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 --2- 172.21.15.47:0/2569115756 >> v2:172.21.15.18:3300/0 conn(0x7f8b5c0b3cf0 0x7f8b5c124b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 >> 172.21.15.47:0/2569115756 conn(0x7f8b5c05c620 msgr2=0x7f8b5c157500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 shutdown_connections 2020-07-18T03:34:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.500+0000 7f8b613d8700 1 -- 172.21.15.47:0/2569115756 wait complete. 2020-07-18T03:34:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.3 2020-07-18T03:34:32.512 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.626+0000 7f51575c4700 1 Processor -- start 2020-07-18T03:34:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.626+0000 7f51575c4700 1 -- start start 2020-07-18T03:34:32.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51500ad100 0x7f51500b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5150147330 con 0x7f51501443f0 2020-07-18T03:34:32.629 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51500603c0 con 0x7f5150147a00 2020-07-18T03:34:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51500b57e0 con 0x7f51500ad100 2020-07-18T03:34:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5155360700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51500ad100 0x7f51500b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5154b5f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5150147a00 0x7f515005f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45338/0 (socket says 172.21.15.47:45338) 2020-07-18T03:34:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5154b5f700 1 -- 172.21.15.47:0/2563634752 learned_addr learned my addr 172.21.15.47:0/2563634752 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5155360700 1 -- 172.21.15.47:0/2563634752 >> v1:172.21.15.47:6789/0 conn(0x7f5150147a00 legacy=0x7f515005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5155360700 1 -- 172.21.15.47:0/2563634752 >> v1:172.21.15.18:6789/0 conn(0x7f51501443f0 legacy=0x7f515005fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f514ffff700 1 -- 172.21.15.47:0/2563634752 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441230878 0 0) 0x7f51500603c0 con 0x7f5150147a00 2020-07-18T03:34:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5155360700 1 -- 172.21.15.47:0/2563634752 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51500b5a80 con 0x7f51500ad100 2020-07-18T03:34:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f5155360700 1 --2- 172.21.15.47:0/2563634752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51500ad100 0x7f51500b4c10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f515005ca30 tx=0x7f51480091d0).ready entity=mon.1 client_cookie=61f4aec7062c073f server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f514ffff700 1 -- 172.21.15.47:0/2563634752 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514800ba10 con 0x7f51500ad100 2020-07-18T03:34:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f514ffff700 1 -- 172.21.15.47:0/2563634752 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514800bb70 con 0x7f51500ad100 2020-07-18T03:34:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f514ffff700 1 -- 172.21.15.47:0/2563634752 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f514800bce0 con 0x7f51500ad100 2020-07-18T03:34:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 -- 172.21.15.47:0/2563634752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51500ad100 msgr2=0x7f51500b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 --2- 172.21.15.47:0/2563634752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51500ad100 0x7f51500b4c10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f515005ca30 tx=0x7f51480091d0).stop 2020-07-18T03:34:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 -- 172.21.15.47:0/2563634752 shutdown_connections 2020-07-18T03:34:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 --2- 172.21.15.47:0/2563634752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51500ad100 0x7f51500b4c10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.627+0000 7f51575c4700 1 -- 172.21.15.47:0/2563634752 >> 172.21.15.47:0/2563634752 conn(0x7f51500b9340 msgr2=0x7f51500b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.628+0000 7f51575c4700 1 -- 172.21.15.47:0/2563634752 shutdown_connections 2020-07-18T03:34:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.628+0000 7f51575c4700 1 -- 172.21.15.47:0/2563634752 wait complete. 2020-07-18T03:34:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.629+0000 7f51575c4700 1 Processor -- start 2020-07-18T03:34:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.629+0000 7f51575c4700 1 -- start start 2020-07-18T03:34:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f51575c4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f51501443f0 0x7f5150066520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f51575c4700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5150063ed0 con 0x7f5150147a00 2020-07-18T03:34:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f51575c4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5150065db0 con 0x7f51501558a0 2020-07-18T03:34:32.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f51575c4700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5150065bc0 con 0x7f51501443f0 2020-07-18T03:34:32.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f5155b61700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51501558a0 0x7f5150155d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45346/0 (socket says 172.21.15.47:45346) 2020-07-18T03:34:32.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f5155b61700 1 -- 172.21.15.47:0/947607813 learned_addr learned my addr 172.21.15.47:0/947607813 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:32.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f5155360700 1 --2- 172.21.15.47:0/947607813 >> v2:172.21.15.18:3300/0 conn(0x7f51501443f0 0x7f5150066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369542778 0 0) 0x7f5150065db0 con 0x7f51501558a0 2020-07-18T03:34:32.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128003240 con 0x7f51501558a0 2020-07-18T03:34:32.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 115693829 0 0) 0x7f5128003240 con 0x7f51501558a0 2020-07-18T03:34:32.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5150065db0 con 0x7f51501558a0 2020-07-18T03:34:32.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f5140002890 con 0x7f51501558a0 2020-07-18T03:34:32.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 >> v2:172.21.15.18:3300/0 conn(0x7f51501443f0 msgr2=0x7f5150066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 --2- 172.21.15.47:0/947607813 >> v2:172.21.15.18:3300/0 conn(0x7f51501443f0 0x7f5150066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.630+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 >> v1:172.21.15.47:6789/0 conn(0x7f51501558a0 legacy=0x7f5150155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 >> v1:172.21.15.18:6789/0 conn(0x7f5150147a00 legacy=0x7f51500654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5128003800 0x7f5128003be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51280044e0 0x7f5128004940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5128004e30 0x7f512800b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800bc20 con 0x7f5128003800 2020-07-18T03:34:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800bd60 con 0x7f51280044e0 2020-07-18T03:34:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512800bea0 con 0x7f5128004e30 2020-07-18T03:34:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5155360700 1 --2- 172.21.15.47:0/947607813 >> v2:172.21.15.18:3300/0 conn(0x7f51501443f0 0x7f5150066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51280044e0 0x7f5128004940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5155b61700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5128004e30 0x7f512800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5155360700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5128003800 0x7f5128003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 -- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5128004e30 msgr2=0x7f512800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5128004e30 0x7f512800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 -- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5128003800 msgr2=0x7f5128003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5128003800 0x7f5128003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150066050 con 0x7f51280044e0 2020-07-18T03:34:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5150156440 con 0x7f51280044e0 2020-07-18T03:34:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5155360700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5128003800 0x7f5128003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5155b61700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f5128004e30 0x7f512800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5155b61700 1 -- 172.21.15.47:0/947607813 reap_dead start 2020-07-18T03:34:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5150062c70 con 0x7f51280044e0 2020-07-18T03:34:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f5154b5f700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51280044e0 0x7f5128004940 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f5144001030 tx=0x7f5144007bb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5144007320 con 0x7f51280044e0 2020-07-18T03:34:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144007480 con 0x7f51280044e0 2020-07-18T03:34:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.631+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51440057e0 con 0x7f51280044e0 2020-07-18T03:34:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.632+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51500adf50 con 0x7f51280044e0 2020-07-18T03:34:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.632+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f5144010030 con 0x7f51280044e0 2020-07-18T03:34:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.633+0000 7f514d7fa700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5128058540 0x7f512805aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.633+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5144067930 con 0x7f51280044e0 2020-07-18T03:34:32.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.633+0000 7f5155360700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5128058540 0x7f512805aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:32.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.633+0000 7f5155360700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5128058540 0x7f512805aa00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f514801b000 tx=0x7f5148016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:32.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.636+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f5144040060 con 0x7f51280044e0 2020-07-18T03:34:32.844 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.844+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5150142bc0 con 0x7f51280044e0 2020-07-18T03:34:32.845 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.844+0000 7f514d7fa700 1 -- 172.21.15.47:0/947607813 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f5144043680 con 0x7f51280044e0 2020-07-18T03:34:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:32.846 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6639,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2078214370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2078214370},{"type":"v1","addr":"172.21.15.47:6835","nonce":2078214370}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:34:28.745915+0000","modified":"2020-07-18T03:34:30.375173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6645":{"gid":6645,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/303188627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":303188627},{"type":"v1","addr":"172.21.15.18:6833","nonce":303188627}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-07-18T03:34:32.847 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.845+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5128058540 msgr2=0x7f512805aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.845+0000 7f51575c4700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5128058540 0x7f512805aa00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f514801b000 tx=0x7f5148016000).stop 2020-07-18T03:34:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.845+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51280044e0 msgr2=0x7f5128004940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.845+0000 7f51575c4700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51280044e0 0x7f5128004940 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f5144001030 tx=0x7f5144007bb0).stop 2020-07-18T03:34:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.846+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 shutdown_connections 2020-07-18T03:34:32.848 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.846+0000 7f51575c4700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f5128058540 0x7f512805aa00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.846+0000 7f51575c4700 1 --2- 172.21.15.47:0/947607813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51280044e0 0x7f5128004940 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.846+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 >> 172.21.15.47:0/947607813 conn(0x7f51500b9340 msgr2=0x7f51500b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.846+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 shutdown_connections 2020-07-18T03:34:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:32.846+0000 7f51575c4700 1 -- 172.21.15.47:0/947607813 wait complete. 2020-07-18T03:34:32.849 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2020-07-18T03:34:33.058 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:34:28.745915+0000', 'modified': '2020-07-18T03:34:30.375173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6645}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6645': {'gid': 6645, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/303188627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 303188627}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 303188627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:34:33.059 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:34:33.060 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-07-18T03:34:33.209 INFO:teuthology.orchestra.run.smithi018.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6645,"want_state":"up:active","state":"up:active","rank_uptime":4.4522835289999998,"mdsmap_epoch":42,"osdmap_epoch":79,"osdmap_epoch_barrier":79,"uptime":9.6776236220000005} 2020-07-18T03:34:33.209 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6645, "mdsmap_epoch": 42, "osdmap_epoch": 79, "osdmap_epoch_barrier": 79, "rank_uptime": 4.452283529, "state": "up:active", "uptime": 9.677623622, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:34:33.210 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.328+0000 7fc5745af700 1 Processor -- start 2020-07-18T03:34:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.328+0000 7fc5745af700 1 -- start start 2020-07-18T03:34:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc5745af700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc56c0b41c0 0x7fc56c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc5745af700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0ad7c0 0x7fc56c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc5745af700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c0b3ac0 con 0x7fc56c0b5370 2020-07-18T03:34:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc5745af700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0b5a10 con 0x7fc56c0b41c0 2020-07-18T03:34:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc5745af700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0b0040 con 0x7fc56c0ad7c0 2020-07-18T03:34:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc572b4c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0ad7c0 0x7fc56c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc571b4a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc56c0b41c0 0x7fc56c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc57234b700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fc56c0b5370 0x7fc56c05fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:35890/0 (socket says 172.21.15.47:35890) 2020-07-18T03:34:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc57234b700 1 -- 172.21.15.47:0/255675199 learned_addr learned my addr 172.21.15.47:0/255675199 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc571349700 1 -- 172.21.15.47:0/255675199 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657042622 0 0) 0x7fc56c0b3ac0 con 0x7fc56c0b5370 2020-07-18T03:34:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc572b4c700 1 -- 172.21.15.47:0/255675199 >> v2:172.21.15.18:3300/0 conn(0x7fc56c0b41c0 msgr2=0x7fc56c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc572b4c700 1 --2- 172.21.15.47:0/255675199 >> v2:172.21.15.18:3300/0 conn(0x7fc56c0b41c0 0x7fc56c0ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc572b4c700 1 -- 172.21.15.47:0/255675199 >> v1:172.21.15.18:6789/0 conn(0x7fc56c0b5370 legacy=0x7fc56c05fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc572b4c700 1 -- 172.21.15.47:0/255675199 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0b3d20 con 0x7fc56c0ad7c0 2020-07-18T03:34:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc571b4a700 1 --2- 172.21.15.47:0/255675199 >> v2:172.21.15.18:3300/0 conn(0x7fc56c0b41c0 0x7fc56c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.329+0000 7fc572b4c700 1 --2- 172.21.15.47:0/255675199 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0ad7c0 0x7fc56c1428f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fc56c0afb30 tx=0x7fc5640091d0).ready entity=mon.2 client_cookie=c348e64ec0f265e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc571349700 1 -- 172.21.15.47:0/255675199 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc56400b990 con 0x7fc56c0ad7c0 2020-07-18T03:34:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc571349700 1 -- 172.21.15.47:0/255675199 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56400baf0 con 0x7fc56c0ad7c0 2020-07-18T03:34:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc571349700 1 -- 172.21.15.47:0/255675199 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc56400bc60 con 0x7fc56c0ad7c0 2020-07-18T03:34:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 -- 172.21.15.47:0/255675199 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0ad7c0 msgr2=0x7fc56c1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 --2- 172.21.15.47:0/255675199 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0ad7c0 0x7fc56c1428f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fc56c0afb30 tx=0x7fc5640091d0).stop 2020-07-18T03:34:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 -- 172.21.15.47:0/255675199 shutdown_connections 2020-07-18T03:34:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 --2- 172.21.15.47:0/255675199 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0ad7c0 0x7fc56c1428f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 --2- 172.21.15.47:0/255675199 >> v2:172.21.15.18:3300/0 conn(0x7fc56c0b41c0 0x7fc56c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 -- 172.21.15.47:0/255675199 >> 172.21.15.47:0/255675199 conn(0x7fc56c0b9340 msgr2=0x7fc56c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:33.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 -- 172.21.15.47:0/255675199 shutdown_connections 2020-07-18T03:34:33.338 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.330+0000 7fc5745af700 1 -- 172.21.15.47:0/255675199 wait complete. 2020-07-18T03:34:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.331+0000 7fc5745af700 1 Processor -- start 2020-07-18T03:34:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.331+0000 7fc5745af700 1 -- start start 2020-07-18T03:34:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc5745af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc56c0b41c0 0x7fc56c140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc5745af700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0b5370 0x7fc56c13d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc5745af700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c143840 con 0x7fc56c14b750 2020-07-18T03:34:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc5745af700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c143660 con 0x7fc56c0b41c0 2020-07-18T03:34:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc5745af700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c062d90 con 0x7fc56c0b5370 2020-07-18T03:34:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc56c0b41c0 0x7fc56c140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc56c0b41c0 0x7fc56c140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36042/0 (socket says 172.21.15.47:36042) 2020-07-18T03:34:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 -- 172.21.15.47:0/2876871373 learned_addr learned my addr 172.21.15.47:0/2876871373 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc571b4a700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0b5370 0x7fc56c13d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 -- 172.21.15.47:0/2876871373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0b5370 msgr2=0x7fc56c13d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0b5370 0x7fc56c13d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 -- 172.21.15.47:0/2876871373 >> v1:172.21.15.18:6789/0 conn(0x7fc56c14b750 legacy=0x7fc56c061910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.332+0000 7fc57234b700 1 -- 172.21.15.47:0/2876871373 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c1432d0 con 0x7fc56c0b41c0 2020-07-18T03:34:33.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.333+0000 7fc57234b700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc56c0b41c0 0x7fc56c140ea0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fc568000b80 tx=0x7fc568007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:33.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.333+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5680072e0 con 0x7fc56c0b41c0 2020-07-18T03:34:33.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.333+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc568007440 con 0x7fc56c0b41c0 2020-07-18T03:34:33.344 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.333+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c141390 con 0x7fc56c0b41c0 2020-07-18T03:34:33.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.333+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5680057e0 con 0x7fc56c0b41c0 2020-07-18T03:34:33.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.333+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c064480 con 0x7fc56c0b41c0 2020-07-18T03:34:33.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.334+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c057450 con 0x7fc56c0b41c0 2020-07-18T03:34:33.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.334+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc568010070 con 0x7fc56c0b41c0 2020-07-18T03:34:33.345 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.334+0000 7fc55a7fc700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc544050040 0x7fc544052500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.334+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc568067290 con 0x7fc56c0b41c0 2020-07-18T03:34:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.334+0000 7fc571b4a700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc544050040 0x7fc544052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.334+0000 7fc571b4a700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc544050040 0x7fc544052500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fc55c008000 tx=0x7fc55c008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.337+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc56803fa20 con 0x7fc56c0b41c0 2020-07-18T03:34:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.496+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc56c062f00 con 0x7fc56c0b41c0 2020-07-18T03:34:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.497+0000 7fc55a7fc700 1 -- 172.21.15.47:0/2876871373 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fc568043040 con 0x7fc56c0b41c0 2020-07-18T03:34:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:33.499 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:29.747072+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:34:26.480589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-07-18T03:34:27.720390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:33.501 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc544050040 msgr2=0x7fc544052500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc544050040 0x7fc544052500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fc55c008000 tx=0x7fc55c008040).stop 2020-07-18T03:34:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc56c0b41c0 msgr2=0x7fc56c140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc56c0b41c0 0x7fc56c140ea0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fc568000b80 tx=0x7fc568007a60).stop 2020-07-18T03:34:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 shutdown_connections 2020-07-18T03:34:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc544050040 0x7fc544052500 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc56c0b41c0 0x7fc56c140ea0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.504 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 --2- 172.21.15.47:0/2876871373 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc56c0b5370 0x7fc56c13d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 >> 172.21.15.47:0/2876871373 conn(0x7fc56c0b9340 msgr2=0x7fc56c05d450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.498+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 shutdown_connections 2020-07-18T03:34:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.499+0000 7fc5745af700 1 -- 172.21.15.47:0/2876871373 wait complete. 2020-07-18T03:34:33.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:34:26.480589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:34:27.720390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:34:33.513 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:34:33.632 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.631+0000 7f7e3e98d700 1 Processor -- start 2020-07-18T03:34:33.633 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.631+0000 7f7e3e98d700 1 -- start start 2020-07-18T03:34:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e3e98d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7e38141fb0 0x7f7e38142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e3e98d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38061db0 con 0x7f7e38144e70 2020-07-18T03:34:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e3e98d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38062050 con 0x7f7e380b04b0 2020-07-18T03:34:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e3e98d700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e38145250 con 0x7f7e38141fb0 2020-07-18T03:34:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e3cf2a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e380b04b0 0x7f7e380b0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45374/0 (socket says 172.21.15.47:45374) 2020-07-18T03:34:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e3cf2a700 1 -- 172.21.15.47:0/396913874 learned_addr learned my addr 172.21.15.47:0/396913874 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658440331 0 0) 0x7f7e38062050 con 0x7f7e380b04b0 2020-07-18T03:34:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e24003240 con 0x7f7e380b04b0 2020-07-18T03:34:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.632+0000 7f7e377fe700 1 --2- 172.21.15.47:0/396913874 >> v2:172.21.15.18:3300/0 conn(0x7f7e38141fb0 0x7f7e38142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3333285974 0 0) 0x7f7e24003240 con 0x7f7e380b04b0 2020-07-18T03:34:33.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 >> v2:172.21.15.18:3300/0 conn(0x7f7e38141fb0 msgr2=0x7f7e38142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:33.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 --2- 172.21.15.47:0/396913874 >> v2:172.21.15.18:3300/0 conn(0x7f7e38141fb0 0x7f7e38142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.643 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 >> v1:172.21.15.18:6789/0 conn(0x7f7e38144e70 legacy=0x7f7e380b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380622f0 con 0x7f7e380b04b0 2020-07-18T03:34:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f7e30001a50 con 0x7f7e380b04b0 2020-07-18T03:34:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 >> v1:172.21.15.47:6789/0 conn(0x7f7e380b04b0 legacy=0x7f7e380b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e24003730 0x7f7e24003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e240044e0 0x7f7e24004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.645 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e24004e50 0x7f7e2400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38145250 con 0x7f7e24003730 2020-07-18T03:34:33.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e24004350 con 0x7f7e240044e0 2020-07-18T03:34:33.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400bc50 con 0x7f7e24004e50 2020-07-18T03:34:33.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e240044e0 0x7f7e24004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e37fff700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e24003730 0x7f7e24003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.650 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e3cf2a700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e24004e50 0x7f7e2400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 -- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e24004e50 msgr2=0x7f7e2400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f7e24004e50 0x7f7e2400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 -- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e24003730 msgr2=0x7f7e24003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e24003730 0x7f7e24003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 -- 172.21.15.47:0/396913874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e24003240 con 0x7f7e240044e0 2020-07-18T03:34:33.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e3cf2a700 1 -- 172.21.15.47:0/396913874 reap_dead start 2020-07-18T03:34:33.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e377fe700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e240044e0 0x7f7e24004960 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f7e2c00c000 tx=0x7f7e2c008040).ready entity=mon.1 client_cookie=2c09c4c43a83dd20 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:33.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c00b2d0 con 0x7f7e240044e0 2020-07-18T03:34:33.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c00e8a0 con 0x7f7e240044e0 2020-07-18T03:34:33.652 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e36ffd700 1 -- 172.21.15.47:0/396913874 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c00ec00 con 0x7f7e240044e0 2020-07-18T03:34:33.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e3e98d700 1 -- 172.21.15.47:0/396913874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e240044e0 msgr2=0x7f7e24004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e3e98d700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e240044e0 0x7f7e24004960 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f7e2c00c000 tx=0x7f7e2c008040).stop 2020-07-18T03:34:33.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.633+0000 7f7e3e98d700 1 -- 172.21.15.47:0/396913874 shutdown_connections 2020-07-18T03:34:33.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.634+0000 7f7e3e98d700 1 --2- 172.21.15.47:0/396913874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e240044e0 0x7f7e24004960 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:33.653 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.634+0000 7f7e3e98d700 1 -- 172.21.15.47:0/396913874 >> 172.21.15.47:0/396913874 conn(0x7f7e3805f5a0 msgr2=0x7f7e3805f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:33.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.634+0000 7f7e3e98d700 1 -- 172.21.15.47:0/396913874 shutdown_connections 2020-07-18T03:34:33.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.634+0000 7f7e3e98d700 1 -- 172.21.15.47:0/396913874 wait complete. 2020-07-18T03:34:33.654 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.635+0000 7f7e3e98d700 1 Processor -- start 2020-07-18T03:34:33.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.635+0000 7f7e3e98d700 1 -- start start 2020-07-18T03:34:33.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e3e98d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e38132f30 0x7f7e38066a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e3e98d700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38155d70 con 0x7f7e38132b50 2020-07-18T03:34:33.655 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e3e98d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38140da0 con 0x7f7e380b3a10 2020-07-18T03:34:33.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e3e98d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e381414e0 con 0x7f7e38132f30 2020-07-18T03:34:33.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e377fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e38132f30 0x7f7e38066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e3cf2a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e380b3a10 0x7f7e380b3ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45386/0 (socket says 172.21.15.47:45386) 2020-07-18T03:34:33.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e3cf2a700 1 -- 172.21.15.47:0/2391903556 learned_addr learned my addr 172.21.15.47:0/2391903556 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:33.656 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e377fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e38132f30 0x7f7e38066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36062/0 (socket says 172.21.15.47:36062) 2020-07-18T03:34:33.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e377fe700 1 -- 172.21.15.47:0/2391903556 >> v1:172.21.15.47:6789/0 conn(0x7f7e380b3a10 legacy=0x7f7e380b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e377fe700 1 -- 172.21.15.47:0/2391903556 >> v1:172.21.15.18:6789/0 conn(0x7f7e38132b50 legacy=0x7f7e38064410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:33.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e377fe700 1 -- 172.21.15.47:0/2391903556 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38141040 con 0x7f7e38132f30 2020-07-18T03:34:33.657 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105068640 0 0) 0x7f7e38140da0 con 0x7f7e380b3a10 2020-07-18T03:34:33.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.636+0000 7f7e377fe700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e38132f30 0x7f7e38066a70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f7e380658e0 tx=0x7f7e2c01a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:33.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.637+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c01c830 con 0x7f7e38132f30 2020-07-18T03:34:33.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.637+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e38133520 con 0x7f7e38132f30 2020-07-18T03:34:33.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.637+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c01c990 con 0x7f7e38132f30 2020-07-18T03:34:33.658 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.637+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e2c01ccf0 con 0x7f7e38132f30 2020-07-18T03:34:33.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.637+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e38155940 con 0x7f7e38132f30 2020-07-18T03:34:33.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.638+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e380b64b0 con 0x7f7e38132f30 2020-07-18T03:34:33.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.638+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f7e2c02b060 con 0x7f7e38132f30 2020-07-18T03:34:33.659 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.638+0000 7f7e17fff700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e1004fe40 0x7f7e10052300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:33.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.638+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7e2c069720 con 0x7f7e38132f30 2020-07-18T03:34:33.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.638+0000 7f7e37fff700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e1004fe40 0x7f7e10052300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:33.660 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.638+0000 7f7e37fff700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e1004fe40 0x7f7e10052300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7e2800d000 tx=0x7f7e28007b70).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:33.661 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.642+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f7e2c041ef0 con 0x7f7e38132f30 2020-07-18T03:34:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.799+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7e38155b90 con 0x7f7e38132f30 2020-07-18T03:34:34.000 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.799+0000 7f7e17fff700 1 -- 172.21.15.47:0/2391903556 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v79) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7e2c045450 con 0x7f7e38132f30 2020-07-18T03:34:34.000 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:34:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e1004fe40 msgr2=0x7f7e10052300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.002 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e1004fe40 0x7f7e10052300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7e2800d000 tx=0x7f7e28007b70).stop 2020-07-18T03:34:34.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e38132f30 msgr2=0x7f7e38066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e38132f30 0x7f7e38066a70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f7e380658e0 tx=0x7f7e2c01a000).stop 2020-07-18T03:34:34.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 shutdown_connections 2020-07-18T03:34:34.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7e1004fe40 0x7f7e10052300 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.003 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 --2- 172.21.15.47:0/2391903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e38132f30 0x7f7e38066a70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 >> 172.21.15.47:0/2391903556 conn(0x7f7e3805f5a0 msgr2=0x7f7e380b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 shutdown_connections 2020-07-18T03:34:34.004 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:33.800+0000 7f7e3e98d700 1 -- 172.21.15.47:0/2391903556 wait complete. 2020-07-18T03:34:34.005 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:34:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.125+0000 7f38c12ae700 1 Processor -- start 2020-07-18T03:34:34.126 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.125+0000 7f38c12ae700 1 -- start start 2020-07-18T03:34:34.128 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f38bc061c60 0x7f38bc0620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38bc0b3af0 0x7f38bc0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc0af4b0 con 0x7f38bc142a20 2020-07-18T03:34:34.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0b6d80 con 0x7f38bc061c60 2020-07-18T03:34:34.129 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0af710 con 0x7f38bc0b3af0 2020-07-18T03:34:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38bc0b3af0 0x7f38bc0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38bc0b3af0 0x7f38bc0b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36074/0 (socket says 172.21.15.47:36074) 2020-07-18T03:34:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 -- 172.21.15.47:0/211425184 learned_addr learned my addr 172.21.15.47:0/211425184 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:34.130 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38ba7fc700 1 --2- 172.21.15.47:0/211425184 >> v2:172.21.15.18:3300/0 conn(0x7f38bc061c60 0x7f38bc0620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 -- 172.21.15.47:0/211425184 >> v2:172.21.15.18:3300/0 conn(0x7f38bc061c60 msgr2=0x7f38bc0620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:34.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 --2- 172.21.15.47:0/211425184 >> v2:172.21.15.18:3300/0 conn(0x7f38bc061c60 0x7f38bc0620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 -- 172.21.15.47:0/211425184 >> v1:172.21.15.18:6789/0 conn(0x7f38bc142a20 legacy=0x7f38bc142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.131 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 -- 172.21.15.47:0/211425184 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0af9b0 con 0x7f38bc0b3af0 2020-07-18T03:34:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38bb7fe700 1 --2- 172.21.15.47:0/211425184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc0b3af0 0x7f38bc0b3f10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f38b40032b0 tx=0x7f38b4009270).ready entity=mon.1 client_cookie=a904551612cd94eb server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38b9ffb700 1 -- 172.21.15.47:0/211425184 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38b400bc00 con 0x7f38bc0b3af0 2020-07-18T03:34:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38b9ffb700 1 -- 172.21.15.47:0/211425184 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b400bd60 con 0x7f38bc0b3af0 2020-07-18T03:34:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38b9ffb700 1 -- 172.21.15.47:0/211425184 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38b4011400 con 0x7f38bc0b3af0 2020-07-18T03:34:34.132 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 -- 172.21.15.47:0/211425184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc0b3af0 msgr2=0x7f38bc0b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 --2- 172.21.15.47:0/211425184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc0b3af0 0x7f38bc0b3f10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f38b40032b0 tx=0x7f38b4009270).stop 2020-07-18T03:34:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 -- 172.21.15.47:0/211425184 shutdown_connections 2020-07-18T03:34:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 --2- 172.21.15.47:0/211425184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc0b3af0 0x7f38bc0b3f10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 --2- 172.21.15.47:0/211425184 >> v2:172.21.15.18:3300/0 conn(0x7f38bc061c60 0x7f38bc0620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.133 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.126+0000 7f38c12ae700 1 -- 172.21.15.47:0/211425184 >> 172.21.15.47:0/211425184 conn(0x7f38bc05f5a0 msgr2=0x7f38bc05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.127+0000 7f38c12ae700 1 -- 172.21.15.47:0/211425184 shutdown_connections 2020-07-18T03:34:34.134 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.127+0000 7f38c12ae700 1 -- 172.21.15.47:0/211425184 wait complete. 2020-07-18T03:34:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.128+0000 7f38c12ae700 1 Processor -- start 2020-07-18T03:34:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.128+0000 7f38c12ae700 1 -- start start 2020-07-18T03:34:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38c12ae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38bc142a20 0x7f38bc064b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38c12ae700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc065ef0 con 0x7f38bc061c60 2020-07-18T03:34:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38c12ae700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc066120 con 0x7f38bc063550 2020-07-18T03:34:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38c12ae700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0663c0 con 0x7f38bc142a20 2020-07-18T03:34:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38ba7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38bc142a20 0x7f38bc064b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38bb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38bc063550 0x7f38bc0b7ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45398/0 (socket says 172.21.15.47:45398) 2020-07-18T03:34:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38bb7fe700 1 -- 172.21.15.47:0/3646405059 learned_addr learned my addr 172.21.15.47:0/3646405059 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476931265 0 0) 0x7f38bc066120 con 0x7f38bc063550 2020-07-18T03:34:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38ba7fc700 1 -- 172.21.15.47:0/3646405059 >> v1:172.21.15.47:6789/0 conn(0x7f38bc063550 legacy=0x7f38bc0b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38ba7fc700 1 -- 172.21.15.47:0/3646405059 >> v1:172.21.15.18:6789/0 conn(0x7f38bc061c60 legacy=0x7f38bc066b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38ba7fc700 1 -- 172.21.15.47:0/3646405059 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc066660 con 0x7f38bc142a20 2020-07-18T03:34:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.129+0000 7f38ba7fc700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc142a20 0x7f38bc064b60 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f38ac00a000 tx=0x7f38ac008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.130+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38ac011940 con 0x7f38bc142a20 2020-07-18T03:34:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.130+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac011aa0 con 0x7f38bc142a20 2020-07-18T03:34:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.130+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc13ab60 con 0x7f38bc142a20 2020-07-18T03:34:34.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.130+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38ac011e00 con 0x7f38bc142a20 2020-07-18T03:34:34.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.130+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc13b0b0 con 0x7f38bc142a20 2020-07-18T03:34:34.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.131+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc05b120 con 0x7f38bc142a20 2020-07-18T03:34:34.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.131+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f38ac012420 con 0x7f38bc142a20 2020-07-18T03:34:34.140 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.131+0000 7f38a2ffd700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f389804fdf0 0x7f38980522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.131+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f38ac068000 con 0x7f38bc142a20 2020-07-18T03:34:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.131+0000 7f38baffd700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f389804fdf0 0x7f38980522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.132+0000 7f38baffd700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f389804fdf0 0x7f38980522b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f38bc0b87b0 tx=0x7f38b0007800).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.135+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f38ac0406b0 con 0x7f38bc142a20 2020-07-18T03:34:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.292+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38bc0690d0 con 0x7f38bc142a20 2020-07-18T03:34:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.292+0000 7f38a2ffd700 1 -- 172.21.15.47:0/3646405059 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38ac043cd0 con 0x7f38bc142a20 2020-07-18T03:34:34.294 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f389804fdf0 msgr2=0x7f38980522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f389804fdf0 0x7f38980522b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f38bc0b87b0 tx=0x7f38b0007800).stop 2020-07-18T03:34:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc142a20 msgr2=0x7f38bc064b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc142a20 0x7f38bc064b60 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f38ac00a000 tx=0x7f38ac008040).stop 2020-07-18T03:34:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 shutdown_connections 2020-07-18T03:34:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f389804fdf0 0x7f38980522b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 --2- 172.21.15.47:0/3646405059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38bc142a20 0x7f38bc064b60 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 >> 172.21.15.47:0/3646405059 conn(0x7f38bc05f5a0 msgr2=0x7f38bc05d400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 shutdown_connections 2020-07-18T03:34:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.294+0000 7f38c12ae700 1 -- 172.21.15.47:0/3646405059 wait complete. 2020-07-18T03:34:34.306 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:34:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.422+0000 7fdea8ae0700 1 Processor -- start 2020-07-18T03:34:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.422+0000 7fdea8ae0700 1 -- start start 2020-07-18T03:34:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea8ae0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea8ae0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea40b3a10 0x7fdea40b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea8ae0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea4147de0 con 0x7fdea40b59b0 2020-07-18T03:34:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea8ae0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea40ae000 con 0x7fdea40b3a10 2020-07-18T03:34:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea8ae0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea4148010 con 0x7fdea40ad330 2020-07-18T03:34:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea40b3a10 0x7fdea40b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea40b3a10 0x7fdea40b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36086/0 (socket says 172.21.15.47:36086) 2020-07-18T03:34:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 -- 172.21.15.47:0/1972250402 learned_addr learned my addr 172.21.15.47:0/1972250402 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea259c700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 -- 172.21.15.47:0/1972250402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 msgr2=0x7fdea4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea4143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 -- 172.21.15.47:0/1972250402 >> v1:172.21.15.18:6789/0 conn(0x7fdea40b59b0 legacy=0x7fdea40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea2d9d700 1 -- 172.21.15.47:0/1972250402 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40ad710 con 0x7fdea40b3a10 2020-07-18T03:34:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.423+0000 7fdea259c700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea2d9d700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b3a10 0x7fdea40b3ea0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fde9c0032b0 tx=0x7fde9c009270).ready entity=mon.1 client_cookie=ce513d9fb607d2b9 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea159a700 1 -- 172.21.15.47:0/1972250402 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde9c00bc00 con 0x7fdea40b3a10 2020-07-18T03:34:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea159a700 1 -- 172.21.15.47:0/1972250402 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c00bd60 con 0x7fdea40b3a10 2020-07-18T03:34:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea159a700 1 -- 172.21.15.47:0/1972250402 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde9c011400 con 0x7fdea40b3a10 2020-07-18T03:34:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 -- 172.21.15.47:0/1972250402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b3a10 msgr2=0x7fdea40b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b3a10 0x7fdea40b3ea0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fde9c0032b0 tx=0x7fde9c009270).stop 2020-07-18T03:34:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 -- 172.21.15.47:0/1972250402 shutdown_connections 2020-07-18T03:34:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b3a10 0x7fdea40b3ea0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/1972250402 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 -- 172.21.15.47:0/1972250402 >> 172.21.15.47:0/1972250402 conn(0x7fdea405f5a0 msgr2=0x7fdea405f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 -- 172.21.15.47:0/1972250402 shutdown_connections 2020-07-18T03:34:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.424+0000 7fdea8ae0700 1 -- 172.21.15.47:0/1972250402 wait complete. 2020-07-18T03:34:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.425+0000 7fdea8ae0700 1 Processor -- start 2020-07-18T03:34:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.425+0000 7fdea8ae0700 1 -- start start 2020-07-18T03:34:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea8ae0700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea40b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea8ae0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea40b59b0 0x7fdea4064f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea8ae0700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea4064cd0 con 0x7fdea4065470 2020-07-18T03:34:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea8ae0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea4149700 con 0x7fdea40b59b0 2020-07-18T03:34:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea8ae0700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41343d0 con 0x7fdea40ad330 2020-07-18T03:34:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea40b59b0 0x7fdea4064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea40b59b0 0x7fdea4064f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36090/0 (socket says 172.21.15.47:36090) 2020-07-18T03:34:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea1d9b700 1 -- 172.21.15.47:0/2449925073 learned_addr learned my addr 172.21.15.47:0/2449925073 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.426+0000 7fdea259c700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea40b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea1d9b700 1 -- 172.21.15.47:0/2449925073 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 msgr2=0x7fdea40b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea1d9b700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea40b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea1d9b700 1 -- 172.21.15.47:0/2449925073 >> v1:172.21.15.18:6789/0 conn(0x7fdea4065470 legacy=0x7fdea4133cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea1d9b700 1 -- 172.21.15.47:0/2449925073 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4134510 con 0x7fdea40b59b0 2020-07-18T03:34:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea259c700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea40b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea1d9b700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b59b0 0x7fdea4064f80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fdea40b0830 tx=0x7fde94007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde94007030 con 0x7fdea40b59b0 2020-07-18T03:34:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde94007190 con 0x7fdea40b59b0 2020-07-18T03:34:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea40b2400 con 0x7fdea40b59b0 2020-07-18T03:34:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde940064f0 con 0x7fdea40b59b0 2020-07-18T03:34:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.427+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea4149e20 con 0x7fdea40b59b0 2020-07-18T03:34:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.428+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea40b6a00 con 0x7fdea40b59b0 2020-07-18T03:34:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.428+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fde94010070 con 0x7fdea40b59b0 2020-07-18T03:34:34.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.428+0000 7fde927fc700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde78050090 0x7fde78052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.428+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fde94066f40 con 0x7fdea40b59b0 2020-07-18T03:34:34.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.428+0000 7fdea259c700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde78050090 0x7fde78052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.429+0000 7fdea259c700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde78050090 0x7fde78052550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fde98008000 tx=0x7fde98008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.450 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.432+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fde9403f6d0 con 0x7fdea40b59b0 2020-07-18T03:34:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.599+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdea40b4e40 con 0x7fdea40b59b0 2020-07-18T03:34:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.600+0000 7fde927fc700 1 -- 172.21.15.47:0/2449925073 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+10 (secure 0 0 0) 0x7fde94042cf0 con 0x7fdea40b59b0 2020-07-18T03:34:34.601 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde78050090 msgr2=0x7fde78052550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde78050090 0x7fde78052550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fde98008000 tx=0x7fde98008040).stop 2020-07-18T03:34:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b59b0 msgr2=0x7fdea4064f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b59b0 0x7fdea4064f80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fdea40b0830 tx=0x7fde94007c10).stop 2020-07-18T03:34:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 shutdown_connections 2020-07-18T03:34:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fde78050090 0x7fde78052550 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fdea40ad330 0x7fdea40b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 --2- 172.21.15.47:0/2449925073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea40b59b0 0x7fdea4064f80 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 >> 172.21.15.47:0/2449925073 conn(0x7fdea405f5a0 msgr2=0x7fdea40624a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 shutdown_connections 2020-07-18T03:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.601+0000 7fdea8ae0700 1 -- 172.21.15.47:0/2449925073 wait complete. 2020-07-18T03:34:34.613 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:34:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.730+0000 7fabaee89700 1 Processor -- start 2020-07-18T03:34:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabaee89700 1 -- start start 2020-07-18T03:34:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabaee89700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7faba8064710 0x7faba8061800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabaee89700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faba8143780 0x7faba8143c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabaee89700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba813d850 con 0x7faba805fff0 2020-07-18T03:34:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabaee89700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8143390 con 0x7faba8064710 2020-07-18T03:34:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabaee89700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8064de0 con 0x7faba8143780 2020-07-18T03:34:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabad686700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faba805fff0 0x7faba80ad250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45416/0 (socket says 172.21.15.47:45416) 2020-07-18T03:34:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabad686700 1 -- 172.21.15.47:0/2496378278 learned_addr learned my addr 172.21.15.47:0/2496378278 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.731+0000 7fabae688700 1 --2- 172.21.15.47:0/2496378278 >> v2:172.21.15.47:3300/0 conn(0x7faba8143780 0x7faba8143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabace85700 1 -- 172.21.15.47:0/2496378278 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219672903 0 0) 0x7faba813d850 con 0x7faba805fff0 2020-07-18T03:34:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabace85700 1 -- 172.21.15.47:0/2496378278 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab90003240 con 0x7faba805fff0 2020-07-18T03:34:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabade87700 1 --2- 172.21.15.47:0/2496378278 >> v2:172.21.15.18:3300/0 conn(0x7faba8064710 0x7faba8061800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabae688700 1 -- 172.21.15.47:0/2496378278 >> v2:172.21.15.18:3300/0 conn(0x7faba8064710 msgr2=0x7faba8061800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabae688700 1 --2- 172.21.15.47:0/2496378278 >> v2:172.21.15.18:3300/0 conn(0x7faba8064710 0x7faba8061800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabae688700 1 -- 172.21.15.47:0/2496378278 >> v1:172.21.15.47:6789/0 conn(0x7faba805fff0 legacy=0x7faba80ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabae688700 1 -- 172.21.15.47:0/2496378278 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8065080 con 0x7faba8143780 2020-07-18T03:34:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabade87700 1 --2- 172.21.15.47:0/2496378278 >> v2:172.21.15.18:3300/0 conn(0x7faba8064710 0x7faba8061800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabae688700 1 --2- 172.21.15.47:0/2496378278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba8143780 0x7faba8143c30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7faba00032b0 tx=0x7faba00091d0).ready entity=mon.1 client_cookie=b6434cf81fd9efd5 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabace85700 1 -- 172.21.15.47:0/2496378278 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba000b9a0 con 0x7faba8143780 2020-07-18T03:34:34.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabace85700 1 -- 172.21.15.47:0/2496378278 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba000bb00 con 0x7faba8143780 2020-07-18T03:34:34.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabace85700 1 -- 172.21.15.47:0/2496378278 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba000bc70 con 0x7faba8143780 2020-07-18T03:34:34.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 -- 172.21.15.47:0/2496378278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba8143780 msgr2=0x7faba8143c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.746 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 --2- 172.21.15.47:0/2496378278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba8143780 0x7faba8143c30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7faba00032b0 tx=0x7faba00091d0).stop 2020-07-18T03:34:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 -- 172.21.15.47:0/2496378278 shutdown_connections 2020-07-18T03:34:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 --2- 172.21.15.47:0/2496378278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba8143780 0x7faba8143c30 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 --2- 172.21.15.47:0/2496378278 >> v2:172.21.15.18:3300/0 conn(0x7faba8064710 0x7faba8061800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 -- 172.21.15.47:0/2496378278 >> 172.21.15.47:0/2496378278 conn(0x7faba80b91d0 msgr2=0x7faba80b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 -- 172.21.15.47:0/2496378278 shutdown_connections 2020-07-18T03:34:34.748 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.732+0000 7fabaee89700 1 -- 172.21.15.47:0/2496378278 wait complete. 2020-07-18T03:34:34.766 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.734+0000 7fabaee89700 1 Processor -- start 2020-07-18T03:34:34.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.734+0000 7fabaee89700 1 -- start start 2020-07-18T03:34:34.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabaee89700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faba813ef10 0x7faba813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabaee89700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba80b7250 con 0x7faba8064710 2020-07-18T03:34:34.767 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabaee89700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba80b74f0 con 0x7faba805fff0 2020-07-18T03:34:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabaee89700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8144340 con 0x7faba813ef10 2020-07-18T03:34:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabade87700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faba805fff0 0x7faba8140690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45422/0 (socket says 172.21.15.47:45422) 2020-07-18T03:34:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabade87700 1 -- 172.21.15.47:0/1674253368 learned_addr learned my addr 172.21.15.47:0/1674253368 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabae688700 1 --2- 172.21.15.47:0/1674253368 >> v2:172.21.15.47:3300/0 conn(0x7faba813ef10 0x7faba813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570452915 0 0) 0x7faba80b74f0 con 0x7faba805fff0 2020-07-18T03:34:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab80003240 con 0x7faba805fff0 2020-07-18T03:34:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabae688700 1 -- 172.21.15.47:0/1674253368 >> v1:172.21.15.47:6789/0 conn(0x7faba805fff0 legacy=0x7faba8140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabae688700 1 -- 172.21.15.47:0/1674253368 >> v1:172.21.15.18:6789/0 conn(0x7faba8064710 legacy=0x7faba8143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabae688700 1 -- 172.21.15.47:0/1674253368 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80b7790 con 0x7faba813ef10 2020-07-18T03:34:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabae688700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba813ef10 0x7faba813fd50 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7faba00032b0 tx=0x7faba0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba00107a0 con 0x7faba813ef10 2020-07-18T03:34:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0010900 con 0x7faba813ef10 2020-07-18T03:34:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba80b6f40 con 0x7faba813ef10 2020-07-18T03:34:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faba0010c60 con 0x7faba813ef10 2020-07-18T03:34:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.735+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba8136ff0 con 0x7faba813ef10 2020-07-18T03:34:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.736+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba80b7920 con 0x7faba813ef10 2020-07-18T03:34:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.736+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7faba001e070 con 0x7faba813ef10 2020-07-18T03:34:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.737+0000 7fab95ffb700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fab80050320 0x7fab800527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.737+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faba0068d10 con 0x7faba813ef10 2020-07-18T03:34:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.737+0000 7fabade87700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fab80050320 0x7fab800527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.737+0000 7fabade87700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fab80050320 0x7fab800527e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fab98002f30 tx=0x7fab98008040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.740+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7faba00414a0 con 0x7faba813ef10 2020-07-18T03:34:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.898+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faba80b5020 con 0x7faba813ef10 2020-07-18T03:34:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.898+0000 7fab95ffb700 1 -- 172.21.15.47:0/1674253368 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+10 (secure 0 0 0) 0x7faba00150a0 con 0x7faba813ef10 2020-07-18T03:34:34.900 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fab80050320 msgr2=0x7fab800527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fab80050320 0x7fab800527e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fab98002f30 tx=0x7fab98008040).stop 2020-07-18T03:34:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba813ef10 msgr2=0x7faba813fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba813ef10 0x7faba813fd50 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7faba00032b0 tx=0x7faba0016000).stop 2020-07-18T03:34:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 shutdown_connections 2020-07-18T03:34:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faba813ef10 0x7faba813fd50 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 --2- 172.21.15.47:0/1674253368 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fab80050320 0x7fab800527e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 >> 172.21.15.47:0/1674253368 conn(0x7faba80b91d0 msgr2=0x7faba80639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 shutdown_connections 2020-07-18T03:34:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:34.900+0000 7fabaee89700 1 -- 172.21.15.47:0/1674253368 wait complete. 2020-07-18T03:34:34.911 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:34:34.912 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:35.026 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.025+0000 7f70c2865700 1 Processor -- start 2020-07-18T03:34:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.025+0000 7f70c2865700 1 -- start start 2020-07-18T03:34:35.029 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c2865700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70bc0ad100 0x7f70bc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:35.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c2865700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f70bc147a00 0x7f70bc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:35.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c2865700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc147330 con 0x7f70bc05f5a0 2020-07-18T03:34:35.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c2865700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc0ae530 con 0x7f70bc147a00 2020-07-18T03:34:35.030 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c2865700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0b57e0 con 0x7f70bc0ad100 2020-07-18T03:34:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c0e02700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f70bc05f5a0 0x7f70bc05fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45434/0 (socket says 172.21.15.47:45434) 2020-07-18T03:34:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70c0e02700 1 -- 172.21.15.47:0/53526755 learned_addr learned my addr 172.21.15.47:0/53526755 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70bb7fe700 1 --2- 172.21.15.47:0/53526755 >> v2:172.21.15.18:3300/0 conn(0x7f70bc147a00 0x7f70bc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.026+0000 7f70bbfff700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70bc0ad100 0x7f70bc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70baffd700 1 -- 172.21.15.47:0/53526755 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435520462 0 0) 0x7f70bc147330 con 0x7f70bc05f5a0 2020-07-18T03:34:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70baffd700 1 -- 172.21.15.47:0/53526755 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a8003240 con 0x7f70bc05f5a0 2020-07-18T03:34:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70bb7fe700 1 -- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70bc0ad100 msgr2=0x7f70bc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70bb7fe700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70bc0ad100 0x7f70bc0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70bb7fe700 1 -- 172.21.15.47:0/53526755 >> v1:172.21.15.47:6789/0 conn(0x7f70bc05f5a0 legacy=0x7f70bc05fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70bb7fe700 1 -- 172.21.15.47:0/53526755 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0b5a80 con 0x7f70bc147a00 2020-07-18T03:34:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70bbfff700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70bc0ad100 0x7f70bc0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70bb7fe700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70bc147a00 0x7f70bc147e40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f70b0002f30 tx=0x7f70b0006dd0).ready entity=mon.0 client_cookie=40f37b145e5e1900 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70baffd700 1 -- 172.21.15.47:0/53526755 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70b00096a0 con 0x7f70bc147a00 2020-07-18T03:34:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70baffd700 1 -- 172.21.15.47:0/53526755 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0009800 con 0x7f70bc147a00 2020-07-18T03:34:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70baffd700 1 -- 172.21.15.47:0/53526755 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70b0009b60 con 0x7f70bc147a00 2020-07-18T03:34:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 -- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70bc147a00 msgr2=0x7f70bc147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70bc147a00 0x7f70bc147e40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f70b0002f30 tx=0x7f70b0006dd0).stop 2020-07-18T03:34:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 -- 172.21.15.47:0/53526755 shutdown_connections 2020-07-18T03:34:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f70bc0ad100 0x7f70bc0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 --2- 172.21.15.47:0/53526755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70bc147a00 0x7f70bc147e40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 -- 172.21.15.47:0/53526755 >> 172.21.15.47:0/53526755 conn(0x7f70bc0b9340 msgr2=0x7f70bc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 -- 172.21.15.47:0/53526755 shutdown_connections 2020-07-18T03:34:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.027+0000 7f70c2865700 1 -- 172.21.15.47:0/53526755 wait complete. 2020-07-18T03:34:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.029+0000 7f70c2865700 1 Processor -- start 2020-07-18T03:34:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.029+0000 7f70c2865700 1 -- start start 2020-07-18T03:34:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c2865700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f70bc151890 0x7f70bc15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c2865700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc13f730 con 0x7f70bc05f5a0 2020-07-18T03:34:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c2865700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc151d40 con 0x7f70bc0ad100 2020-07-18T03:34:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c2865700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc140b90 con 0x7f70bc151890 2020-07-18T03:34:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70bb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f70bc0ad100 0x7f70bc13e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45438/0 (socket says 172.21.15.47:45438) 2020-07-18T03:34:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70bb7fe700 1 -- 172.21.15.47:0/994314849 learned_addr learned my addr 172.21.15.47:0/994314849 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c0e02700 1 --2- 172.21.15.47:0/994314849 >> v2:172.21.15.47:3300/0 conn(0x7f70bc151890 0x7f70bc15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078964271 0 0) 0x7f70bc151d40 con 0x7f70bc0ad100 2020-07-18T03:34:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7098003240 con 0x7f70bc0ad100 2020-07-18T03:34:35.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c0e02700 1 -- 172.21.15.47:0/994314849 >> v1:172.21.15.47:6789/0 conn(0x7f70bc0ad100 legacy=0x7f70bc13e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:35.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635315597 0 0) 0x7f70bc13f730 con 0x7f70bc05f5a0 2020-07-18T03:34:35.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c0e02700 1 -- 172.21.15.47:0/994314849 >> v1:172.21.15.18:6789/0 conn(0x7f70bc05f5a0 legacy=0x7f70bc0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:35.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c0e02700 1 -- 172.21.15.47:0/994314849 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc13f150 con 0x7f70bc151890 2020-07-18T03:34:35.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c0e02700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70bc151890 0x7f70bc15b4b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f70b4001450 tx=0x7f70b4007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:35.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70b4005970 con 0x7f70bc151890 2020-07-18T03:34:35.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc13f3f0 con 0x7f70bc151890 2020-07-18T03:34:35.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b4005ad0 con 0x7f70bc151890 2020-07-18T03:34:35.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70b4005e30 con 0x7f70bc151890 2020-07-18T03:34:35.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.030+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc0b05d0 con 0x7f70bc151890 2020-07-18T03:34:35.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.031+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc057450 con 0x7f70bc151890 2020-07-18T03:34:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.032+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f70b4010030 con 0x7f70bc151890 2020-07-18T03:34:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.033+0000 7f70a3fff700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7098050290 0x7f7098052750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.033+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f70b4068240 con 0x7f70bc151890 2020-07-18T03:34:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.033+0000 7f70bbfff700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7098050290 0x7f7098052750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:35.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.033+0000 7f70bbfff700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7098050290 0x7f7098052750 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f70ac002f30 tx=0x7f70ac006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:35.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.036+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f70b4040930 con 0x7f70bc151890 2020-07-18T03:34:35.251 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.246+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70bc1429d0 con 0x7f70bc151890 2020-07-18T03:34:35.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.246+0000 7f70a3fff700 1 -- 172.21.15.47:0/994314849 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f70b4043f50 con 0x7f70bc151890 2020-07-18T03:34:35.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7098050290 msgr2=0x7f7098052750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:35.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7098050290 0x7f7098052750 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f70ac002f30 tx=0x7f70ac006dd0).stop 2020-07-18T03:34:35.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70bc151890 msgr2=0x7f70bc15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:35.252 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70bc151890 0x7f70bc15b4b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f70b4001450 tx=0x7f70b4007c80).stop 2020-07-18T03:34:35.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 shutdown_connections 2020-07-18T03:34:35.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70bc151890 0x7f70bc15b4b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:35.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 --2- 172.21.15.47:0/994314849 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f7098050290 0x7f7098052750 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:35.253 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 >> 172.21.15.47:0/994314849 conn(0x7f70bc0b9340 msgr2=0x7f70bc1431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:35.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 shutdown_connections 2020-07-18T03:34:35.254 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:35.248+0000 7f70c2865700 1 -- 172.21.15.47:0/994314849 wait complete. 2020-07-18T03:34:35.254 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2020-07-18T03:34:35.254 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:35.254 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6639,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2078214370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2078214370},{"type":"v1","addr":"172.21.15.47:6835","nonce":2078214370}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:34:28.745915+0000","modified":"2020-07-18T03:34:30.375173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6645":{"gid":6645,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/303188627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":303188627},{"type":"v1","addr":"172.21.15.18:6833","nonce":303188627}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-07-18T03:34:35.462 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:34:28.745915+0000', 'modified': '2020-07-18T03:34:30.375173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6645}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6645': {'gid': 6645, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/303188627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 303188627}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 303188627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:34:35.463 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:34:35.464 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-07-18T03:34:35.614 INFO:teuthology.orchestra.run.smithi018.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6645,"want_state":"up:active","state":"up:active","rank_uptime":6.8575737459999999,"mdsmap_epoch":42,"osdmap_epoch":79,"osdmap_epoch_barrier":79,"uptime":12.082913148999999} 2020-07-18T03:34:35.615 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6645, "mdsmap_epoch": 42, "osdmap_epoch": 79, "osdmap_epoch_barrier": 79, "rank_uptime": 6.857573746, "state": "up:active", "uptime": 12.082913149, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:34:35.616 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:34:35.616 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:34:35.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.618 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:34:35.636 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:35.636 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:35.637 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:35.637 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:35.637 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:35.638 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:35.638 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:35.639 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:35.639 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:35.639 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:35.640 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:35.640 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42315sec preferred_lft 42315sec 2020-07-18T03:34:35.640 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:34:35.640 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:35.640 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:35.641 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:35.641 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:35.641 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:35.641 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-07-18T03:34:35.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.643 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-07-18T03:34:35.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.710 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set ceph-brx up' 2020-07-18T03:34:35.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.784 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-07-18T03:34:35.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.858 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:34:35.934 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:34:35.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.935 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:34:35.990 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:34:35.991 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:34:35.991 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:34:35.991 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:34:35.991 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:34:35.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:35.993 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:34:36.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.067 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:34:36.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.139 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:34:36.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.212 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:34:36.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.267 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:34:36.321 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 2020-07-18T03:34:36.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.322 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-07-18T03:34:36.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.395 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-07-18T03:34:36.467 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-07-18T03:34:36.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.468 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-07-18T03:34:36.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.540 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:34:36.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.643 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-07-18T03:34:36.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.701 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-07-18T03:34:36.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.754 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-07-18T03:34:36.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.801 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 up' 2020-07-18T03:34:36.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:36.852 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-07-18T03:34:36.914 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-07-18T03:34:36.914 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-07-18T03:34:36.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:36.915 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:36.938 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:34:37.011 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:34:37.012 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:34:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:34:37.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:34:37.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:37.085 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:34:37.143 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-07-18T03:34:37.144 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:37.185 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:34:37.217 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:34:37.218 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:34:37.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2020-07-18T03:34:37.281+0000 7fe6dcd0c1c0 -1 init, newargv = 0x55d1c0b80100 newargc=15 2020-07-18T03:34:37.283 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[52561]: starting ceph client 2020-07-18T03:34:37.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[52561]: starting fuse 2020-07-18T03:34:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:34:37.290 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:34:37.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:37.292 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:34:37.345 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:34:37.346 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-07-18T03:34:37.347 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:34:37.347 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:34:37.348 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:34:37.349 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:34:37.350 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:34:37.350 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:37.352 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:34:37.353 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-07-18T03:34:37.354 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:34:37.355 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-07-18T03:34:37.356 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:37.356 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:34:37.357 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:34:37.358 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:34:37.358 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:34:37.359 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:37.360 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:34:37.361 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:34:37.362 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:34:37.363 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:34:37.363 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:34:37.364 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:34:37.365 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:34:37.365 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:34:37.366 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:37.367 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2020-07-18T03:34:37.367 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:34:37.478 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.52561.asok 2020-07-18T03:34:37.658 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52561.asok 2020-07-18T03:34:37.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:37.660 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52561.asok status 2020-07-18T03:34:37.812 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:34:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:34:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:34:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:34:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2020-07-18T03:34:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:34:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-07-18T03:34:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "52561", 2020-07-18T03:34:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:34:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:34:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:34:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7177, 2020-07-18T03:34:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:34:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:34:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:34:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7177 2020-07-18T03:34:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:34:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:34:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2204386172 2020-07-18T03:34:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:34:37.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:37.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2020-07-18T03:34:37.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2204386172 2020-07-18T03:34:37.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:37.820 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.7177 192.168.0.1:0/2204386172", 2020-07-18T03:34:37.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2204386172", 2020-07-18T03:34:37.821 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:34:37.821 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 42, 2020-07-18T03:34:37.821 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 79, 2020-07-18T03:34:37.821 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:34:37.822 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:34:37.826 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:34:37.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:37.828 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:34:37.828 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:37.844 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:34:37.844 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-07-18T03:34:37.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:37.845 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:37.916 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-07-18T03:34:38.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.070+0000 7fd7042c8700 1 Processor -- start 2020-07-18T03:34:38.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.070+0000 7fd7042c8700 1 -- start start 2020-07-18T03:34:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd7042c8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b31f0 0x7fd6fc0b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd7042c8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6fc0b2620 0x7fd6fc1454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd7042c8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc0afbd0 con 0x7fd6fc0b2e10 2020-07-18T03:34:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd7042c8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc144e30 con 0x7fd6fc0b2620 2020-07-18T03:34:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd7042c8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc1491c0 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6fc0b2620 0x7fd6fc1454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6fc0b2620 0x7fd6fc1454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36142/0 (socket says 172.21.15.47:36142) 2020-07-18T03:34:38.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 -- 172.21.15.47:0/2218927129 learned_addr learned my addr 172.21.15.47:0/2218927129 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:38.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd701863700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b31f0 0x7fd6fc0b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 -- 172.21.15.47:0/2218927129 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b31f0 msgr2=0x7fd6fc0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:38.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b31f0 0x7fd6fc0b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 -- 172.21.15.47:0/2218927129 >> v1:172.21.15.18:6789/0 conn(0x7fd6fc0b2e10 legacy=0x7fd6fc0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 -- 172.21.15.47:0/2218927129 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc145050 con 0x7fd6fc0b2620 2020-07-18T03:34:38.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd701863700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b31f0 0x7fd6fc0b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:38.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd702865700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b2620 0x7fd6fc1454f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fd6ec002f30 tx=0x7fd6ec006e90).ready entity=mon.1 client_cookie=6d2ebe1b7accbeb0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.071+0000 7fd701062700 1 -- 172.21.15.47:0/2218927129 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6ec009970 con 0x7fd6fc0b2620 2020-07-18T03:34:38.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd701062700 1 -- 172.21.15.47:0/2218927129 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec009ad0 con 0x7fd6fc0b2620 2020-07-18T03:34:38.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd701062700 1 -- 172.21.15.47:0/2218927129 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6ec009e30 con 0x7fd6fc0b2620 2020-07-18T03:34:38.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 -- 172.21.15.47:0/2218927129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b2620 msgr2=0x7fd6fc1454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b2620 0x7fd6fc1454f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fd6ec002f30 tx=0x7fd6ec006e90).stop 2020-07-18T03:34:38.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 -- 172.21.15.47:0/2218927129 shutdown_connections 2020-07-18T03:34:38.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b2620 0x7fd6fc1454f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2218927129 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b31f0 0x7fd6fc0b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 -- 172.21.15.47:0/2218927129 >> 172.21.15.47:0/2218927129 conn(0x7fd6fc0b9340 msgr2=0x7fd6fc0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 -- 172.21.15.47:0/2218927129 shutdown_connections 2020-07-18T03:34:38.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.072+0000 7fd7042c8700 1 -- 172.21.15.47:0/2218927129 wait complete. 2020-07-18T03:34:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.073+0000 7fd7042c8700 1 Processor -- start 2020-07-18T03:34:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.073+0000 7fd7042c8700 1 -- start start 2020-07-18T03:34:38.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd7042c8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b2e10 0x7fd6fc144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd7042c8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6fc0b31f0 0x7fd6fc156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd7042c8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc1574e0 con 0x7fd6fc141f70 2020-07-18T03:34:38.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd7042c8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc144120 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.083 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd7042c8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc1445c0 con 0x7fd6fc0b2e10 2020-07-18T03:34:38.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6fc0b31f0 0x7fd6fc156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd6fc0b31f0 0x7fd6fc156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36148/0 (socket says 172.21.15.47:36148) 2020-07-18T03:34:38.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 -- 172.21.15.47:0/2256109181 learned_addr learned my addr 172.21.15.47:0/2256109181 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:38.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd702064700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b2e10 0x7fd6fc144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.084 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 -- 172.21.15.47:0/2256109181 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b2e10 msgr2=0x7fd6fc144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b2e10 0x7fd6fc144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 -- 172.21.15.47:0/2256109181 >> v1:172.21.15.47:6789/0 conn(0x7fd6fc141f70 legacy=0x7fd6fc13fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 -- 172.21.15.47:0/2256109181 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc156f00 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd701863700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b31f0 0x7fd6fc156a10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fd6f0006920 tx=0x7fd6f0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.085 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.074+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6f0009980 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.075+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f0009ae0 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.075+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc1571b0 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.075+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6f0009e40 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.075+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc1405c0 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.086 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.076+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc057450 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.076+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd6f0013480 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.076+0000 7fd6fa7fc700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d404ffe0 0x7fd6d40524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.076+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd6f0068370 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.076+0000 7fd702064700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d404ffe0 0x7fd6d40524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.087 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.077+0000 7fd702064700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d404ffe0 0x7fd6d40524a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd6f400d000 tx=0x7fd6f4007ef0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.088 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.080+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd6f0040aa0 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.238+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6fc066b60 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.239+0000 7fd6fa7fc700 1 -- 172.21.15.47:0/2256109181 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fd6f00440c0 con 0x7fd6fc0b31f0 2020-07-18T03:34:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","created":"2020-07-18T03:25:24.387359+0000","modified":"2020-07-18T03:34:29.747072+0000","last_up_change":"2020-07-18T03:25:33.482553+0000","last_in_change":"2020-07-18T03:25:33.482553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-07-18T03:25:27.725131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-07-18T03:25:37.896092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-07-18T03:34:26.480589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-07-18T03:34:27.720390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6e1e33a-c51a-4404-a9b3-1cb18c81fc2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":23865},{"type":"v1","addr":"172.21.15.47:6821","nonce":23865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":23865},{"type":"v1","addr":"172.21.15.47:6825","nonce":23865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6831","nonce":23865},{"type":"v1","addr":"172.21.15.47:6832","nonce":23865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6827","nonce":23865},{"type":"v1","addr":"172.21.15.47:6829","nonce":23865}]},"public_addr":"172.21.15.47:6821/23865","cluster_addr":"172.21.15.47:6825/23865","heartbeat_back_addr":"172.21.15.47:6832/23865","heartbeat_front_addr":"172.21.15.47:6829/23865","state":["exists","up"]},{"osd":1,"uuid":"ec8353f5-7ad4-45bf-b8e5-99115dd53e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":23853},{"type":"v1","addr":"172.21.15.47:6803","nonce":23853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":23853},{"type":"v1","addr":"172.21.15.47:6805","nonce":23853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":23853},{"type":"v1","addr":"172.21.15.47:6809","nonce":23853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":23853},{"type":"v1","addr":"172.21.15.47:6807","nonce":23853}]},"public_addr":"172.21.15.47:6803/23853","cluster_addr":"172.21.15.47:6805/23853","heartbeat_back_addr":"172.21.15.47:6809/23853","heartbeat_front_addr":"172.21.15.47:6807/23853","state":["exists","up"]},{"osd":2,"uuid":"dc0b1a97-efca-483d-839c-37c286699283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":23872},{"type":"v1","addr":"172.21.15.47:6819","nonce":23872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":23872},{"type":"v1","addr":"172.21.15.47:6824","nonce":23872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":23872},{"type":"v1","addr":"172.21.15.47:6833","nonce":23872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":23872},{"type":"v1","addr":"172.21.15.47:6828","nonce":23872}]},"public_addr":"172.21.15.47:6819/23872","cluster_addr":"172.21.15.47:6824/23872","heartbeat_back_addr":"172.21.15.47:6833/23872","heartbeat_front_addr":"172.21.15.47:6828/23872","state":["exists","up"]},{"osd":3,"uuid":"3f949e90-b2a0-4ef4-a63b-90f457a32c86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":23854},{"type":"v1","addr":"172.21.15.47:6811","nonce":23854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":23854},{"type":"v1","addr":"172.21.15.47:6813","nonce":23854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":23854},{"type":"v1","addr":"172.21.15.47:6817","nonce":23854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":23854},{"type":"v1","addr":"172.21.15.47:6815","nonce":23854}]},"public_addr":"172.21.15.47:6811/23854","cluster_addr":"172.21.15.47:6813/23854","heartbeat_back_addr":"172.21.15.47:6817/23854","heartbeat_front_addr":"172.21.15.47:6815/23854","state":["exists","up"]},{"osd":4,"uuid":"83234bc7-02d5-405a-af0d-2c30c2f80681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":24092},{"type":"v1","addr":"172.21.15.18:6809","nonce":24092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":24092},{"type":"v1","addr":"172.21.15.18:6811","nonce":24092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":24092},{"type":"v1","addr":"172.21.15.18:6815","nonce":24092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":24092},{"type":"v1","addr":"172.21.15.18:6813","nonce":24092}]},"public_addr":"172.21.15.18:6809/24092","cluster_addr":"172.21.15.18:6811/24092","heartbeat_back_addr":"172.21.15.18:6815/24092","heartbeat_front_addr":"172.21.15.18:6813/24092","state":["exists","up"]},{"osd":5,"uuid":"a8bc62ad-b199-4f39-bbd5-10a71cd953c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":24091},{"type":"v1","addr":"172.21.15.18:6801","nonce":24091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":24091},{"type":"v1","addr":"172.21.15.18:6803","nonce":24091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":24091},{"type":"v1","addr":"172.21.15.18:6807","nonce":24091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":24091},{"type":"v1","addr":"172.21.15.18:6805","nonce":24091}]},"public_addr":"172.21.15.18:6801/24091","cluster_addr":"172.21.15.18:6803/24091","heartbeat_back_addr":"172.21.15.18:6807/24091","heartbeat_front_addr":"172.21.15.18:6805/24091","state":["exists","up"]},{"osd":6,"uuid":"3561e1c4-c30d-462b-a364-ae0be63149c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":24093},{"type":"v1","addr":"172.21.15.18:6817","nonce":24093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":24093},{"type":"v1","addr":"172.21.15.18:6821","nonce":24093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":24093},{"type":"v1","addr":"172.21.15.18:6830","nonce":24093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":24093},{"type":"v1","addr":"172.21.15.18:6826","nonce":24093}]},"public_addr":"172.21.15.18:6817/24093","cluster_addr":"172.21.15.18:6821/24093","heartbeat_back_addr":"172.21.15.18:6830/24093","heartbeat_front_addr":"172.21.15.18:6826/24093","state":["exists","up"]},{"osd":7,"uuid":"cb66e979-d9f3-40eb-946a-20ccb8c8002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":24094},{"type":"v1","addr":"172.21.15.18:6820","nonce":24094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":24094},{"type":"v1","addr":"172.21.15.18:6824","nonce":24094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6829","nonce":24094},{"type":"v1","addr":"172.21.15.18:6831","nonce":24094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6825","nonce":24094},{"type":"v1","addr":"172.21.15.18:6827","nonce":24094}]},"public_addr":"172.21.15.18:6820/24094","cluster_addr":"172.21.15.18:6824/24094","heartbeat_back_addr":"172.21.15.18:6831/24094","heartbeat_front_addr":"172.21.15.18:6827/24094","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.933644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.929073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.974285+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.909674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.935590+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.916457+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.990672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-07-18T03:25:31.948777+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-07-18T03:34:38.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d404ffe0 msgr2=0x7fd6d40524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d404ffe0 0x7fd6d40524a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd6f400d000 tx=0x7fd6f4007ef0).stop 2020-07-18T03:34:38.244 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b31f0 msgr2=0x7fd6fc156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b31f0 0x7fd6fc156a10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fd6f0006920 tx=0x7fd6f0008040).stop 2020-07-18T03:34:38.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 shutdown_connections 2020-07-18T03:34:38.245 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd6d404ffe0 0x7fd6d40524a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.247 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd6fc0b2e10 0x7fd6fc144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 --2- 172.21.15.47:0/2256109181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6fc0b31f0 0x7fd6fc156a10 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 >> 172.21.15.47:0/2256109181 conn(0x7fd6fc0b9340 msgr2=0x7fd6fc0b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.248 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 shutdown_connections 2020-07-18T03:34:38.249 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.241+0000 7fd7042c8700 1 -- 172.21.15.47:0/2256109181 wait complete. 2020-07-18T03:34:38.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-07-18T03:25:27.725131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-07-18T03:25:37.896092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-07-18T03:34:26.480589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-07-18T03:34:27.720390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-07-18T03:34:38.261 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-07-18T03:34:38.379 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.378+0000 7f151385b700 1 Processor -- start 2020-07-18T03:34:38.380 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.378+0000 7f151385b700 1 -- start start 2020-07-18T03:34:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f151385b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f150c0647a0 0x7f150c0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f151385b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f150c0b7ff0 0x7f150c12cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f151385b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f150c12d1f0 0x7f150c148690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f151385b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c12dff0 con 0x7f150c12d1f0 2020-07-18T03:34:38.388 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f151385b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c12e290 con 0x7f150c0647a0 2020-07-18T03:34:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f151385b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c12e530 con 0x7f150c0b7ff0 2020-07-18T03:34:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f150c0647a0 0x7f150c0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f150c0647a0 0x7f150c0b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36152/0 (socket says 172.21.15.47:36152) 2020-07-18T03:34:38.393 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 -- 172.21.15.47:0/1768731060 learned_addr learned my addr 172.21.15.47:0/1768731060 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f1510df6700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f150c0b7ff0 0x7f150c12cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f1511df8700 1 --2- 172.21.15.47:0/1768731060 >> v2:172.21.15.18:3300/0 conn(0x7f150c12d1f0 0x7f150c148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 -- 172.21.15.47:0/1768731060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f150c0b7ff0 msgr2=0x7f150c12cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f150c0b7ff0 0x7f150c12cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 -- 172.21.15.47:0/1768731060 >> v2:172.21.15.18:3300/0 conn(0x7f150c12d1f0 msgr2=0x7f150c148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 --2- 172.21.15.47:0/1768731060 >> v2:172.21.15.18:3300/0 conn(0x7f150c12d1f0 0x7f150c148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 -- 172.21.15.47:0/1768731060 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c064b80 con 0x7f150c0647a0 2020-07-18T03:34:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f1510df6700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f150c0b7ff0 0x7f150c12cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f1511df8700 1 --2- 172.21.15.47:0/1768731060 >> v2:172.21.15.18:3300/0 conn(0x7f150c12d1f0 0x7f150c148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-07-18T03:34:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.379+0000 7f15115f7700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c0647a0 0x7f150c0b5b90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f14fc002f30 tx=0x7f14fc006e90).ready entity=mon.1 client_cookie=a5278597f6cc1c89 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f14fbfff700 1 -- 172.21.15.47:0/1768731060 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14fc009920 con 0x7f150c0647a0 2020-07-18T03:34:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f14fbfff700 1 -- 172.21.15.47:0/1768731060 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc009a80 con 0x7f150c0647a0 2020-07-18T03:34:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f14fbfff700 1 -- 172.21.15.47:0/1768731060 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14fc009de0 con 0x7f150c0647a0 2020-07-18T03:34:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 -- 172.21.15.47:0/1768731060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c0647a0 msgr2=0x7f150c0b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c0647a0 0x7f150c0b5b90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f14fc002f30 tx=0x7f14fc006e90).stop 2020-07-18T03:34:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 -- 172.21.15.47:0/1768731060 shutdown_connections 2020-07-18T03:34:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 --2- 172.21.15.47:0/1768731060 >> v2:172.21.15.18:3300/0 conn(0x7f150c12d1f0 0x7f150c148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c0647a0 0x7f150c0b5b90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 --2- 172.21.15.47:0/1768731060 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f150c0b7ff0 0x7f150c12cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 -- 172.21.15.47:0/1768731060 >> 172.21.15.47:0/1768731060 conn(0x7f150c05f5a0 msgr2=0x7f150c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 -- 172.21.15.47:0/1768731060 shutdown_connections 2020-07-18T03:34:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.380+0000 7f151385b700 1 -- 172.21.15.47:0/1768731060 wait complete. 2020-07-18T03:34:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.381+0000 7f151385b700 1 Processor -- start 2020-07-18T03:34:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.381+0000 7f151385b700 1 -- start start 2020-07-18T03:34:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f151385b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f150c12d1f0 0x7f150c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f151385b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f150c143dd0 0x7f150c144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f151385b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c1576c0 con 0x7f150c0b7ff0 2020-07-18T03:34:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f151385b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0b5330 con 0x7f150c143dd0 2020-07-18T03:34:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f151385b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0b55d0 con 0x7f150c12d1f0 2020-07-18T03:34:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f1510df6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f150c12d1f0 0x7f150c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f1510df6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f150c12d1f0 0x7f150c1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36158/0 (socket says 172.21.15.47:36158) 2020-07-18T03:34:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.382+0000 7f1510df6700 1 -- 172.21.15.47:0/2675156051 learned_addr learned my addr 172.21.15.47:0/2675156051 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f1510df6700 1 -- 172.21.15.47:0/2675156051 >> v2:172.21.15.18:3300/0 conn(0x7f150c143dd0 msgr2=0x7f150c144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f1510df6700 1 --2- 172.21.15.47:0/2675156051 >> v2:172.21.15.18:3300/0 conn(0x7f150c143dd0 0x7f150c144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-07-18T03:34:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f1510df6700 1 -- 172.21.15.47:0/2675156051 >> v1:172.21.15.47:6789/0 conn(0x7f150c0b7ff0 legacy=0x7f150c146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f1510df6700 1 -- 172.21.15.47:0/2675156051 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0b26d0 con 0x7f150c12d1f0 2020-07-18T03:34:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f1510df6700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c12d1f0 0x7f150c1571d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f1508006920 tx=0x7f1508008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1508009980 con 0x7f150c12d1f0 2020-07-18T03:34:38.401 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508009ae0 con 0x7f150c12d1f0 2020-07-18T03:34:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c144730 con 0x7f150c12d1f0 2020-07-18T03:34:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1508009e40 con 0x7f150c12d1f0 2020-07-18T03:34:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.383+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c0b2b70 con 0x7f150c12d1f0 2020-07-18T03:34:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.384+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c0b5170 con 0x7f150c12d1f0 2020-07-18T03:34:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.384+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1508013480 con 0x7f150c12d1f0 2020-07-18T03:34:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.384+0000 7f14f97fa700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f14e4050010 0x7f14e40524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.384+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f15080685c0 con 0x7f150c12d1f0 2020-07-18T03:34:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.384+0000 7f15115f7700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f14e4050010 0x7f14e40524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.385+0000 7f15115f7700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f14e4050010 0x7f14e40524d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f150c1475c0 tx=0x7f14fc015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.388+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1508040cf0 con 0x7f150c12d1f0 2020-07-18T03:34:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.546+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f150c0bbb00 con 0x7f150c12d1f0 2020-07-18T03:34:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.546+0000 7f14f97fa700 1 -- 172.21.15.47:0/2675156051 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v79) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1508044310 con 0x7f150c12d1f0 2020-07-18T03:34:38.548 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2020-07-18T03:34:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.547+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f14e4050010 msgr2=0x7f14e40524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.547+0000 7f151385b700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f14e4050010 0x7f14e40524d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f150c1475c0 tx=0x7f14fc015000).stop 2020-07-18T03:34:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.547+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c12d1f0 msgr2=0x7f150c1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.547+0000 7f151385b700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c12d1f0 0x7f150c1571d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f1508006920 tx=0x7f1508008040).stop 2020-07-18T03:34:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 shutdown_connections 2020-07-18T03:34:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 --2- 172.21.15.47:0/2675156051 >> v2:172.21.15.18:3300/0 conn(0x7f150c143dd0 0x7f150c144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-07-18T03:34:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f14e4050010 0x7f14e40524d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 --2- 172.21.15.47:0/2675156051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f150c12d1f0 0x7f150c1571d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 >> 172.21.15.47:0/2675156051 conn(0x7f150c05f5a0 msgr2=0x7f150c12cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 shutdown_connections 2020-07-18T03:34:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.548+0000 7f151385b700 1 -- 172.21.15.47:0/2675156051 wait complete. 2020-07-18T03:34:38.559 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-07-18T03:34:38.696 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.682+0000 7fc220f0c700 1 Processor -- start 2020-07-18T03:34:38.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.682+0000 7fc220f0c700 1 -- start start 2020-07-18T03:34:38.698 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.683+0000 7fc220f0c700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc21c144e40 0x7fc21c063e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.683+0000 7fc220f0c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc21c148b20 0x7fc21c148f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.683+0000 7fc220f0c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c145550 con 0x7fc21c0b7fa0 2020-07-18T03:34:38.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.683+0000 7fc220f0c700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c1457f0 con 0x7fc21c144e40 2020-07-18T03:34:38.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.683+0000 7fc220f0c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c145a90 con 0x7fc21c148b20 2020-07-18T03:34:38.699 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.683+0000 7fc219d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc21c148b20 0x7fc21c148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc21c148b20 0x7fc21c148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36166/0 (socket says 172.21.15.47:36166) 2020-07-18T03:34:38.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 -- 172.21.15.47:0/3799728732 learned_addr learned my addr 172.21.15.47:0/3799728732 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:38.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc21a59c700 1 --2- 172.21.15.47:0/3799728732 >> v2:172.21.15.18:3300/0 conn(0x7fc21c144e40 0x7fc21c063e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.700 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 -- 172.21.15.47:0/3799728732 >> v2:172.21.15.18:3300/0 conn(0x7fc21c144e40 msgr2=0x7fc21c063e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:38.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 --2- 172.21.15.47:0/3799728732 >> v2:172.21.15.18:3300/0 conn(0x7fc21c144e40 0x7fc21c063e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 -- 172.21.15.47:0/3799728732 >> v1:172.21.15.47:6789/0 conn(0x7fc21c0b7fa0 legacy=0x7fc21c0b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 -- 172.21.15.47:0/3799728732 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c145220 con 0x7fc21c148b20 2020-07-18T03:34:38.701 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc21a59c700 1 --2- 172.21.15.47:0/3799728732 >> v2:172.21.15.18:3300/0 conn(0x7fc21c144e40 0x7fc21c063e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc219d9b700 1 --2- 172.21.15.47:0/3799728732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc21c148b20 0x7fc21c148f80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7fc20c002f30 tx=0x7fc20c006e90).ready entity=mon.1 client_cookie=e320ad6be033deaf server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc21959a700 1 -- 172.21.15.47:0/3799728732 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20c0099c0 con 0x7fc21c148b20 2020-07-18T03:34:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc21959a700 1 -- 172.21.15.47:0/3799728732 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c009b20 con 0x7fc21c148b20 2020-07-18T03:34:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc21959a700 1 -- 172.21.15.47:0/3799728732 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20c009e80 con 0x7fc21c148b20 2020-07-18T03:34:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 -- 172.21.15.47:0/3799728732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc21c148b20 msgr2=0x7fc21c148f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 --2- 172.21.15.47:0/3799728732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc21c148b20 0x7fc21c148f80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7fc20c002f30 tx=0x7fc20c006e90).stop 2020-07-18T03:34:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 -- 172.21.15.47:0/3799728732 shutdown_connections 2020-07-18T03:34:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 --2- 172.21.15.47:0/3799728732 >> v2:172.21.15.18:3300/0 conn(0x7fc21c144e40 0x7fc21c063e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 --2- 172.21.15.47:0/3799728732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc21c148b20 0x7fc21c148f80 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 -- 172.21.15.47:0/3799728732 >> 172.21.15.47:0/3799728732 conn(0x7fc21c05f5a0 msgr2=0x7fc21c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 -- 172.21.15.47:0/3799728732 shutdown_connections 2020-07-18T03:34:38.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.684+0000 7fc220f0c700 1 -- 172.21.15.47:0/3799728732 wait complete. 2020-07-18T03:34:38.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.686+0000 7fc220f0c700 1 Processor -- start 2020-07-18T03:34:38.704 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.686+0000 7fc220f0c700 1 -- start start 2020-07-18T03:34:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.686+0000 7fc220f0c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc21c0b7fa0 0x7fc21c13bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.686+0000 7fc220f0c700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c146ef0 con 0x7fc21c144e40 2020-07-18T03:34:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.686+0000 7fc220f0c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c147190 con 0x7fc21c145e50 2020-07-18T03:34:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.686+0000 7fc220f0c700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c147ed0 con 0x7fc21c0b7fa0 2020-07-18T03:34:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc21ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc21c145e50 0x7fc21c13feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45494/0 (socket says 172.21.15.47:45494) 2020-07-18T03:34:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc21ad9d700 1 -- 172.21.15.47:0/993222423 learned_addr learned my addr 172.21.15.47:0/993222423 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc21a59c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc21c0b7fa0 0x7fc21c13bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.707 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182782543 0 0) 0x7fc21c147190 con 0x7fc21c145e50 2020-07-18T03:34:38.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f8003240 con 0x7fc21c145e50 2020-07-18T03:34:38.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305311479 0 0) 0x7fc21c146ef0 con 0x7fc21c144e40 2020-07-18T03:34:38.708 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc21c147190 con 0x7fc21c144e40 2020-07-18T03:34:38.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1246801580 0 0) 0x7fc1f8003240 con 0x7fc21c145e50 2020-07-18T03:34:38.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc21c146ef0 con 0x7fc21c145e50 2020-07-18T03:34:38.709 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7fc214001c30 con 0x7fc21c145e50 2020-07-18T03:34:38.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc21c0b7fa0 msgr2=0x7fc21c13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.710 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc21c0b7fa0 0x7fc21c13bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 >> v1:172.21.15.47:6789/0 conn(0x7fc21c145e50 legacy=0x7fc21c13feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 >> v1:172.21.15.18:6789/0 conn(0x7fc21c144e40 legacy=0x7fc21c135530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f8003970 0x7fc1f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f80044e0 0x7fc1f8004920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.711 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1f8004e10 0x7fc1f800b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c147ed0 con 0x7fc1f8004e10 2020-07-18T03:34:38.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f800bc20 con 0x7fc1f8003970 2020-07-18T03:34:38.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f800bde0 con 0x7fc1f80044e0 2020-07-18T03:34:38.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc21a59c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc21c0b7fa0 0x7fc21c13bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:38.712 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc21a59c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f8003970 0x7fc1f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc219d9b700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f80044e0 0x7fc1f8004920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.687+0000 7fc21ad9d700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1f8004e10 0x7fc1f800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc21a59c700 1 -- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f80044e0 msgr2=0x7fc1f8004920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.713 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc21a59c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc1f80044e0 0x7fc1f8004920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc21a59c700 1 -- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1f8004e10 msgr2=0x7fc1f800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc21a59c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1f8004e10 0x7fc1f800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc21a59c700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c13c2c0 con 0x7fc1f8003970 2020-07-18T03:34:38.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c135c40 con 0x7fc1f8003970 2020-07-18T03:34:38.714 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c13fc20 con 0x7fc1f8003970 2020-07-18T03:34:38.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc21a59c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f8003970 0x7fc1f8003d90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7fc1f80054b0 tx=0x7fc204008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20400a330 con 0x7fc1f8003970 2020-07-18T03:34:38.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20400c040 con 0x7fc1f8003970 2020-07-18T03:34:38.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20400da80 con 0x7fc1f8003970 2020-07-18T03:34:38.715 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.688+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c0600e0 con 0x7fc1f8003970 2020-07-18T03:34:38.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.689+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc20400dbe0 con 0x7fc1f8003970 2020-07-18T03:34:38.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.689+0000 7fc2127fc700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1f8058ad0 0x7fc1f805af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.689+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc204068040 con 0x7fc1f8003970 2020-07-18T03:34:38.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.689+0000 7fc219d9b700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1f8058ad0 0x7fc1f805af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.716 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.690+0000 7fc219d9b700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1f8058ad0 0x7fc1f805af90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc20c01a000 tx=0x7fc20c015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.717 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.693+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc204040750 con 0x7fc1f8003970 2020-07-18T03:34:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.850+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc21c05b120 con 0x7fc1f8003970 2020-07-18T03:34:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.850+0000 7fc2127fc700 1 -- 172.21.15.47:0/993222423 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc204017020 con 0x7fc1f8003970 2020-07-18T03:34:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1f8058ad0 msgr2=0x7fc1f805af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc1f8058ad0 0x7fc1f805af90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc20c01a000 tx=0x7fc20c015000).stop 2020-07-18T03:34:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f8003970 msgr2=0x7fc1f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f8003970 0x7fc1f8003d90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7fc1f80054b0 tx=0x7fc204008040).stop 2020-07-18T03:34:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc21ad9d700 1 -- 172.21.15.47:0/993222423 reap_dead start 2020-07-18T03:34:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 shutdown_connections 2020-07-18T03:34:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 --2- 172.21.15.47:0/993222423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f8003970 0x7fc1f8003d90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 >> 172.21.15.47:0/993222423 conn(0x7fc21c05f5a0 msgr2=0x7fc21c1386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 shutdown_connections 2020-07-18T03:34:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.852+0000 7fc220f0c700 1 -- 172.21.15.47:0/993222423 wait complete. 2020-07-18T03:34:38.856 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:38.863 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-07-18T03:34:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.980+0000 7f1d97acf700 1 Processor -- start 2020-07-18T03:34:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.980+0000 7f1d97acf700 1 -- start start 2020-07-18T03:34:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.980+0000 7f1d97acf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d90142a20 0x7f1d90142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.980+0000 7f1d97acf700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900af4b0 con 0x7f1d90061c60 2020-07-18T03:34:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900b6d80 con 0x7f1d900b3af0 2020-07-18T03:34:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900af710 con 0x7f1d90142a20 2020-07-18T03:34:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d90142a20 0x7f1d90142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d90142a20 0x7f1d90142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36184/0 (socket says 172.21.15.47:36184) 2020-07-18T03:34:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 -- 172.21.15.47:0/1939683683 learned_addr learned my addr 172.21.15.47:0/1939683683 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9606c700 1 --1- 172.21.15.47:0/1939683683 >> v1:172.21.15.47:6789/0 conn(0x7f1d900b3af0 0x7f1d900b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45508/0 (socket says 172.21.15.47:45508) 2020-07-18T03:34:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 -- 172.21.15.47:0/1939683683 >> v1:172.21.15.47:6789/0 conn(0x7f1d900b3af0 legacy=0x7f1d900b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 -- 172.21.15.47:0/1939683683 >> v1:172.21.15.18:6789/0 conn(0x7f1d90061c60 legacy=0x7f1d900620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 -- 172.21.15.47:0/1939683683 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900af9b0 con 0x7f1d90142a20 2020-07-18T03:34:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d9586b700 1 --2- 172.21.15.47:0/1939683683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90142a20 0x7f1d90142e00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f1d80002f30 tx=0x7f1d80006e90).ready entity=mon.1 client_cookie=fcff69bd54f45eed server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d94869700 1 -- 172.21.15.47:0/1939683683 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d80009a10 con 0x7f1d90142a20 2020-07-18T03:34:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d94869700 1 -- 172.21.15.47:0/1939683683 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80009b70 con 0x7f1d90142a20 2020-07-18T03:34:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d94869700 1 -- 172.21.15.47:0/1939683683 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d80011400 con 0x7f1d90142a20 2020-07-18T03:34:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- 172.21.15.47:0/1939683683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90142a20 msgr2=0x7f1d90142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 --2- 172.21.15.47:0/1939683683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90142a20 0x7f1d90142e00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f1d80002f30 tx=0x7f1d80006e90).stop 2020-07-18T03:34:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- 172.21.15.47:0/1939683683 shutdown_connections 2020-07-18T03:34:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 --2- 172.21.15.47:0/1939683683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90142a20 0x7f1d90142e00 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- 172.21.15.47:0/1939683683 >> 172.21.15.47:0/1939683683 conn(0x7f1d9005f5a0 msgr2=0x7f1d9005f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- 172.21.15.47:0/1939683683 shutdown_connections 2020-07-18T03:34:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.981+0000 7f1d97acf700 1 -- 172.21.15.47:0/1939683683 wait complete. 2020-07-18T03:34:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.983+0000 7f1d97acf700 1 Processor -- start 2020-07-18T03:34:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.983+0000 7f1d97acf700 1 -- start start 2020-07-18T03:34:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.983+0000 7f1d97acf700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1d90061c60 0x7f1d90155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d97acf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d90145170 0x7f1d900b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d97acf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d90155c90 con 0x7f1d900b3af0 2020-07-18T03:34:38.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d97acf700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d90155f30 con 0x7f1d90061c60 2020-07-18T03:34:38.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d97acf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d901561d0 con 0x7f1d90145170 2020-07-18T03:34:38.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9506a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d900b3af0 0x7f1d901595b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45512/0 (socket says 172.21.15.47:45512) 2020-07-18T03:34:38.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9506a700 1 -- 172.21.15.47:0/2110649437 learned_addr learned my addr 172.21.15.47:0/2110649437 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:38.992 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9606c700 1 --2- 172.21.15.47:0/2110649437 >> v2:172.21.15.47:3300/0 conn(0x7f1d90145170 0x7f1d900b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9586b700 1 --2- 172.21.15.47:0/2110649437 >> v2:172.21.15.18:3300/0 conn(0x7f1d90061c60 0x7f1d90155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9606c700 1 -- 172.21.15.47:0/2110649437 >> v2:172.21.15.18:3300/0 conn(0x7f1d90061c60 msgr2=0x7f1d90155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9606c700 1 --2- 172.21.15.47:0/2110649437 >> v2:172.21.15.18:3300/0 conn(0x7f1d90061c60 0x7f1d90155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:38.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9606c700 1 -- 172.21.15.47:0/2110649437 >> v1:172.21.15.47:6789/0 conn(0x7f1d900b3af0 legacy=0x7f1d901595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:38.993 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9606c700 1 -- 172.21.15.47:0/2110649437 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90064910 con 0x7f1d90145170 2020-07-18T03:34:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518461061 0 0) 0x7f1d90155c90 con 0x7f1d900b3af0 2020-07-18T03:34:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9586b700 1 --2- 172.21.15.47:0/2110649437 >> v2:172.21.15.18:3300/0 conn(0x7f1d90061c60 0x7f1d90155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-07-18T03:34:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d9606c700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90145170 0x7f1d900b9520 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f1d900b6f10 tx=0x7f1d88007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d88006320 con 0x7f1d90145170 2020-07-18T03:34:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88006480 con 0x7f1d90145170 2020-07-18T03:34:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90064b00 con 0x7f1d90145170 2020-07-18T03:34:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d88004860 con 0x7f1d90145170 2020-07-18T03:34:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.984+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d900b9c50 con 0x7f1d90145170 2020-07-18T03:34:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.985+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d90143010 con 0x7f1d90145170 2020-07-18T03:34:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.985+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1d88010030 con 0x7f1d90145170 2020-07-18T03:34:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.986+0000 7f1d7dffb700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6804fdd0 0x7f1d68052290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.986+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d88067b10 con 0x7f1d90145170 2020-07-18T03:34:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.986+0000 7f1d9586b700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6804fdd0 0x7f1d68052290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.986+0000 7f1d9586b700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6804fdd0 0x7f1d68052290 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f1d80017000 tx=0x7f1d80015000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:38.989+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1d88040240 con 0x7f1d90145170 2020-07-18T03:34:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.150+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d90156370 con 0x7f1d90145170 2020-07-18T03:34:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.151+0000 7f1d7dffb700 1 -- 172.21.15.47:0/2110649437 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1d88043860 con 0x7f1d90145170 2020-07-18T03:34:39.152 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6804fdd0 msgr2=0x7f1d68052290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6804fdd0 0x7f1d68052290 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f1d80017000 tx=0x7f1d80015000).stop 2020-07-18T03:34:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90145170 msgr2=0x7f1d900b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90145170 0x7f1d900b9520 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f1d900b6f10 tx=0x7f1d88007dd0).stop 2020-07-18T03:34:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 shutdown_connections 2020-07-18T03:34:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d90145170 0x7f1d900b9520 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 --2- 172.21.15.47:0/2110649437 >> v2:172.21.15.18:3300/0 conn(0x7f1d90061c60 0x7f1d90155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 --2- 172.21.15.47:0/2110649437 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d6804fdd0 0x7f1d68052290 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 >> 172.21.15.47:0/2110649437 conn(0x7f1d9005f5a0 msgr2=0x7f1d90146a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 shutdown_connections 2020-07-18T03:34:39.156 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.152+0000 7f1d97acf700 1 -- 172.21.15.47:0/2110649437 wait complete. 2020-07-18T03:34:39.164 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-07-18T03:34:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 Processor -- start 2020-07-18T03:34:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 -- start start 2020-07-18T03:34:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c142a20 0x7fd88c142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd88c061c60 0x7fd88c0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c0af4b0 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0b6d80 con 0x7fd88c061c60 2020-07-18T03:34:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.282+0000 7fd89448b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0af710 con 0x7fd88c142a20 2020-07-18T03:34:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892a28700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd88c0b5dc0 0x7fd88c146ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45518/0 (socket says 172.21.15.47:45518) 2020-07-18T03:34:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892a28700 1 -- 172.21.15.47:0/123773265 learned_addr learned my addr 172.21.15.47:0/123773265 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892227700 1 --2- 172.21.15.47:0/123773265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c142a20 0x7fd88c142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd891a26700 1 --2- 172.21.15.47:0/123773265 >> v2:172.21.15.18:3300/0 conn(0x7fd88c061c60 0x7fd88c0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd891225700 1 -- 172.21.15.47:0/123773265 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190597550 0 0) 0x7fd88c0af4b0 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd891225700 1 -- 172.21.15.47:0/123773265 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd878003240 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892227700 1 -- 172.21.15.47:0/123773265 >> v2:172.21.15.18:3300/0 conn(0x7fd88c061c60 msgr2=0x7fd88c0620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892227700 1 --2- 172.21.15.47:0/123773265 >> v2:172.21.15.18:3300/0 conn(0x7fd88c061c60 0x7fd88c0620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892227700 1 -- 172.21.15.47:0/123773265 >> v1:172.21.15.47:6789/0 conn(0x7fd88c0b5dc0 legacy=0x7fd88c146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892227700 1 -- 172.21.15.47:0/123773265 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0af9b0 con 0x7fd88c142a20 2020-07-18T03:34:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd892227700 1 --2- 172.21.15.47:0/123773265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c142a20 0x7fd88c142e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fd8840032b0 tx=0x7fd8840091d0).ready entity=mon.2 client_cookie=3983242b3b6b755e server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd891225700 1 -- 172.21.15.47:0/123773265 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd88400b940 con 0x7fd88c142a20 2020-07-18T03:34:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd891225700 1 -- 172.21.15.47:0/123773265 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88400baa0 con 0x7fd88c142a20 2020-07-18T03:34:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd891225700 1 -- 172.21.15.47:0/123773265 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd88400bc10 con 0x7fd88c142a20 2020-07-18T03:34:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd89448b700 1 -- 172.21.15.47:0/123773265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c142a20 msgr2=0x7fd88c142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.283+0000 7fd89448b700 1 --2- 172.21.15.47:0/123773265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c142a20 0x7fd88c142e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fd8840032b0 tx=0x7fd8840091d0).stop 2020-07-18T03:34:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.284+0000 7fd89448b700 1 -- 172.21.15.47:0/123773265 shutdown_connections 2020-07-18T03:34:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.284+0000 7fd89448b700 1 --2- 172.21.15.47:0/123773265 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c142a20 0x7fd88c142e00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.284+0000 7fd89448b700 1 --2- 172.21.15.47:0/123773265 >> v2:172.21.15.18:3300/0 conn(0x7fd88c061c60 0x7fd88c0620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.284+0000 7fd89448b700 1 -- 172.21.15.47:0/123773265 >> 172.21.15.47:0/123773265 conn(0x7fd88c05f5a0 msgr2=0x7fd88c05f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.284+0000 7fd89448b700 1 -- 172.21.15.47:0/123773265 shutdown_connections 2020-07-18T03:34:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.284+0000 7fd89448b700 1 -- 172.21.15.47:0/123773265 wait complete. 2020-07-18T03:34:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.285+0000 7fd89448b700 1 Processor -- start 2020-07-18T03:34:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.285+0000 7fd89448b700 1 -- start start 2020-07-18T03:34:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c061c60 0x7fd88c0636d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd88c0b5dc0 0x7fd88c140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd88c158470 0x7fd88c13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c064830 con 0x7fd88c158470 2020-07-18T03:34:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c063bc0 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c063e50 con 0x7fd88c061c60 2020-07-18T03:34:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd88c0b5dc0 0x7fd88c140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd88c0b5dc0 0x7fd88c140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36208/0 (socket says 172.21.15.47:36208) 2020-07-18T03:34:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 -- 172.21.15.47:0/1100886595 learned_addr learned my addr 172.21.15.47:0/1100886595 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:39.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd892227700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c061c60 0x7fd88c0636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd892a28700 1 --2- 172.21.15.47:0/1100886595 >> v2:172.21.15.18:3300/0 conn(0x7fd88c158470 0x7fd88c13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 -- 172.21.15.47:0/1100886595 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c061c60 msgr2=0x7fd88c0636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c061c60 0x7fd88c0636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.311 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 -- 172.21.15.47:0/1100886595 >> v2:172.21.15.18:3300/0 conn(0x7fd88c158470 msgr2=0x7fd88c13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 --2- 172.21.15.47:0/1100886595 >> v2:172.21.15.18:3300/0 conn(0x7fd88c158470 0x7fd88c13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 -- 172.21.15.47:0/1100886595 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0640f0 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd891a26700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd88c0b5dc0 0x7fd88c140290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fd88000c000 tx=0x7fd880006dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd88000ca10 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000cb70 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c064390 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8800123f0 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.286+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c13f460 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.287+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c064520 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.288+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fd8800125e0 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.288+0000 7fd88a7fc700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd864050030 0x7fd8640524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.288+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd880068130 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.288+0000 7fd892227700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd864050030 0x7fd8640524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.288+0000 7fd892227700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd864050030 0x7fd8640524f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd88401c000 tx=0x7fd884016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.291+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fd880040860 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.448+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd88c05b120 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.449 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.448+0000 7fd88a7fc700 1 -- 172.21.15.47:0/1100886595 <== mon.1 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd880043e80 con 0x7fd88c0b5dc0 2020-07-18T03:34:39.450 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2020-07-18T03:34:39.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd864050030 msgr2=0x7fd8640524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd864050030 0x7fd8640524f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd88401c000 tx=0x7fd884016000).stop 2020-07-18T03:34:39.451 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd88c0b5dc0 msgr2=0x7fd88c140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd88c0b5dc0 0x7fd88c140290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fd88000c000 tx=0x7fd880006dd0).stop 2020-07-18T03:34:39.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 shutdown_connections 2020-07-18T03:34:39.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fd864050030 0x7fd8640524f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 --2- 172.21.15.47:0/1100886595 >> v2:172.21.15.18:3300/0 conn(0x7fd88c158470 0x7fd88c13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.452 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fd88c061c60 0x7fd88c0636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 --2- 172.21.15.47:0/1100886595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd88c0b5dc0 0x7fd88c140290 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 >> 172.21.15.47:0/1100886595 conn(0x7fd88c05f5a0 msgr2=0x7fd88c141250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:39.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 shutdown_connections 2020-07-18T03:34:39.453 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.450+0000 7fd89448b700 1 -- 172.21.15.47:0/1100886595 wait complete. 2020-07-18T03:34:39.461 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-07-18T03:34:39.462 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-07-18T03:34:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 Processor -- start 2020-07-18T03:34:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 -- start start 2020-07-18T03:34:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1da40ad100 0x7f1da40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1da4147a00 0x7f1da4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da4147330 con 0x7f1da405f5a0 2020-07-18T03:34:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da40ae530 con 0x7f1da4147a00 2020-07-18T03:34:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.577+0000 7f1daa0a8700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40b57e0 con 0x7f1da40ad100 2020-07-18T03:34:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1da405f5a0 0x7f1da405fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45532/0 (socket says 172.21.15.47:45532) 2020-07-18T03:34:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da3fff700 1 -- 172.21.15.47:0/1189818380 learned_addr learned my addr 172.21.15.47:0/1189818380 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da37fe700 1 --2- 172.21.15.47:0/1189818380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1da40ad100 0x7f1da40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da2ffd700 1 --2- 172.21.15.47:0/1189818380 >> v2:172.21.15.18:3300/0 conn(0x7f1da4147a00 0x7f1da4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da27fc700 1 -- 172.21.15.47:0/1189818380 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232086963 0 0) 0x7f1da4147330 con 0x7f1da405f5a0 2020-07-18T03:34:39.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da27fc700 1 -- 172.21.15.47:0/1189818380 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d84003240 con 0x7f1da405f5a0 2020-07-18T03:34:39.594 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da37fe700 1 -- 172.21.15.47:0/1189818380 >> v2:172.21.15.18:3300/0 conn(0x7f1da4147a00 msgr2=0x7f1da4147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da37fe700 1 --2- 172.21.15.47:0/1189818380 >> v2:172.21.15.18:3300/0 conn(0x7f1da4147a00 0x7f1da4147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da37fe700 1 -- 172.21.15.47:0/1189818380 >> v1:172.21.15.47:6789/0 conn(0x7f1da405f5a0 legacy=0x7f1da405fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da27fc700 1 -- 172.21.15.47:0/1189818380 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2687838037 0 0) 0x7f1d84003240 con 0x7f1da405f5a0 2020-07-18T03:34:39.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da37fe700 1 -- 172.21.15.47:0/1189818380 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da40b5a80 con 0x7f1da40ad100 2020-07-18T03:34:39.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da37fe700 1 --2- 172.21.15.47:0/1189818380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1da40ad100 0x7f1da40b4c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1d94001870 tx=0x7f1d9c0091d0).ready entity=mon.2 client_cookie=baa031caebcc124d server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:39.596 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.578+0000 7f1da27fc700 1 -- 172.21.15.47:0/1189818380 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d940016d0 con 0x7f1da40ad100 2020-07-18T03:34:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1da27fc700 1 -- 172.21.15.47:0/1189818380 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00b800 con 0x7f1da40ad100 2020-07-18T03:34:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1da27fc700 1 -- 172.21.15.47:0/1189818380 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9c00b970 con 0x7f1da40ad100 2020-07-18T03:34:39.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 -- 172.21.15.47:0/1189818380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1da40ad100 msgr2=0x7f1da40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/1189818380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1da40ad100 0x7f1da40b4c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1d94001870 tx=0x7f1d9c0091d0).stop 2020-07-18T03:34:39.598 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 -- 172.21.15.47:0/1189818380 shutdown_connections 2020-07-18T03:34:39.599 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/1189818380 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7f1da40ad100 0x7f1da40b4c10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/1189818380 >> v2:172.21.15.18:3300/0 conn(0x7f1da4147a00 0x7f1da4147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 -- 172.21.15.47:0/1189818380 >> 172.21.15.47:0/1189818380 conn(0x7f1da40b9340 msgr2=0x7f1da40b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:39.606 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 -- 172.21.15.47:0/1189818380 shutdown_connections 2020-07-18T03:34:39.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.579+0000 7f1daa0a8700 1 -- 172.21.15.47:0/1189818380 wait complete. 2020-07-18T03:34:39.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.580+0000 7f1daa0a8700 1 Processor -- start 2020-07-18T03:34:39.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.580+0000 7f1daa0a8700 1 -- start start 2020-07-18T03:34:39.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1daa0a8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f1da40b30d0 0x7f1da40629b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.607 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1daa0a8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da4064f10 con 0x7f1da405f5a0 2020-07-18T03:34:39.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1daa0a8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da40651b0 con 0x7f1da4147a00 2020-07-18T03:34:39.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1daa0a8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da4064d00 con 0x7f1da40b30d0 2020-07-18T03:34:39.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1da2ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1da4147a00 0x7f1da40663a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45538/0 (socket says 172.21.15.47:45538) 2020-07-18T03:34:39.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1da2ffd700 1 -- 172.21.15.47:0/3329895211 learned_addr learned my addr 172.21.15.47:0/3329895211 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:39.608 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1da3fff700 1 --2- 172.21.15.47:0/3329895211 >> v2:172.21.15.18:3300/0 conn(0x7f1da40b30d0 0x7f1da40629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184600061 0 0) 0x7f1da40651b0 con 0x7f1da4147a00 2020-07-18T03:34:39.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.581+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d80003240 con 0x7f1da4147a00 2020-07-18T03:34:39.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.1 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2244621892 0 0) 0x7f1d80003240 con 0x7f1da4147a00 2020-07-18T03:34:39.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 --> v1:172.21.15.47:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1da40651b0 con 0x7f1da4147a00 2020-07-18T03:34:39.609 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.1 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1361411873 0 0) 0x7f1d8c00b7f0 con 0x7f1da4147a00 2020-07-18T03:34:39.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1da3fff700 1 -- 172.21.15.47:0/3329895211 >> v1:172.21.15.47:6789/0 conn(0x7f1da4147a00 legacy=0x7f1da40663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1da3fff700 1 -- 172.21.15.47:0/3329895211 >> v1:172.21.15.18:6789/0 conn(0x7f1da405f5a0 legacy=0x7f1da41528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1da3fff700 1 -- 172.21.15.47:0/3329895211 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da4066ab0 con 0x7f1da40b30d0 2020-07-18T03:34:39.610 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1da3fff700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1da40b30d0 0x7f1da40629b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f1d9c006ca0 tx=0x7f1d940013b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:39.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d94007170 con 0x7f1da40b30d0 2020-07-18T03:34:39.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d940072d0 con 0x7f1da40b30d0 2020-07-18T03:34:39.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d940046f0 con 0x7f1da40b30d0 2020-07-18T03:34:39.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da4065650 con 0x7f1da40b30d0 2020-07-18T03:34:39.611 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.582+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da4065b40 con 0x7f1da40b30d0 2020-07-18T03:34:39.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.583+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da40adf10 con 0x7f1da40b30d0 2020-07-18T03:34:39.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.583+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7f1d940048f0 con 0x7f1da40b30d0 2020-07-18T03:34:39.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.583+0000 7f1d9b7fe700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d800505f0 0x7f1d80052ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:39.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.584+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d94067430 con 0x7f1da40b30d0 2020-07-18T03:34:39.612 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.584+0000 7f1da37fe700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d800505f0 0x7f1d80052ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.584+0000 7f1da37fe700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d800505f0 0x7f1d80052ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f1d9c0091d0 tx=0x7f1d9c016000).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.588+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7f1d9403faa0 con 0x7f1da40b30d0 2020-07-18T03:34:39.797 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.796+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1da4057450 con 0x7f1da40b30d0 2020-07-18T03:34:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.797+0000 7f1d9b7fe700 1 -- 172.21.15.47:0/3329895211 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f1d940430c0 con 0x7f1da40b30d0 2020-07-18T03:34:39.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2020-07-18T03:34:39.798 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6639,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2078214370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2078214370},{"type":"v1","addr":"172.21.15.47:6835","nonce":2078214370}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-07-18T03:34:28.745915+0000","modified":"2020-07-18T03:34:30.375173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6645":{"gid":6645,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.18:6833/303188627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":303188627},{"type":"v1","addr":"172.21.15.18:6833","nonce":303188627}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-07-18T03:34:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d800505f0 msgr2=0x7f1d80052ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d800505f0 0x7f1d80052ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f1d9c0091d0 tx=0x7f1d9c016000).stop 2020-07-18T03:34:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1da40b30d0 msgr2=0x7f1da40629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1da40b30d0 0x7f1da40629b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f1d9c006ca0 tx=0x7f1d940013b0).stop 2020-07-18T03:34:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 shutdown_connections 2020-07-18T03:34:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1da40b30d0 0x7f1da40629b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 --2- 172.21.15.47:0/3329895211 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7f1d800505f0 0x7f1d80052ab0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 >> 172.21.15.47:0/3329895211 conn(0x7f1da40b9340 msgr2=0x7f1da41469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 shutdown_connections 2020-07-18T03:34:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:39.798+0000 7f1daa0a8700 1 -- 172.21.15.47:0/3329895211 wait complete. 2020-07-18T03:34:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2020-07-18T03:34:40.014 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-07-18T03:34:28.745915+0000', 'modified': '2020-07-18T03:34:30.375173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6645}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6645': {'gid': 6645, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.18:6833/303188627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6832', 'nonce': 303188627}, {'type': 'v1', 'addr': '172.21.15.18:6833', 'nonce': 303188627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-07-18T03:34:40.015 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-07-18T03:34:40.016 INFO:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-07-18T03:34:40.167 INFO:teuthology.orchestra.run.smithi018.stdout:{"cluster_fsid":"e5c9cb72-d58e-4215-ba0e-aa8d854b8526","whoami":0,"id":6645,"want_state":"up:active","state":"up:active","rank_uptime":11.410326313000001,"mdsmap_epoch":42,"osdmap_epoch":79,"osdmap_epoch_barrier":79,"uptime":16.635670116} 2020-07-18T03:34:40.167 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e5c9cb72-d58e-4215-ba0e-aa8d854b8526", "id": 6645, "mdsmap_epoch": 42, "osdmap_epoch": 79, "osdmap_epoch_barrier": 79, "rank_uptime": 11.410326313, "state": "up:active", "uptime": 16.635670116, "want_state": "up:active", "whoami": 0 } 2020-07-18T03:34:40.168 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-07-18T03:34:40.168 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:34:40.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.170 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:34:40.186 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:40.187 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:40.187 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:40.187 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.187 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:40.187 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.188 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:40.188 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:40.188 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:40.188 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:40.188 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:40.189 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42311sec preferred_lft 42311sec 2020-07-18T03:34:40.189 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:34:40.189 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.189 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:40.189 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:40.190 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:40.190 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:40.190 INFO:teuthology.orchestra.run.smithi047.stdout:24: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:34:40.190 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:bd:33:4b:ed:d6 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:40.190 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:34:40.191 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.191 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::2c58:d5ff:fe48:6050/64 scope link 2020-07-18T03:34:40.191 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.191 INFO:teuthology.orchestra.run.smithi047.stdout:25: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:34:40.191 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:bd:33:4b:ed:d6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:34:40.192 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::d8bd:33ff:fe4b:edd6/64 scope link 2020-07-18T03:34:40.192 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.193 INFO:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-07-18T03:34:40.259 INFO:teuthology.orchestra.run.smithi047.stdout:1 2020-07-18T03:34:40.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.261 INFO:teuthology.orchestra.run.smithi047:> route 2020-07-18T03:34:40.318 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2020-07-18T03:34:40.331 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-07-18T03:34:40.332 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-07-18T03:34:40.332 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-07-18T03:34:40.332 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-07-18T03:34:40.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.333 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT' 2020-07-18T03:34:40.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.393 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT' 2020-07-18T03:34:40.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.468 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE' 2020-07-18T03:34:40.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.541 INFO:teuthology.orchestra.run.smithi047:> ip netns list 2020-07-18T03:34:40.595 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-07-18T03:34:40.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.596 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:34:40.649 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:34:40.650 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 2020-07-18T03:34:40.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.651 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:34:40.704 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:34:40.705 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 2020-07-18T03:34:40.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.706 INFO:teuthology.orchestra.run.smithi047:> ip netns list-id 2020-07-18T03:34:40.759 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-07-18T03:34:40.760 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 1 2020-07-18T03:34:40.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.762 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:34:40.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.839 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 2' 2020-07-18T03:34:40.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:40.912 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:34:40.992 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:40.993 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:40.993 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:40.993 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.994 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:40.994 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.994 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if25: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:34:40.994 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 6e:4b:f7:5c:8a:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:34:40.994 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:34:40.995 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:40.995 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::6c4b:f7ff:fe5c:8a5b/64 scope link 2020-07-18T03:34:40.995 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:41.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.002 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr' 2020-07-18T03:34:41.082 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:41.082 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:41.082 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:41.083 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:41.083 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:41.083 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:41.083 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if25: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:34:41.083 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 6e:4b:f7:5c:8a:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-07-18T03:34:41.084 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-07-18T03:34:41.084 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:41.084 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::6c4b:f7ff:fe5c:8a5b/64 scope link 2020-07-18T03:34:41.084 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:41.093 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-07-18T03:34:41.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.143 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.2' 2020-07-18T03:34:41.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.179 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0' 2020-07-18T03:34:41.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.287 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up' 2020-07-18T03:34:41.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.361 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up' 2020-07-18T03:34:41.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.445 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254' 2020-07-18T03:34:41.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.510 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.2 up' 2020-07-18T03:34:41.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:41.602 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set dev brx.2 master ceph-brx' 2020-07-18T03:34:41.704 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-07-18T03:34:41.705 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi047.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-07-18T03:34:41.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:41.706 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:41.722 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:34:41.804 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:34:41.804 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:34:41.877 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:34:41.877 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:34:41.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:41.879 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:34:41.932 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:34:41.933 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-07-18T03:34:41.933 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:41.975 INFO:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2020-07-18T03:34:42.007 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-07-18T03:34:42.007 INFO:teuthology.orchestra.run.smithi047:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-07-18T03:34:42.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:2020-07-18T03:34:42.070+0000 7ff8137811c0 -1 init, newargv = 0x555dbfd0d680 newargc=15ceph-fuse[53400]: starting ceph client 2020-07-18T03:34:42.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr: 2020-07-18T03:34:42.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[53400]: starting fuse 2020-07-18T03:34:42.081 INFO:teuthology.orchestra.run.smithi047.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-07-18T03:34:42.082 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-07-18T03:34:42.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:42.083 INFO:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2020-07-18T03:34:42.137 INFO:teuthology.orchestra.run.smithi047.stdout:47 2020-07-18T03:34:42.138 INFO:teuthology.orchestra.run.smithi047.stdout:48 2020-07-18T03:34:42.138 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-07-18T03:34:42.139 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:34:42.139 INFO:teuthology.orchestra.run.smithi047:> import glob 2020-07-18T03:34:42.140 INFO:teuthology.orchestra.run.smithi047:> import re 2020-07-18T03:34:42.140 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:34:42.140 INFO:teuthology.orchestra.run.smithi047:> import subprocess 2020-07-18T03:34:42.141 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:42.141 INFO:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2020-07-18T03:34:42.142 INFO:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-07-18T03:34:42.142 INFO:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2020-07-18T03:34:42.143 INFO:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2020-07-18T03:34:42.143 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:42.144 INFO:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2020-07-18T03:34:42.144 INFO:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2020-07-18T03:34:42.144 INFO:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2020-07-18T03:34:42.145 INFO:teuthology.orchestra.run.smithi047:> return files[0] 2020-07-18T03:34:42.145 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:42.147 INFO:teuthology.orchestra.run.smithi047:> for f in files: 2020-07-18T03:34:42.148 INFO:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-07-18T03:34:42.148 INFO:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2020-07-18T03:34:42.149 INFO:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-07-18T03:34:42.149 INFO:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2020-07-18T03:34:42.149 INFO:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2020-07-18T03:34:42.150 INFO:teuthology.orchestra.run.smithi047:> return f 2020-07-18T03:34:42.150 INFO:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-07-18T03:34:42.151 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:42.151 INFO:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.1")) 2020-07-18T03:34:42.152 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:34:42.269 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.1.53400.asok 2020-07-18T03:34:42.450 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.53400.asok 2020-07-18T03:34:42.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:42.451 INFO:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.53400.asok status 2020-07-18T03:34:42.603 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2020-07-18T03:34:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2020-07-18T03:34:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "0baf19aeafe8fa5b354a28b0fb65984f752d8b9d", 2020-07-18T03:34:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 16.0.0-3461-g0baf19a (0baf19aeafe8fa5b354a28b0fb65984f752d8b9d) pacific (dev)", 2020-07-18T03:34:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "1", 2020-07-18T03:34:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2020-07-18T03:34:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-07-18T03:34:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "53400", 2020-07-18T03:34:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2020-07-18T03:34:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2020-07-18T03:34:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2020-07-18T03:34:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 7237, 2020-07-18T03:34:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2020-07-18T03:34:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2020-07-18T03:34:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2020-07-18T03:34:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7237 2020-07-18T03:34:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:34:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:34:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 673814781 2020-07-18T03:34:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2020-07-18T03:34:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2020-07-18T03:34:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2020-07-18T03:34:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2020-07-18T03:34:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 673814781 2020-07-18T03:34:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2020-07-18T03:34:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.7237 192.168.0.2:0/673814781", 2020-07-18T03:34:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/673814781", 2020-07-18T03:34:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2020-07-18T03:34:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 42, 2020-07-18T03:34:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 79, 2020-07-18T03:34:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2020-07-18T03:34:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklisted": false 2020-07-18T03:34:42.610 INFO:teuthology.orchestra.run.smithi047.stdout:} 2020-07-18T03:34:42.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:42.612 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-07-18T03:34:42.613 INFO:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:42.629 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2020-07-18T03:34:42.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-07-18T03:34:42.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:42.631 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:42.703 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2020-07-18T03:34:42.874 INFO:teuthology.orchestra.run.smithi047.stdout:{"mds_reconnect_timeout":"45.000000"} 2020-07-18T03:34:42.875 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2020-07-18T03:34:42.876 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2020-07-18T03:34:43.007 INFO:teuthology.orchestra.run.smithi047.stdout:{"ms_max_backoff":"15.000000"} 2020-07-18T03:34:43.008 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2020-07-18T03:34:43.009 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:34:43.009 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:34:43.010 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:43.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi047.stderr:ceph-fuse[53400]: fuse finished with error 0 and tester_r 0 2020-07-18T03:34:43.063 INFO:teuthology.orchestra.run:waiting for 900 2020-07-18T03:34:43.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:43.066 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-07-18T03:34:43.121 INFO:tasks.cephfs.mount:Cleaning the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-07-18T03:34:43.122 INFO:tasks.cephfs.mount:Removing the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:34:43.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:43.123 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.2 down' 2020-07-18T03:34:43.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:43.204 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link delete brx.2' 2020-07-18T03:34:43.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:43.317 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1' 2020-07-18T03:34:43.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:43.357 INFO:teuthology.orchestra.run.smithi047:> ip addr 2020-07-18T03:34:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-07-18T03:34:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2020-07-18T03:34:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2020-07-18T03:34:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-07-18T03:34:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-07-18T03:34:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42308sec preferred_lft 42308sec 2020-07-18T03:34:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2020-07-18T03:34:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-07-18T03:34:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:24: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-07-18T03:34:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:bd:33:4b:ed:d6 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-07-18T03:34:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::2c58:d5ff:fe48:6050/64 scope link 2020-07-18T03:34:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:25: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-07-18T03:34:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:bd:33:4b:ed:d6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:34:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::d8bd:33ff:fe4b:edd6/64 scope link 2020-07-18T03:34:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2020-07-18T03:34:43.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:43.420 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link show' 2020-07-18T03:34:43.499 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.499 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-07-18T03:34:43.499 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.500 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.500 INFO:teuthology.orchestra.run.smithi047.stdout:3: ens1f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.500 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.500 INFO:teuthology.orchestra.run.smithi047.stdout:4: eno2: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.500 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.501 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.501 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.501 INFO:teuthology.orchestra.run.smithi047.stdout:24: ceph-brx: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.501 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:bd:33:4b:ed:d6 brd ff:ff:ff:ff:ff:ff 2020-07-18T03:34:43.501 INFO:teuthology.orchestra.run.smithi047.stdout:25: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP mode DEFAULT group default qlen 1000 2020-07-18T03:34:43.502 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether da:bd:33:4b:ed:d6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2020-07-18T03:34:43.503 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-07-18T03:34:43.504 INFO:teuthology.orchestra.run.smithi047:> import os 2020-07-18T03:34:43.504 INFO:teuthology.orchestra.run.smithi047:> import time 2020-07-18T03:34:43.504 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:43.505 INFO:teuthology.orchestra.run.smithi047:> path = "/home/ubuntu/cephtest/mnt.0/subdir" 2020-07-18T03:34:43.505 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:43.510 INFO:teuthology.orchestra.run.smithi047:> print("Starting creation...") 2020-07-18T03:34:43.510 INFO:teuthology.orchestra.run.smithi047:> start = time.time() 2020-07-18T03:34:43.511 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:43.511 INFO:teuthology.orchestra.run.smithi047:> os.mkdir(path) 2020-07-18T03:34:43.512 INFO:teuthology.orchestra.run.smithi047:> dfd = os.open(path, os.O_DIRECTORY) 2020-07-18T03:34:43.512 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:43.512 INFO:teuthology.orchestra.run.smithi047:> fd = open(os.path.join(path, "childfile"), "w") 2020-07-18T03:34:43.513 INFO:teuthology.orchestra.run.smithi047:> print("Finished creation in {0}s".format(time.time() - start)) 2020-07-18T03:34:43.513 INFO:teuthology.orchestra.run.smithi047:> 2020-07-18T03:34:43.514 INFO:teuthology.orchestra.run.smithi047:> print("Starting fsync...") 2020-07-18T03:34:43.514 INFO:teuthology.orchestra.run.smithi047:> start = time.time() 2020-07-18T03:34:43.514 INFO:teuthology.orchestra.run.smithi047:> if True: 2020-07-18T03:34:43.515 INFO:teuthology.orchestra.run.smithi047:> os.fsync(dfd) 2020-07-18T03:34:43.515 INFO:teuthology.orchestra.run.smithi047:> else: 2020-07-18T03:34:43.516 INFO:teuthology.orchestra.run.smithi047:> os.fsync(fd) 2020-07-18T03:34:43.518 INFO:teuthology.orchestra.run.smithi047:> print("Finished fsync in {0}s".format(time.time() - start)) 2020-07-18T03:34:43.519 INFO:teuthology.orchestra.run.smithi047:> ' 2020-07-18T03:34:43.638 INFO:teuthology.orchestra.run.smithi047.stdout:Starting creation... 2020-07-18T03:34:43.639 INFO:teuthology.orchestra.run.smithi047.stdout:Finished creation in 0.0073053836822509766s 2020-07-18T03:34:43.639 INFO:teuthology.orchestra.run.smithi047.stdout:Starting fsync... 2020-07-18T03:34:43.639 INFO:teuthology.orchestra.run.smithi047.stdout:Finished fsync in 0.0023374557495117188s 2020-07-18T03:34:43.817 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-07-18T03:34:43.818 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:34:43.818 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-07-18T03:34:43.819 INFO:teuthology.orchestra.run:waiting for 300 2020-07-18T03:34:43.829 INFO:tasks.ceph.mds.b:Stopped 2020-07-18T03:34:43.830 DEBUG:teuthology.parallel:result is None 2020-07-18T03:34:43.866 INFO:tasks.ceph.mds.a:Stopped 2020-07-18T03:34:43.866 DEBUG:teuthology.parallel:result is None 2020-07-18T03:34:43.868 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail a 2020-07-18T03:34:43.869 INFO:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail b 2020-07-18T03:34:44.027 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.025+0000 7fc330fc5700 1 Processor -- start 2020-07-18T03:34:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.025+0000 7fc330fc5700 1 -- start start 2020-07-18T03:34:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.025+0000 7fe70d763700 1 Processor -- start 2020-07-18T03:34:44.028 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.025+0000 7fe70d763700 1 -- start start 2020-07-18T03:34:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc330fc5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c05ffb0 0x7fc32c0625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc330fc5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc330fc5700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c0b5830 con 0x7fc32c0aea60 2020-07-18T03:34:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc330fc5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c0b5a60 con 0x7fc32c1471d0 2020-07-18T03:34:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc330fc5700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c060390 con 0x7fc32c05ffb0 2020-07-18T03:34:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc32ad9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc32a59c700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c05ffb0 0x7fc32c0625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc329d9b700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fc32c0aea60 0x7fc32c142e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.47:36098/0 (socket says 172.21.15.47:36098) 2020-07-18T03:34:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fc329d9b700 1 -- 172.21.15.47:0/855087437 learned_addr learned my addr 172.21.15.47:0/855087437 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32959a700 1 -- 172.21.15.47:0/855087437 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846465425 0 0) 0x7fc32c0b5830 con 0x7fc32c0aea60 2020-07-18T03:34:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32959a700 1 -- 172.21.15.47:0/855087437 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc310003240 con 0x7fc32c0aea60 2020-07-18T03:34:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32a59c700 1 -- 172.21.15.47:0/855087437 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 msgr2=0x7fc32c147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-07-18T03:34:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32a59c700 1 --2- 172.21.15.47:0/855087437 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32a59c700 1 -- 172.21.15.47:0/855087437 >> v1:172.21.15.18:6789/0 conn(0x7fc32c0aea60 legacy=0x7fc32c142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32a59c700 1 -- 172.21.15.47:0/855087437 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c146da0 con 0x7fc32c05ffb0 2020-07-18T03:34:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32ad9d700 1 --2- 172.21.15.47:0/855087437 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32a59c700 1 --2- 172.21.15.47:0/855087437 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c05ffb0 0x7fc32c0625d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fc31c002f30 tx=0x7fc31c006dd0).ready entity=mon.2 client_cookie=4a0a9d1ed57958dc server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32959a700 1 -- 172.21.15.47:0/855087437 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c0096a0 con 0x7fc32c05ffb0 2020-07-18T03:34:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32959a700 1 -- 172.21.15.47:0/855087437 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c009800 con 0x7fc32c05ffb0 2020-07-18T03:34:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc32959a700 1 -- 172.21.15.47:0/855087437 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c009b60 con 0x7fc32c05ffb0 2020-07-18T03:34:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 -- 172.21.15.47:0/855087437 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c05ffb0 msgr2=0x7fc32c0625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 --2- 172.21.15.47:0/855087437 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c05ffb0 0x7fc32c0625d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fc31c002f30 tx=0x7fc31c006dd0).stop 2020-07-18T03:34:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 -- 172.21.15.47:0/855087437 shutdown_connections 2020-07-18T03:34:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 --2- 172.21.15.47:0/855087437 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 --2- 172.21.15.47:0/855087437 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c05ffb0 0x7fc32c0625d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 -- 172.21.15.47:0/855087437 >> 172.21.15.47:0/855087437 conn(0x7fc32c0b93c0 msgr2=0x7fc32c0b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 -- 172.21.15.47:0/855087437 shutdown_connections 2020-07-18T03:34:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fc330fc5700 1 -- 172.21.15.47:0/855087437 wait complete. 2020-07-18T03:34:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.029+0000 7fc330fc5700 1 Processor -- start 2020-07-18T03:34:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.029+0000 7fc330fc5700 1 -- start start 2020-07-18T03:34:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc330fc5700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c0aea60 0x7fc32c13f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc330fc5700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c066340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.046 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc330fc5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c154050 con 0x7fc32c0625d0 2020-07-18T03:34:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc330fc5700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c1127e0 con 0x7fc32c1471d0 2020-07-18T03:34:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc330fc5700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1542f0 con 0x7fc32c0aea60 2020-07-18T03:34:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc32ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc32c0625d0 0x7fc32c066830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45580/0 (socket says 172.21.15.47:45580) 2020-07-18T03:34:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc32ad9d700 1 -- 172.21.15.47:0/3829515160 learned_addr learned my addr 172.21.15.47:0/3829515160 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:44.047 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc32a59c700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c0aea60 0x7fc32c13f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc329d9b700 1 --2- 172.21.15.47:0/3829515160 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732508948 0 0) 0x7fc32c154050 con 0x7fc32c0625d0 2020-07-18T03:34:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc300003240 con 0x7fc32c0625d0 2020-07-18T03:34:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc32a59c700 1 -- 172.21.15.47:0/3829515160 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 msgr2=0x7fc32c066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc32a59c700 1 --2- 172.21.15.47:0/3829515160 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c066340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fc32a59c700 1 -- 172.21.15.47:0/3829515160 >> v1:172.21.15.47:6789/0 conn(0x7fc32c0625d0 legacy=0x7fc32c066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc32a59c700 1 -- 172.21.15.47:0/3829515160 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c064020 con 0x7fc32c0aea60 2020-07-18T03:34:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc329d9b700 1 --2- 172.21.15.47:0/3829515160 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c066340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:44.049 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc32a59c700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c0aea60 0x7fc32c13f460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fc31c002f30 tx=0x7fc31c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:44.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c011e00 con 0x7fc32c0aea60 2020-07-18T03:34:44.050 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c01e460 con 0x7fc32c0aea60 2020-07-18T03:34:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c064210 con 0x7fc32c0aea60 2020-07-18T03:34:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c01e7c0 con 0x7fc32c0aea60 2020-07-18T03:34:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c13fb10 con 0x7fc32c0aea60 2020-07-18T03:34:44.051 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.032+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c153ea0 con 0x7fc32c0aea60 2020-07-18T03:34:44.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.032+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fc31c025030 con 0x7fc32c0aea60 2020-07-18T03:34:44.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fc31a7fc700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc300050360 0x7fc300052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.052 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc31c068fe0 con 0x7fc32c0aea60 2020-07-18T03:34:44.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fc329d9b700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc300050360 0x7fc300052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fc329d9b700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc300050360 0x7fc300052820 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fc32000d000 tx=0x7fc320009040).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:44.053 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe70d763700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe708143300 0x7fe7081436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe70d763700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080b1820 con 0x7fe708062b40 2020-07-18T03:34:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe70d763700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080b1ac0 con 0x7fe7080b5c80 2020-07-18T03:34:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe70d763700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080b1640 con 0x7fe708143300 2020-07-18T03:34:44.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe7077fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe7080b5c80 0x7fe70814dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45574/0 (socket says 172.21.15.47:45574) 2020-07-18T03:34:44.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe7077fe700 1 -- 172.21.15.47:0/3538635531 learned_addr learned my addr 172.21.15.47:0/3538635531 (peer_addr_for_me v1:172.21.15.47:0/0) 2020-07-18T03:34:44.055 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.026+0000 7fe706ffd700 1 --2- 172.21.15.47:0/3538635531 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe708143300 0x7fe7081436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe705ffb700 1 -- 172.21.15.47:0/3538635531 <== mon.1 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149452018 0 0) 0x7fe7080b1ac0 con 0x7fe7080b5c80 2020-07-18T03:34:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe705ffb700 1 -- 172.21.15.47:0/3538635531 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f4003240 con 0x7fe7080b5c80 2020-07-18T03:34:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe705ffb700 1 -- 172.21.15.47:0/3538635531 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192716311 0 0) 0x7fe7080b1820 con 0x7fe708062b40 2020-07-18T03:34:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe706ffd700 1 -- 172.21.15.47:0/3538635531 >> v1:172.21.15.47:6789/0 conn(0x7fe7080b5c80 legacy=0x7fe70814dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe706ffd700 1 -- 172.21.15.47:0/3538635531 >> v1:172.21.15.18:6789/0 conn(0x7fe708062b40 legacy=0x7fe708062fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe706ffd700 1 -- 172.21.15.47:0/3538635531 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080b1d60 con 0x7fe708143300 2020-07-18T03:34:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe706ffd700 1 --2- 172.21.15.47:0/3538635531 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe708143300 0x7fe7081436e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fe700001bf0 tx=0x7fe6f8006dd0).ready entity=mon.2 client_cookie=eab69639c7aa9c03 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe705ffb700 1 -- 172.21.15.47:0/3538635531 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe700001a50 con 0x7fe708143300 2020-07-18T03:34:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe705ffb700 1 -- 172.21.15.47:0/3538635531 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f80096a0 con 0x7fe708143300 2020-07-18T03:34:44.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe705ffb700 1 -- 172.21.15.47:0/3538635531 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f8009810 con 0x7fe708143300 2020-07-18T03:34:44.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe70d763700 1 -- 172.21.15.47:0/3538635531 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe708143300 msgr2=0x7fe7081436e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.059 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe70d763700 1 --2- 172.21.15.47:0/3538635531 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe708143300 0x7fe7081436e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fe700001bf0 tx=0x7fe6f8006dd0).stop 2020-07-18T03:34:44.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe70d763700 1 -- 172.21.15.47:0/3538635531 shutdown_connections 2020-07-18T03:34:44.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe70d763700 1 --2- 172.21.15.47:0/3538635531 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe708143300 0x7fe7081436e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.060 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe70d763700 1 -- 172.21.15.47:0/3538635531 >> 172.21.15.47:0/3538635531 conn(0x7fe7080ba4e0 msgr2=0x7fe7080ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:44.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.027+0000 7fe70d763700 1 -- 172.21.15.47:0/3538635531 shutdown_connections 2020-07-18T03:34:44.070 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.028+0000 7fe70d763700 1 -- 172.21.15.47:0/3538635531 wait complete. 2020-07-18T03:34:44.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.029+0000 7fe70d763700 1 Processor -- start 2020-07-18T03:34:44.071 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.029+0000 7fe70d763700 1 -- start start 2020-07-18T03:34:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe70d763700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 0x7fe7080aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe70d763700 1 --2- >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7080b5c80 0x7fe70813e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe70d763700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080ae450 con 0x7fe70813e8c0 2020-07-18T03:34:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe70d763700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7080ae6f0 con 0x7fe708062b40 2020-07-18T03:34:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe70d763700 1 -- --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080afd80 con 0x7fe7080b5c80 2020-07-18T03:34:44.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe706ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 0x7fe7080aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe706ffd700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 0x7fe7080aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.47:50174/0 (socket says 172.21.15.47:50174) 2020-07-18T03:34:44.074 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe706ffd700 1 -- 172.21.15.47:0/2572615963 learned_addr learned my addr 172.21.15.47:0/2572615963 (peer_addr_for_me v2:172.21.15.47:0/0) 2020-07-18T03:34:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.030+0000 7fe7067fc700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7080b5c80 0x7fe70813e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe7067fc700 1 -- 172.21.15.47:0/2572615963 >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 msgr2=0x7fe7080aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe7067fc700 1 --2- 172.21.15.47:0/2572615963 >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 0x7fe7080aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe7067fc700 1 -- 172.21.15.47:0/2572615963 >> v1:172.21.15.18:6789/0 conn(0x7fe70813e8c0 legacy=0x7fe708155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe7067fc700 1 -- 172.21.15.47:0/2572615963 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080afef0 con 0x7fe7080b5c80 2020-07-18T03:34:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe706ffd700 1 --2- 172.21.15.47:0/2572615963 >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 0x7fe7080aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-07-18T03:34:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe7067fc700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7080b5c80 0x7fe70813e3d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe6fc000b80 tx=0x7fe6fc007990).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6fc00f470 con 0x7fe7080b5c80 2020-07-18T03:34:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc00f5d0 con 0x7fe7080b5c80 2020-07-18T03:34:44.077 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080af1c0 con 0x7fe7080b5c80 2020-07-18T03:34:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6fc00f930 con 0x7fe7080b5c80 2020-07-18T03:34:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.031+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe708155d10 con 0x7fe7080b5c80 2020-07-18T03:34:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.032+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70805c670 con 0x7fe7080b5c80 2020-07-18T03:34:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 4 ==== mgrmap(e 4) v1 ==== 62075+0+0 (secure 0 0 0) 0x7fe6fc019030 con 0x7fe7080b5c80 2020-07-18T03:34:44.078 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fe6eeffd700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6e404ffe0 0x7fe6e40524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-07-18T03:34:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe6fc0672e0 con 0x7fe7080b5c80 2020-07-18T03:34:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fe706ffd700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6e404ffe0 0x7fe6e40524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-07-18T03:34:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.033+0000 7fe706ffd700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6e404ffe0 0x7fe6e40524a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fe6f801a000 tx=0x7fe6f8006dd0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-07-18T03:34:44.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.036+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fe6fc03fac0 con 0x7fe7080b5c80 2020-07-18T03:34:44.080 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.040+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+132361 (secure 0 0 0) 0x7fc31c0416f0 con 0x7fc32c0aea60 2020-07-18T03:34:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.256+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fe708057450 con 0x7fe7080b5c80 2020-07-18T03:34:44.260 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.258+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 --> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fc32c057450 con 0x7fc32c0aea60 2020-07-18T03:34:44.266 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.264+0000 7fe6eeffd700 1 -- 172.21.15.47:0/2572615963 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6645 v43) v1 ==== 95+0+0 (secure 0 0 0) 0x7fe6fc034090 con 0x7fe7080b5c80 2020-07-18T03:34:44.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6e404ffe0 msgr2=0x7fe6e40524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6e404ffe0 0x7fe6e40524a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fe6f801a000 tx=0x7fe6f8006dd0).stop 2020-07-18T03:34:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7080b5c80 msgr2=0x7fe70813e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7080b5c80 0x7fe70813e3d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe6fc000b80 tx=0x7fe6fc007990).stop 2020-07-18T03:34:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 shutdown_connections 2020-07-18T03:34:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fe6e404ffe0 0x7fe6e40524a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 --2- 172.21.15.47:0/2572615963 >> v2:172.21.15.18:3300/0 conn(0x7fe708062b40 0x7fe7080aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 --2- 172.21.15.47:0/2572615963 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fe7080b5c80 0x7fe70813e3d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.266+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 >> 172.21.15.47:0/2572615963 conn(0x7fe7080ba4e0 msgr2=0x7fe7080b9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.267+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 shutdown_connections 2020-07-18T03:34:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:44.267+0000 7fe70d763700 1 -- 172.21.15.47:0/2572615963 wait complete. 2020-07-18T03:34:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 6645 2020-07-18T03:34:44.279 DEBUG:teuthology.parallel:result is '' 2020-07-18T03:34:45.265 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:34:45.264+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-07-18T03:34:45.268 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.266+0000 7fc31a7fc700 1 -- 172.21.15.47:0/3829515160 <== mon.2 v2:172.21.15.18:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6639 v44) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc31c023070 con 0x7fc32c0aea60 2020-07-18T03:34:45.270 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc300050360 msgr2=0x7fc300052820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc300050360 0x7fc300052820 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fc32000d000 tx=0x7fc320009040).stop 2020-07-18T03:34:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c0aea60 msgr2=0x7fc32c13f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-07-18T03:34:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c0aea60 0x7fc32c13f460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fc31c002f30 tx=0x7fc31c015000).stop 2020-07-18T03:34:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 shutdown_connections 2020-07-18T03:34:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.47:6800/23227,v1:172.21.15.47:6801/23227] conn(0x7fc300050360 0x7fc300052820 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 --2- 172.21.15.47:0/3829515160 >> [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] conn(0x7fc32c0aea60 0x7fc32c13f460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 --2- 172.21.15.47:0/3829515160 >> v2:172.21.15.18:3300/0 conn(0x7fc32c1471d0 0x7fc32c066340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-07-18T03:34:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 >> 172.21.15.47:0/3829515160 conn(0x7fc32c0b93c0 msgr2=0x7fc32c05f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-07-18T03:34:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 shutdown_connections 2020-07-18T03:34:45.272 INFO:teuthology.orchestra.run.smithi047.stderr:2020-07-18T03:34:45.268+0000 7fc330fc5700 1 -- 172.21.15.47:0/3829515160 wait complete. 2020-07-18T03:34:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 6639 2020-07-18T03:34:45.283 DEBUG:teuthology.parallel:result is '' 2020-07-18T03:34:45.284 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:34:45.285 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-07-18T03:34:45.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-07-18T03:34:45.286 INFO:teuthology.orchestra.run.smithi047:> sudo bash -c 'ip link set brx.0 down' 2020-07-18T03:34:45.328 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi047.front.sepia.ceph.com 2020-07-18T03:34:45.329 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2020-07-18T03:34:45.330 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-07-18T03:34:45.330 INFO:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-07-18T03:34:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:fusermount: failed to unmount /home/ubuntu/cephtest/mnt.0: Device or resource busy 2020-07-18T03:34:45.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-07-18T03:34:45.402 INFO:tasks.cephfs.fuse_mount:Failed to unmount ceph-fuse on ubuntu@smithi047.front.sepia.ceph.com, aborting... 2020-07-18T03:34:45.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-07-18T03:34:45.403 INFO:teuthology.orchestra.run.smithi047:> sudo PATH=/usr/sbin:$PATH lsof ; ps auxf 2020-07-18T03:34:50.622 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:34:50.626 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:35:20.704 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:35:20.710 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:35:50.805 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:35:50.812 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:36:20.904 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:36:20.909 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:36:50.979 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:36:50.984 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:37:21.068 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:37:21.073 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:37:51.152 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:37:51.158 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:38:21.245 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:38:21.250 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:38:51.340 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:39:21.416 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:39:21.422 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:39:51.488 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:39:51.495 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:40:21.567 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:40:21.574 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:40:51.652 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:40:51.659 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:41:21.765 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:41:21.771 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:41:51.857 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:41:51.861 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:42:21.945 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:42:21.950 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:42:52.033 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:42:52.057 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:43:22.138 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:43:22.143 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:43:52.240 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:43:52.246 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:44:22.316 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:44:22.321 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:44:52.443 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:44:52.449 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:45:22.532 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:45:22.542 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:45:52.628 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:45:52.633 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:46:22.715 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:46:22.721 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:46:52.795 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:46:52.820 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:47:22.907 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:47:22.911 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:47:53.026 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:47:53.032 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:48:23.111 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:48:23.118 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:48:53.196 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:48:53.201 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:49:23.282 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:49:23.287 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:49:45.449 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-07-18T03:49:45.470 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2020-07-18T03:49:53.355 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:49:53.360 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:50:23.451 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:50:23.457 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:50:53.549 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:50:53.553 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:51:23.805 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:51:23.811 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:51:53.896 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:51:53.903 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:52:23.990 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:52:23.995 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:52:54.096 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:52:54.103 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:53:24.178 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:53:24.183 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:53:54.266 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:53:54.271 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:54:24.355 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:54:24.360 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:54:54.435 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:54:54.441 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:55:24.526 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:55:24.532 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:55:54.613 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:55:54.619 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:56:24.693 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:56:24.699 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:56:54.769 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:56:54.775 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:57:24.849 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:57:24.856 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:57:54.920 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:57:54.925 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:58:24.992 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:58:25.000 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:58:55.096 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:58:55.108 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:59:25.187 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:59:25.192 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:59:55.269 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T03:59:55.279 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:00:00.000 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T03:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T04:00:25.369 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:00:25.394 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:00:55.463 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:00:55.468 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:01:25.551 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:01:25.557 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:01:55.658 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:01:55.663 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:02:25.772 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:02:25.778 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:02:55.886 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:02:55.891 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:03:25.983 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:03:25.992 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:03:56.073 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:03:56.079 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:04:26.186 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:04:26.191 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:04:56.261 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:04:56.266 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:05:26.364 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:05:26.371 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:05:56.478 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:05:56.483 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:06:26.558 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:06:26.569 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:06:56.663 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:06:56.669 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:07:26.735 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:07:26.741 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:07:56.809 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:07:56.814 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:08:26.890 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:08:26.895 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:08:56.988 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:08:56.994 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:09:27.076 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:09:27.082 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:09:57.173 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:09:57.179 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:10:27.257 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:10:27.263 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:10:57.351 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:10:57.356 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:11:27.424 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:11:27.430 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:11:57.521 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:11:57.527 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:12:27.593 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:12:27.600 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:12:57.681 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:12:57.687 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:13:27.752 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:13:27.757 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:13:57.825 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:13:57.830 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:14:27.895 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:14:27.901 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:14:57.979 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:14:57.984 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:15:28.055 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:15:28.062 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:15:58.151 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:15:58.157 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:16:28.237 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:16:28.242 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:16:58.310 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:16:58.316 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:17:28.398 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:17:28.404 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:17:58.495 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:17:58.506 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:18:28.595 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:18:28.610 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:18:58.694 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:18:58.714 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:19:28.790 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:19:28.796 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:19:58.862 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:19:58.867 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:20:28.937 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:20:28.942 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:20:59.020 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:20:59.025 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:21:29.100 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:21:29.106 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:21:59.196 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:21:59.204 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:22:29.285 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:22:29.290 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:22:59.372 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:22:59.378 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:23:29.452 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:23:29.467 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:23:59.538 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:23:59.548 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:24:29.610 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:24:29.631 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:24:59.701 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:24:59.707 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:25:29.796 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:25:29.803 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:25:59.876 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:25:59.889 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:26:29.955 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:26:29.964 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:27:00.043 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:27:00.055 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:27:30.126 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:27:30.131 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:28:00.323 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:28:00.329 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:28:30.412 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:28:30.418 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:29:00.487 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:29:00.492 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:29:30.556 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:29:30.562 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:30:00.630 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:30:00.636 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:30:30.702 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:30:30.716 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:31:00.799 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:31:00.805 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:31:30.896 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:31:30.902 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:32:00.983 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:32:00.988 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:32:31.058 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:32:31.066 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:33:01.125 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:33:01.130 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:33:31.200 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:33:31.206 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:34:01.281 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:34:01.287 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:34:31.370 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:34:31.375 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:35:01.458 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:35:01.464 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:35:31.539 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:35:31.544 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:36:01.612 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:36:01.617 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:36:31.702 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:36:31.709 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:37:01.792 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:37:01.797 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:37:31.881 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:37:31.891 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:38:01.960 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:38:01.966 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:38:32.032 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:38:32.038 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:39:02.111 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:39:02.116 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:39:32.192 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:39:32.197 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:40:02.280 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:40:02.285 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:40:32.376 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:40:32.381 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:41:02.462 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:41:02.471 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:41:32.547 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:41:32.553 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:42:02.627 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:42:02.634 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:42:32.716 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:42:32.721 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:43:02.805 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:43:02.812 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:43:32.885 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:43:32.890 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:44:02.974 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:44:02.979 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:44:33.054 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:44:33.060 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:45:03.126 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:45:03.131 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:45:33.207 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:45:33.222 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:46:03.309 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:46:03.314 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:46:33.387 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:46:33.393 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:47:03.490 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:47:03.496 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:47:33.571 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:47:33.577 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:48:03.659 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:48:03.665 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:48:33.745 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:48:33.750 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:49:03.837 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:49:03.847 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:49:34.024 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:49:34.030 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:50:04.156 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:50:04.162 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:50:34.237 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:50:34.242 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:51:04.427 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:51:04.432 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:51:34.499 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:51:34.505 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:52:04.713 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:52:04.718 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:52:34.784 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:52:34.790 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:53:04.865 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:53:04.872 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:53:34.950 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:53:34.959 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:54:05.033 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:54:05.038 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:54:35.105 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:54:35.112 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:55:05.195 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:55:05.201 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:55:35.268 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:55:35.278 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:56:05.371 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:56:05.387 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:56:35.451 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:56:35.457 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:57:05.537 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:57:05.542 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:57:35.622 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:57:35.627 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:58:05.704 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:58:05.709 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:58:35.776 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:58:35.786 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:59:05.871 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:59:05.880 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:59:35.969 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T04:59:35.974 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:00:00.004 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T04:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T05:00:06.050 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:00:06.054 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:00:36.144 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:00:36.149 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:01:06.228 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:01:06.496 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:01:36.566 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:01:36.571 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:02:06.638 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:02:06.643 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:02:36.861 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:02:36.866 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:03:06.939 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:03:06.949 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:03:37.022 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:03:37.029 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:04:07.093 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:04:07.107 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:04:37.199 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:04:37.205 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:05:07.279 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:05:07.286 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:05:37.383 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:05:37.389 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:06:07.466 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:06:07.472 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:06:37.554 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:06:37.569 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:07:07.762 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:07:07.768 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:07:37.841 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:07:37.855 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:08:07.955 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:08:07.971 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:08:38.060 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:08:38.067 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:09:08.134 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:09:08.141 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:09:38.231 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:09:38.244 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:10:08.324 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:10:08.330 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:10:38.426 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:10:38.432 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:11:08.513 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:11:08.520 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:11:38.713 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:11:38.719 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:12:08.792 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:12:08.798 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:12:38.871 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:12:38.877 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:13:08.952 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:13:08.959 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:13:39.025 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:13:39.049 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:14:09.136 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:14:09.141 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:14:39.232 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:14:39.238 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:15:09.312 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:15:09.317 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:15:39.391 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:15:39.396 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:16:09.477 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:16:09.483 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:16:39.553 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:16:39.563 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:17:09.626 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:17:09.634 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:17:39.716 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:17:39.722 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:18:09.797 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:18:09.804 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:18:39.902 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:18:39.909 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:19:09.983 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:19:09.989 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:19:40.071 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:19:40.077 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:20:10.143 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:20:10.156 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:20:40.230 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:20:40.241 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:21:10.347 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:21:10.359 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:21:40.435 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:21:40.442 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:22:10.524 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:22:10.530 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:22:40.613 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:22:40.618 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:23:10.704 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:23:10.709 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:23:40.782 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:23:40.788 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:24:11.008 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:24:11.016 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:24:41.101 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:24:41.107 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:25:11.184 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:25:11.189 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:25:41.275 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:25:41.281 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:26:11.365 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:26:11.370 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:26:41.448 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:26:41.457 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:27:11.556 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:27:11.562 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:27:41.635 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:27:41.640 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:28:11.716 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:28:11.722 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:28:41.796 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:28:41.808 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:29:11.878 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:29:11.905 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:29:41.976 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:29:41.983 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:30:12.074 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:30:12.082 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:30:42.152 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:30:42.157 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:31:12.375 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:31:12.380 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:31:42.446 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:31:42.464 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:32:12.535 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:32:12.544 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:32:42.620 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:32:42.626 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:33:12.717 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:33:12.723 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:33:42.806 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:33:42.815 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:34:12.890 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:34:12.897 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:34:42.975 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:34:42.980 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:35:13.052 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:35:13.062 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:35:43.149 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:35:43.162 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:36:13.246 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:36:13.250 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:36:43.343 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:36:43.349 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:37:13.432 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:37:13.437 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:37:43.513 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:37:43.518 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:38:13.593 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:38:13.606 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:38:43.684 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:38:43.691 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:39:13.763 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:39:13.769 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:39:43.935 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:39:43.941 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:40:14.016 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:40:14.021 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:40:44.100 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:40:44.105 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:41:14.183 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:41:14.189 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:41:44.271 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:41:44.277 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:42:14.355 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:42:14.361 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:42:44.445 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:42:44.452 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:43:14.537 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:43:14.545 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:43:44.614 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:43:44.623 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:44:14.679 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:44:14.862 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:44:44.924 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:44:44.930 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:45:15.008 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:45:15.016 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:45:45.102 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:45:45.109 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:46:15.200 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:46:15.206 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:46:45.297 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:46:45.303 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:47:15.372 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:47:15.388 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:47:45.477 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:47:45.484 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:48:15.573 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:48:15.579 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:48:45.669 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:48:45.679 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:49:15.866 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:49:15.871 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:49:45.937 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:49:45.944 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:50:16.111 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:50:16.122 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:50:46.191 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:50:46.198 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:51:16.277 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:51:16.287 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:51:46.381 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:51:46.388 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:52:16.460 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:52:16.467 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:52:46.537 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:52:46.543 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:53:16.619 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:53:16.627 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:53:46.716 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:53:46.721 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:54:16.805 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:54:16.812 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:54:46.887 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:54:46.893 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:55:16.992 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:55:17.006 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:55:47.080 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:55:47.102 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:56:17.186 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:56:17.199 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:56:47.270 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:56:47.699 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:57:17.765 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:57:17.771 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:57:47.844 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:57:47.850 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:58:17.949 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:58:17.954 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:58:48.093 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:58:48.104 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:59:18.174 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:59:18.187 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:59:48.258 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T05:59:48.263 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:00:00.001 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T06:00:00.000+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T06:00:18.338 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:00:18.344 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:00:48.422 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:00:48.428 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:01:18.516 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:01:18.521 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:01:48.598 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:01:48.604 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:02:18.679 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:02:18.684 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:02:48.795 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:02:48.800 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:03:18.880 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:03:18.886 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:03:48.951 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:03:48.956 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:04:19.047 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:04:19.055 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:04:49.130 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:04:49.134 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:05:19.211 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:05:19.216 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:05:49.402 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:05:49.407 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:06:19.480 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:06:19.485 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:06:49.577 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:06:49.581 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:07:19.799 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:07:19.805 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:07:49.878 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:07:49.883 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:08:19.958 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:08:19.964 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:08:50.053 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:08:50.058 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:09:20.141 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:09:20.146 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:09:50.230 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:09:50.235 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:10:20.311 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:10:20.317 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:10:50.408 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:10:50.415 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:11:20.500 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:11:20.508 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:11:50.585 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:11:50.595 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:12:20.682 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:12:20.688 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:12:50.900 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:12:50.906 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:13:20.983 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:13:20.990 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:13:51.080 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:13:51.085 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:14:21.155 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:14:21.164 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:14:51.244 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:14:51.258 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:15:21.325 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:15:21.331 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:15:51.420 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:15:51.427 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:16:21.527 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:16:21.532 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:16:51.633 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:16:51.639 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:17:21.713 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:17:21.719 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:17:51.813 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:17:51.820 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:18:21.902 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:18:21.910 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:18:51.989 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:18:52.000 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:19:22.076 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:19:22.088 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:19:52.282 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:19:52.288 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:20:22.362 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:20:22.368 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:20:52.435 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:20:52.441 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:21:22.540 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:21:22.546 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:21:52.621 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:21:52.628 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:22:22.710 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:22:22.715 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:22:52.809 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:22:52.815 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:23:22.888 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:23:22.893 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:23:52.976 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:23:52.982 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:24:23.058 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:24:23.065 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:24:53.155 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:24:53.160 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:25:23.235 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:25:23.242 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:25:53.338 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:25:53.343 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:26:23.417 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:26:23.423 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:26:53.502 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:26:53.508 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:27:23.586 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:27:23.592 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:27:53.655 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:27:53.662 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:28:23.727 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:28:23.735 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:28:53.809 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:28:53.817 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:29:23.905 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:29:23.910 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:29:53.994 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:29:54.000 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:30:24.066 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:30:24.072 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:30:54.145 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:30:54.151 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:31:24.222 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:31:24.227 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:31:54.302 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:31:54.308 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:32:24.397 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:32:24.402 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:32:54.479 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:32:54.484 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:33:24.555 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:33:24.565 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:33:54.621 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:33:54.628 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:34:24.685 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:34:24.691 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:34:54.765 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:34:54.770 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:35:24.856 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:35:24.862 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:35:54.944 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:35:54.950 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:36:25.028 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:36:25.035 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:36:55.107 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:36:55.113 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:37:25.178 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:37:25.183 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:37:55.259 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:37:55.265 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:38:25.345 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:38:25.351 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:38:55.427 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:38:55.439 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:39:25.502 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:39:25.510 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:39:55.595 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:39:55.604 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:40:25.669 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:40:25.678 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:40:55.777 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:40:55.784 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:41:25.867 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:41:25.875 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:41:55.953 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:41:55.958 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:42:26.042 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:42:26.050 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:42:56.120 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:42:56.126 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:43:26.185 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:43:26.195 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:43:56.275 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:43:56.292 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:44:26.368 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:44:26.374 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:44:56.447 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:44:56.457 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:45:26.516 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:45:26.529 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:45:56.615 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:45:56.623 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:46:26.696 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:46:26.707 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:46:56.775 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:46:56.782 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:47:26.901 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:47:26.908 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:47:57.011 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:47:57.020 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:48:27.100 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:48:27.125 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:48:57.187 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:48:57.194 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:49:27.260 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:49:27.266 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:49:57.350 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:49:57.357 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:50:27.418 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:50:27.424 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:50:57.511 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:50:57.518 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:51:27.582 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:51:27.588 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:51:57.664 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:51:57.671 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:52:27.744 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:52:27.750 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:52:57.829 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:52:57.834 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:53:27.897 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:53:27.903 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:53:58.002 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:53:58.025 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:54:28.117 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:54:28.129 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:54:58.214 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:54:58.219 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:55:28.302 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:55:28.309 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:55:58.392 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:55:58.399 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:56:28.464 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:56:28.469 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:56:58.559 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:56:58.565 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:57:28.715 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:57:28.723 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:57:58.804 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:57:58.811 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:58:28.887 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:58:28.898 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:58:58.968 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:58:58.974 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:59:29.040 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:59:29.050 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:59:59.279 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T06:59:59.288 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:00:00.000 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T06:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T07:00:29.341 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:00:29.348 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:00:59.431 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:00:59.448 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:01:29.526 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:01:29.536 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:01:59.600 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:01:59.615 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:02:29.701 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:02:29.709 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:02:59.798 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:02:59.805 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:03:29.868 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:03:29.880 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:03:59.963 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:03:59.968 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:04:30.043 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:04:30.057 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:05:00.153 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:05:00.163 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:05:30.231 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:05:30.240 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:06:00.320 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:06:00.330 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:06:30.416 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:06:30.422 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:07:00.500 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:07:00.519 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:07:30.597 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:07:30.604 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:08:00.684 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:08:00.692 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:08:30.764 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:08:30.777 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:09:00.855 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:09:00.867 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:09:30.951 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:09:30.959 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:10:01.046 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:10:01.053 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:10:31.119 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:10:31.126 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:11:01.203 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:11:01.209 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:11:31.275 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:11:31.280 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:12:01.362 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:12:01.378 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:12:31.458 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:12:31.464 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:13:01.549 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:13:01.570 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:13:31.636 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:13:31.643 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:14:01.724 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:14:01.731 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:14:31.810 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:14:31.815 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:15:01.881 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:15:01.887 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:15:31.961 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:15:31.970 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:16:02.047 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:16:02.058 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:16:32.127 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:16:32.133 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:17:02.200 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:17:02.207 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:17:32.279 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:17:32.286 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:18:02.359 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:18:02.365 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:18:32.441 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:18:32.448 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:19:02.525 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:19:02.530 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:19:32.588 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:19:32.596 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:20:02.676 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:20:02.683 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:20:32.753 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:20:32.759 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:21:02.843 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:21:02.849 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:21:32.941 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:21:32.946 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:22:03.013 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:22:03.020 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:22:33.099 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:22:33.107 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:23:03.178 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:23:03.188 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:23:33.262 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:23:33.268 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:24:03.419 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:24:03.425 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:24:33.493 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:24:33.535 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:25:03.610 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:25:03.635 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:25:33.703 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:25:33.726 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:26:03.807 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:26:03.817 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:26:33.877 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:26:33.884 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:27:03.970 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:27:03.980 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:27:34.058 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:27:34.064 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:28:04.155 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:28:04.162 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:28:34.230 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:28:34.235 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:29:04.326 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:29:04.332 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:29:34.389 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:29:34.395 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:30:04.468 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:30:04.474 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:30:34.557 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:30:34.565 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:31:04.655 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:31:04.661 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:31:34.731 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:31:34.738 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:32:04.829 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:32:04.837 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:32:34.913 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:32:34.918 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:33:05.003 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:33:05.008 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:33:35.074 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:33:35.081 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:34:05.213 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:34:05.220 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:34:35.290 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:34:35.297 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:35:05.369 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:35:05.375 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:35:35.448 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:35:35.464 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:36:05.544 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:36:05.550 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:36:35.617 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:36:35.625 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:37:05.722 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:37:05.728 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:37:35.794 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:37:35.801 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:38:05.891 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:38:05.897 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:38:35.961 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:38:35.967 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:39:06.043 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:39:06.051 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:39:36.116 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:39:36.122 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:40:06.200 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:40:06.205 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:40:36.276 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:40:36.281 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:41:06.368 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:41:06.374 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:41:36.463 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:41:36.471 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:42:06.544 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:42:06.550 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:42:36.615 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:42:36.624 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:43:06.721 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:43:06.730 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:43:36.809 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:43:36.816 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:44:06.889 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:44:06.896 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:44:36.961 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:44:36.966 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:45:07.045 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:45:07.052 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:45:37.136 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:45:37.143 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:46:07.212 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:46:07.219 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:46:37.376 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:46:37.383 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:47:07.461 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:47:07.467 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:47:37.527 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:47:37.532 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:48:07.606 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:48:07.613 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:48:37.697 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:48:37.704 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:49:07.786 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:49:07.800 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:49:37.886 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:49:37.893 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:50:07.981 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:50:07.988 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:50:38.046 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:50:38.052 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:51:08.126 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:51:08.131 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:51:38.213 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:51:38.219 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:52:08.311 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:52:08.322 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:52:38.405 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:52:38.411 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:53:08.485 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:53:08.492 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:53:38.558 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:53:38.564 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:54:08.635 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:54:08.641 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:54:38.708 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:54:38.713 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:55:08.800 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:55:08.807 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:55:38.892 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:55:38.898 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:56:08.979 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:56:08.991 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:56:39.085 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:56:39.093 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:57:09.181 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:57:09.188 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:57:39.257 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:57:39.261 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:58:09.346 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:58:09.352 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:58:39.419 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:58:39.424 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:59:09.513 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:59:09.518 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:59:39.576 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T07:59:39.583 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:00:00.001 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T07:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T08:00:09.653 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:00:09.661 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:00:39.742 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:00:39.748 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:01:09.830 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:01:09.845 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:01:39.928 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:01:39.935 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:02:10.033 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:02:10.039 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:02:40.112 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:02:40.118 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:03:10.202 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:03:10.208 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:03:40.273 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:03:40.279 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:04:10.363 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:04:10.367 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:04:40.435 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:04:40.442 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:05:10.526 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:05:10.533 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:05:40.600 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:05:40.606 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:06:10.754 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:06:10.760 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:06:40.825 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:06:40.843 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:07:10.921 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:07:10.929 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:07:41.002 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:07:41.022 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:08:11.123 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:08:11.133 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:08:41.215 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:08:41.221 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:09:11.304 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:09:11.311 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:09:41.383 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:09:41.393 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:10:11.480 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:10:11.488 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:10:41.552 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:10:41.557 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:11:11.658 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:11:11.664 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:11:41.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:11:41.756 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:12:11.837 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:12:11.844 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:12:41.909 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:12:41.915 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:13:11.993 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:13:11.999 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:13:42.077 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:13:42.087 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:14:12.177 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:14:12.183 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:14:42.255 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:14:42.263 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:15:12.345 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:15:12.353 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:15:42.417 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:15:42.423 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:16:12.505 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:16:12.521 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:16:42.583 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:16:42.605 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:17:12.683 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:17:12.690 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:17:42.773 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:17:42.779 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:18:12.861 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:18:12.867 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:18:42.942 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:18:42.946 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:19:13.031 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:19:13.040 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:19:43.103 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:19:43.111 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:20:13.183 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:20:13.189 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:20:43.281 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:20:43.288 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:21:13.369 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:21:13.380 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:21:43.460 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:21:43.468 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:22:13.550 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:22:13.556 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:22:43.619 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:22:43.627 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:23:13.708 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:23:13.716 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:23:43.831 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:23:43.839 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:24:13.920 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:24:13.925 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:24:43.995 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:24:44.001 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:25:14.081 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:25:14.087 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:25:44.162 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:25:44.168 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:26:14.270 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:26:14.277 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:26:44.365 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:26:44.373 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:27:14.446 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:27:14.453 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:27:44.518 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:27:44.525 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:28:14.597 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:28:14.604 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:28:44.671 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:28:44.684 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:29:14.775 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:29:14.784 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:29:44.869 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:29:44.876 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:30:14.949 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:30:14.958 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:30:45.017 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:30:45.024 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:31:15.098 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:31:15.105 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:31:45.170 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:31:45.176 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:32:15.255 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:32:15.268 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:32:45.340 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:32:45.351 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:33:15.419 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:33:15.428 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:33:45.501 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:33:45.506 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:34:15.590 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:34:15.604 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:34:45.662 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:34:45.668 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:35:15.746 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:35:15.752 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:35:45.819 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:35:45.825 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:36:15.914 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:36:15.919 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:36:45.986 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:36:45.992 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:37:16.073 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:37:16.079 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:37:46.146 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:37:46.151 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:38:16.234 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:38:16.240 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:38:46.315 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:38:46.323 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:39:16.420 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:39:16.431 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:39:46.492 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:39:46.498 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:40:16.575 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:40:16.583 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:40:46.654 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:40:46.661 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:41:16.744 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:41:16.750 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:41:46.816 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:41:46.822 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:42:16.913 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:42:16.925 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:42:46.994 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:42:47.003 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:43:17.082 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:43:17.091 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:43:47.164 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:43:47.174 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:44:17.255 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:44:17.262 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:44:47.323 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:44:47.329 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:45:17.415 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:45:17.422 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:45:47.490 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:45:47.496 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:46:17.574 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:46:17.581 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:46:47.646 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:46:47.651 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:47:17.725 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:47:17.730 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:47:47.800 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:47:47.807 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:48:17.900 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:48:17.906 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:48:47.970 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:48:47.975 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:49:18.058 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:49:18.064 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:49:48.140 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:49:48.147 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:50:18.226 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:50:18.231 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:50:48.308 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:50:48.315 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:51:18.412 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:51:18.419 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:51:48.485 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:51:48.492 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:52:18.574 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:52:18.579 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:52:48.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:52:48.754 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:53:18.830 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:53:18.838 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:53:48.910 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:53:48.922 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:54:19.015 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:54:19.022 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:54:49.171 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:54:49.179 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:55:19.250 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:55:19.258 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:55:49.321 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:55:49.328 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:56:19.427 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:56:19.433 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:56:49.500 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:56:49.510 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:57:19.580 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:57:19.586 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:57:49.678 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:57:49.687 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:58:19.769 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:58:19.778 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:58:49.839 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:58:49.845 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:59:19.927 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:59:19.944 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:59:50.008 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T08:59:50.015 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:00:00.001 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T08:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T09:00:20.089 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:00:20.095 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:00:50.186 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:00:50.191 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:01:20.282 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:01:20.290 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:01:50.349 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:01:50.360 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:02:20.439 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:02:20.446 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:02:50.510 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:02:50.518 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:03:20.605 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:03:20.613 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:03:50.686 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:03:50.691 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:04:20.790 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:04:20.797 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:04:50.862 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:04:50.869 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:05:20.943 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:05:20.949 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:05:51.024 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:05:51.031 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:06:21.121 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:06:21.130 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:06:51.207 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:06:51.214 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:07:21.285 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:07:21.290 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:07:51.355 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:07:51.362 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:08:21.443 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:08:21.450 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:08:51.515 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:08:51.530 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:09:21.604 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:09:21.615 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:09:51.685 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:09:51.695 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:10:21.782 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:10:21.788 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:10:51.860 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:10:51.875 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:11:21.961 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:11:21.966 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:11:52.042 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:11:52.052 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:12:22.140 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:12:22.154 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:12:52.219 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:12:52.227 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:13:22.325 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:13:22.335 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:13:52.411 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:13:52.426 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:14:22.502 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:14:22.510 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:14:52.580 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:14:52.585 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:15:22.663 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:15:23.432 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:15:53.504 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:15:53.515 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:16:23.594 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:16:23.603 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:16:53.672 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:16:53.678 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:17:23.752 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:17:23.767 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:17:53.850 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:17:53.856 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:18:23.928 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:18:23.934 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:18:53.996 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:18:54.003 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:19:24.096 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:19:24.104 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:19:54.181 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:19:54.204 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:20:24.289 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:20:24.308 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:20:54.378 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:20:54.384 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:21:24.458 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:21:24.465 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:21:54.528 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:21:54.533 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:22:24.617 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:22:24.624 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:22:54.689 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:22:54.695 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:23:24.770 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:23:24.777 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:23:54.858 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:23:54.864 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:24:24.947 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:24:24.954 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:24:55.030 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:24:55.038 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:25:25.128 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:25:25.136 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:25:55.207 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:25:55.213 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:26:25.294 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:26:25.304 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:26:55.384 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:26:55.392 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:27:25.482 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:27:25.490 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:27:55.561 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:27:55.568 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:28:25.654 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:28:25.662 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:28:55.723 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:28:55.730 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:29:25.810 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:29:25.819 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:29:56.043 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:29:56.049 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:30:26.130 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:30:26.140 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:30:56.211 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:30:56.218 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:31:26.302 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:31:26.308 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:31:56.372 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:31:56.379 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:32:26.463 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:32:26.472 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:32:56.550 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:32:56.556 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:33:26.639 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:33:26.647 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:33:56.720 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:33:56.732 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:34:26.802 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:34:26.807 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:34:56.897 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:34:56.904 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:35:26.979 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:35:26.985 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:35:57.059 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:35:57.070 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:36:27.147 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:36:27.152 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:36:57.231 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:36:57.238 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:37:27.333 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:37:27.342 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:37:57.415 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:37:57.420 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:38:27.503 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:38:27.511 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:38:57.568 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:38:57.575 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:39:27.657 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:39:27.666 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:39:57.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:39:57.760 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:40:27.843 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:40:27.848 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:40:57.924 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:40:57.931 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:41:28.021 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:41:28.030 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:41:58.101 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:41:58.109 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:42:28.191 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:42:28.196 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:42:58.263 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:42:58.274 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:43:28.351 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:43:28.357 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:43:58.432 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:43:58.439 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:44:28.503 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:44:28.519 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:44:58.584 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:44:58.591 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:45:28.665 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:45:28.672 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:45:58.737 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:45:58.744 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:46:28.935 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:46:28.949 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:46:59.026 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:46:59.036 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:47:29.122 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:47:29.130 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:47:59.202 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:47:59.208 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:48:29.290 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:48:29.296 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:48:59.378 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:48:59.385 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:49:29.460 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:49:29.471 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:49:59.542 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:49:59.550 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:50:29.640 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:50:29.648 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:50:59.713 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:50:59.734 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:51:29.818 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:51:29.832 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:51:59.904 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:51:59.925 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:52:29.995 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:52:30.004 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:53:00.066 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:53:00.083 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:53:30.172 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:53:30.178 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:54:00.248 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:54:00.255 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:54:30.329 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:54:30.337 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:55:00.408 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:55:00.414 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:55:30.588 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:55:30.597 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:56:00.660 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:56:00.667 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:56:30.788 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:56:30.795 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:57:00.861 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:57:00.866 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:57:30.941 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:57:30.947 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:58:01.027 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:58:01.034 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:58:31.152 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:58:31.158 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:59:01.223 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:59:01.243 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:59:31.326 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T09:59:31.332 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:00:00.007 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T09:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T10:00:01.401 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:00:01.404 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:00:31.483 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:00:31.500 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:01:01.579 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:01:01.584 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:01:31.668 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:01:31.673 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:02:01.739 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:02:01.749 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:02:31.844 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:02:31.853 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:03:01.936 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:03:01.944 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:03:32.028 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:03:32.033 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:04:02.091 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:04:02.097 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:04:32.179 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:04:32.186 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:05:02.258 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:05:02.266 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:05:32.365 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:05:32.373 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:06:02.434 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:06:02.440 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:06:32.525 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:06:32.533 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:07:02.615 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:07:02.624 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:07:32.706 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:07:32.711 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:08:02.775 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:08:02.784 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:08:32.864 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:08:32.870 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:09:02.954 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:09:02.961 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:09:33.041 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:09:33.046 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:10:03.114 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:10:03.121 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:10:33.194 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:10:33.209 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:11:03.275 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:11:03.281 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:11:33.372 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:11:33.380 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:12:03.471 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:12:03.480 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:12:33.552 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:12:33.560 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:13:03.622 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:13:03.630 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:13:33.710 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:13:33.716 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:14:03.776 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:14:03.782 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:14:33.855 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:14:33.864 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:15:03.952 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:15:03.959 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:15:34.040 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:15:34.046 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:16:04.121 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:16:04.131 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:16:34.231 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:16:34.243 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:17:04.309 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:17:04.318 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:17:34.397 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:17:34.402 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:18:04.499 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:18:04.515 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:18:34.584 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:18:34.591 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:19:04.663 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:19:04.669 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:19:34.752 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:19:34.758 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:20:04.833 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:20:04.838 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:20:34.912 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:20:34.917 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:21:04.998 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:21:05.004 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:21:35.091 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:21:35.100 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:22:05.171 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:22:05.176 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:22:35.253 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:22:35.259 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:23:05.324 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:23:05.330 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:23:35.404 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:23:35.412 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:24:05.501 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:24:05.507 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:24:35.589 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:24:35.594 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:25:05.663 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:25:05.669 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:25:35.748 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:25:35.758 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:26:05.816 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:26:05.826 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:26:35.892 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:26:35.921 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:27:05.994 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:27:05.999 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:27:36.092 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:27:36.099 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:28:06.162 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:28:06.168 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:28:36.235 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:28:36.243 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:29:06.307 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:29:06.313 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:29:36.402 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:29:36.436 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:30:06.503 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:30:06.516 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:30:36.599 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:30:36.622 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:31:06.688 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:31:06.694 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:31:36.771 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:31:36.781 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:32:06.858 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:32:06.864 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:32:36.955 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:32:36.967 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:33:07.038 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:33:07.047 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:33:37.145 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:33:37.161 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:34:07.231 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:34:07.240 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:34:37.335 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:34:37.342 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:35:07.407 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:35:07.415 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:35:37.509 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:35:37.514 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:36:07.579 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:36:07.587 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:36:37.675 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:36:37.697 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:37:07.772 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:37:07.778 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:37:37.862 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:37:37.869 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:38:07.932 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:38:07.940 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:38:38.013 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:38:38.024 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:39:08.095 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:39:08.103 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:39:38.195 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:39:38.200 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:40:08.263 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:40:08.269 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:40:38.352 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:40:38.358 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:41:08.424 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:41:08.431 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:41:38.515 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:41:38.523 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:42:08.603 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:42:08.609 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:42:38.674 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:42:38.684 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:43:08.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:43:08.766 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:43:38.844 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:43:38.852 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:44:08.937 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:44:08.952 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:44:39.063 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:44:39.072 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:45:09.144 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:45:09.157 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:45:39.240 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:45:39.254 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:46:09.338 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:46:09.345 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:46:39.427 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:46:39.447 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:47:09.583 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:47:09.589 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:47:39.663 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:47:39.670 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:48:09.737 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:48:09.747 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:48:39.815 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:48:39.820 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:49:09.905 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:49:09.911 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:49:40.005 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:49:40.024 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:50:10.085 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:50:10.091 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:50:40.165 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:50:40.189 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:51:10.304 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:51:10.313 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:51:40.374 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:51:40.395 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:52:10.488 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:52:10.494 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:52:40.579 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:52:40.584 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:53:10.666 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:53:10.672 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:53:40.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:53:40.758 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:54:10.827 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:54:10.836 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:54:40.916 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:54:40.921 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:55:11.007 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:55:11.014 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:55:41.094 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:55:41.106 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:56:11.174 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:56:11.180 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:56:41.263 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:56:41.269 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:57:11.336 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:57:11.342 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:57:41.416 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:57:41.424 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:58:11.509 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:58:11.517 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:58:41.589 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:58:41.603 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:59:11.665 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:59:11.671 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:59:41.747 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T10:59:41.753 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:00:00.013 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T10:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T11:00:11.831 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:00:11.840 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:00:41.925 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:00:41.932 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:01:12.015 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:01:12.033 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:01:42.118 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:01:42.126 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:02:12.191 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:02:12.197 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:02:42.268 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:02:42.276 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:03:12.353 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:03:12.360 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:03:42.446 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:03:42.461 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:04:12.539 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:04:12.545 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:04:42.645 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:04:42.652 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:05:12.718 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:05:12.725 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:05:42.797 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:05:42.802 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:06:12.866 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:06:12.873 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:06:42.952 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:06:42.958 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:07:13.039 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:07:13.057 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:07:43.152 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:07:43.158 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:08:13.225 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:08:13.232 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:08:43.306 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:08:43.315 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:09:13.386 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:09:13.395 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:09:43.475 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:09:43.481 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:10:13.554 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:10:13.559 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:10:43.662 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:10:43.668 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:11:13.742 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:11:13.749 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:11:43.826 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:11:43.834 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:12:13.905 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:12:13.911 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:12:43.987 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:12:44.001 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:13:14.089 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:13:14.095 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:13:44.194 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:13:44.208 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:14:14.276 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:14:14.291 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:14:44.366 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:14:44.374 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:15:14.437 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:15:14.444 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:15:44.526 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:15:44.531 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:16:14.593 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:16:14.599 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:16:44.688 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:16:44.694 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:17:14.759 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:17:14.765 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:17:44.857 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:17:44.862 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:18:14.932 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:18:14.939 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:18:45.033 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:18:45.039 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:19:15.105 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:19:15.111 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:19:45.194 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:19:45.200 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:20:15.285 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:20:15.309 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:20:45.398 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:20:45.405 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:21:15.470 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:21:15.482 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:21:45.567 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:21:45.574 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:22:15.640 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:22:15.647 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:22:45.733 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:22:45.746 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:23:15.826 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:23:15.831 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:23:45.916 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:23:45.922 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:24:15.997 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:24:16.002 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:24:46.087 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:24:46.095 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:25:16.158 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:25:16.174 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:25:46.269 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:25:46.284 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:26:16.378 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:26:16.385 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:26:46.466 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:26:46.475 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:27:16.556 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:27:16.567 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:27:46.642 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:27:46.650 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:28:16.725 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:28:16.733 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:28:46.821 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:28:46.828 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:29:16.911 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:29:16.917 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:29:46.994 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:29:47.002 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:30:17.066 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:30:17.073 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:30:47.153 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:30:47.170 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:31:17.234 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:31:17.242 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:31:47.331 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:31:47.337 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:32:17.425 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:32:17.432 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:32:47.525 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:32:47.534 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:33:17.603 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:33:17.608 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:33:47.690 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:33:47.698 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:34:17.775 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:34:17.782 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:34:47.864 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:34:47.871 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:35:17.945 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:35:17.952 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:35:48.025 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:35:48.034 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:36:18.097 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:36:18.105 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:36:48.179 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:36:48.187 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:37:18.255 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:37:18.260 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:37:48.348 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:37:48.356 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:38:18.435 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:38:18.450 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:38:48.541 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:38:48.549 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:39:18.620 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:39:18.626 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:39:48.701 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:39:48.709 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:40:18.768 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:40:18.776 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:40:48.855 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:40:48.866 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:41:19.036 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:41:19.042 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:41:49.133 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:41:49.139 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:42:19.206 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:42:19.221 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:42:49.299 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:42:49.310 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:43:19.382 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:43:19.388 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:43:49.481 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:43:49.487 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:44:19.563 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:44:19.570 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:44:49.657 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:44:49.664 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:45:19.733 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:45:19.749 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:45:49.844 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:45:49.849 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:46:19.909 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:46:20.777 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:46:50.839 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:46:50.854 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:47:20.935 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:47:20.941 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:47:51.042 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:47:51.049 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:48:21.112 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:48:21.118 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:48:51.202 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:48:51.209 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:49:21.308 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:49:21.313 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:49:51.396 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:49:51.402 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:50:21.463 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:50:21.469 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:50:51.566 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:50:51.572 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:51:21.642 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:51:21.648 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:51:51.725 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:51:51.737 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:52:21.799 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:52:21.808 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:52:51.887 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:52:51.892 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:53:21.963 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:53:21.968 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:53:52.061 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:53:52.070 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:54:22.138 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:54:22.144 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:54:52.220 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:54:52.226 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:55:22.290 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:55:22.317 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:55:52.383 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:55:52.394 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:56:22.470 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:56:22.496 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:56:52.583 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:56:52.589 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:57:22.670 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:57:22.676 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:57:52.759 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:57:52.777 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:58:22.843 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:58:22.851 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:58:52.941 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:58:52.948 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:59:23.017 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:59:23.022 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:59:53.107 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T11:59:53.121 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:00:00.001 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T11:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T12:00:23.203 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:00:23.210 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:00:53.309 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:00:53.322 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:01:23.396 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:01:23.403 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:01:53.484 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:01:53.497 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:02:23.559 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:02:23.568 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:02:53.623 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:02:53.631 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:03:23.720 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:03:23.727 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:03:53.794 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:03:53.799 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:04:23.856 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:04:23.862 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:04:53.937 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:04:53.946 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:05:24.020 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:05:24.027 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:05:54.125 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:05:54.136 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:06:24.209 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:06:24.214 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:06:54.283 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:06:54.293 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:07:24.356 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:07:24.371 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:07:54.449 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:07:54.462 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:08:24.541 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:08:24.546 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:08:54.623 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:08:54.631 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:09:24.703 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:09:24.710 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:09:54.782 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:09:54.795 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:10:24.872 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:10:24.878 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:10:54.950 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:10:54.957 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:11:25.024 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:11:25.030 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:11:55.101 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:11:55.107 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:12:25.190 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:12:25.198 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:12:55.285 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:12:55.296 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:13:25.364 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:13:25.368 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:13:55.445 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:13:55.452 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:14:25.516 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:14:25.522 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:14:55.622 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:14:55.636 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:15:25.702 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:15:25.707 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:15:55.800 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:15:55.807 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:16:25.872 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:16:25.883 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:16:55.969 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:16:55.978 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:17:26.060 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:17:26.067 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:17:56.156 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:17:56.164 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:18:26.235 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:18:26.240 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:18:56.333 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:18:56.344 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:19:26.422 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:19:26.431 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:19:56.491 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:19:56.496 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:20:26.563 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:20:26.569 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:20:56.646 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:20:56.659 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:21:26.732 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:21:26.738 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:21:56.823 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:21:56.828 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:22:26.896 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:22:26.903 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:22:56.979 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:22:56.993 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:23:27.066 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:23:27.080 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:23:57.161 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:23:57.167 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:24:27.235 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:24:27.241 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:24:57.323 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:24:57.338 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:25:27.415 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:25:27.420 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:25:57.488 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:25:57.494 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:26:27.569 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:26:27.583 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:26:57.680 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:26:57.685 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:27:27.759 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:27:27.770 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:27:57.857 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:27:57.871 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:28:28.067 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:28:28.074 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:28:58.157 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:28:58.165 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:29:28.239 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:29:28.245 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:29:58.317 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:29:58.323 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:30:28.396 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:30:28.403 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:30:58.493 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:30:58.510 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:31:28.572 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:31:28.577 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:31:58.636 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:31:58.642 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:32:28.710 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:32:28.721 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:32:58.794 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:32:58.799 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:33:28.860 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:33:28.871 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:33:58.941 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:33:58.947 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:34:29.029 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:34:29.035 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:34:59.113 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:34:59.123 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:35:29.191 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:35:29.200 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:35:59.280 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:35:59.285 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:36:29.353 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:36:29.360 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:36:59.433 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:36:59.439 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:37:29.528 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:37:29.533 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:37:59.618 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:37:59.623 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:38:29.687 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:38:29.694 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:38:59.774 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:38:59.780 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:39:29.848 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:39:29.854 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:39:59.934 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:39:59.942 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:40:30.014 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:40:30.021 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:41:00.102 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:41:00.107 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:41:30.191 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:41:30.197 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:42:00.271 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:42:00.280 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:42:30.340 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:42:30.345 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:43:00.415 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:43:00.421 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:43:30.499 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:43:30.506 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:44:00.583 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:44:00.588 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:44:30.656 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:44:30.665 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:45:00.745 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:45:00.751 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:45:30.809 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:45:30.815 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:46:00.889 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:46:00.895 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:46:30.970 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:46:30.980 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:47:01.061 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:47:01.075 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:47:31.150 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:47:31.161 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:48:01.233 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:48:01.241 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:48:31.314 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:48:31.326 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:49:01.405 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:49:01.414 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:49:31.480 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:49:31.486 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:50:01.583 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:50:01.589 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:50:31.648 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:50:31.663 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:51:01.736 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:51:01.743 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:51:31.819 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:51:31.824 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:52:01.898 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:52:01.905 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:52:31.969 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:52:31.976 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:53:02.076 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:53:02.090 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:53:32.159 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:53:32.164 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:54:02.244 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:54:02.258 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:54:32.328 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:54:32.337 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:55:02.430 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:55:02.449 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:55:32.518 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:55:32.523 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:56:02.608 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:56:02.613 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:56:32.681 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:56:32.688 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:57:02.760 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:57:02.766 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:57:32.845 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:57:32.852 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:58:02.930 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:58:02.936 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:58:33.005 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:58:33.014 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:59:03.108 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:59:03.113 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:59:33.189 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T12:59:33.197 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:00:00.003 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T12:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T13:00:03.268 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:00:03.277 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:00:33.350 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:00:33.357 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:01:03.547 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:01:03.552 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:01:33.751 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:01:33.765 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:02:03.844 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:02:03.851 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:02:33.915 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:02:33.926 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:03:04.003 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:03:04.010 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:03:34.083 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:03:34.088 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:04:04.172 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:04:04.180 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:04:34.253 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:04:34.261 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:05:04.354 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:05:04.359 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:05:34.421 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:05:34.438 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:06:04.535 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:06:04.541 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:06:34.615 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:06:34.620 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:07:04.691 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:07:04.698 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:07:34.754 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:07:34.769 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:08:04.841 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:08:04.848 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:08:34.930 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:08:34.936 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:09:05.019 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:09:05.028 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:09:35.098 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:09:35.104 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:10:05.196 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:10:05.202 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:10:35.266 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:10:35.272 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:11:05.349 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:11:05.357 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:11:35.446 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:11:35.456 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:12:05.529 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:12:05.537 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:12:35.643 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:12:35.653 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:13:05.757 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:13:05.764 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:13:35.825 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:13:35.831 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:14:05.909 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:14:05.915 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:14:35.999 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:14:36.006 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:15:06.087 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:15:06.094 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:15:36.159 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:15:36.164 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:16:06.249 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:16:06.254 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:16:36.388 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:16:36.397 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:17:06.486 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:17:06.492 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:17:36.565 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:17:36.580 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:18:06.669 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:18:06.675 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:18:36.734 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:18:36.742 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:19:06.816 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:19:06.823 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:19:36.893 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:19:36.899 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:20:06.972 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:20:06.977 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:20:37.054 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:20:37.064 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:21:07.141 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:21:07.146 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:21:37.243 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:21:37.249 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:22:07.331 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:22:07.337 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:22:37.405 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:22:37.412 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:23:07.511 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:23:07.518 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:23:37.583 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:23:37.603 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:24:07.671 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:24:07.677 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:24:37.749 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:24:37.756 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:25:07.841 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:25:07.848 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:25:37.924 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:25:37.940 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:26:08.021 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:26:08.027 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:26:38.107 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:26:38.112 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:27:08.213 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:27:08.225 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:27:38.284 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:27:38.290 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:28:08.365 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:28:08.372 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:28:38.431 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:28:38.438 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:29:08.501 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:29:08.509 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:29:38.590 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:29:38.598 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:30:08.688 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:30:08.695 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:30:38.762 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:30:38.784 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:31:08.874 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:31:08.881 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:31:38.947 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:31:38.955 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:32:09.050 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:32:09.057 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:32:39.129 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:32:39.136 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:33:09.213 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:33:09.220 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:33:39.291 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:33:39.300 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:34:09.393 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:34:09.398 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:34:39.465 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:34:39.472 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:35:09.553 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:35:09.559 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:35:39.624 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:35:39.632 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:36:09.706 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:36:09.711 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:36:39.785 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:36:39.791 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:37:09.867 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:37:09.876 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:37:39.941 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:37:39.951 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:38:10.024 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:38:10.029 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:38:40.091 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:38:40.106 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:39:10.201 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:39:10.210 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:39:40.270 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:39:40.275 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:40:10.351 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:40:10.356 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:40:40.431 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:40:40.438 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:41:10.511 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:41:10.520 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:41:40.584 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:41:40.590 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:42:10.704 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:42:10.711 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:42:40.777 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:42:40.786 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:43:10.890 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:43:10.905 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:43:40.983 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:43:40.992 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:44:11.064 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:44:11.069 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:44:41.140 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:44:41.148 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:45:11.239 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:45:11.246 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:45:41.335 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:45:41.340 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:46:11.423 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:46:11.430 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:46:41.505 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:46:41.514 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:47:11.618 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:47:11.623 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:47:41.697 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:47:41.710 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:48:11.788 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:48:11.797 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:48:41.869 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:48:41.875 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:49:11.952 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:49:11.956 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:49:42.023 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:49:42.030 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:50:12.109 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:50:12.114 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:50:42.180 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:50:42.187 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:51:12.271 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:51:12.278 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:51:42.359 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:51:42.376 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:52:12.458 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:52:12.464 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:52:42.544 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:52:42.550 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:53:12.624 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:53:12.630 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:53:42.706 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:53:42.712 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:54:12.794 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:54:12.802 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:54:42.886 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:54:42.896 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:55:12.973 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:55:12.979 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:55:43.044 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:55:43.049 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:56:13.117 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:56:13.124 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:56:43.197 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:56:43.211 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:57:13.294 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:57:13.299 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:57:43.385 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:57:43.396 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:58:13.469 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:58:13.476 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:58:43.546 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:58:43.556 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:59:13.629 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:59:13.634 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:59:43.697 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T13:59:43.703 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:00:00.001 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T13:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T14:00:13.785 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:00:13.794 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:00:43.874 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:00:43.878 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:01:13.954 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:01:13.970 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:01:44.035 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:01:44.042 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:02:14.129 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:02:14.134 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:02:44.213 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:02:44.220 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:03:14.301 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:03:14.306 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:03:44.381 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:03:44.386 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:04:14.479 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:04:14.487 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:04:44.552 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:04:44.559 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:05:14.641 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:05:14.648 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:05:44.712 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:05:44.720 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:06:14.793 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:06:14.800 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:06:44.872 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:06:44.879 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:07:14.962 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:07:14.969 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:07:45.042 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:07:45.047 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:08:15.120 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:08:15.128 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:08:45.191 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:08:45.197 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:09:15.271 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:09:15.280 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:09:45.347 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:09:45.352 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:10:15.446 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:10:15.451 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:10:45.517 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:10:45.524 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:11:15.598 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:11:15.604 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:11:45.677 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:11:45.682 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:12:15.757 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:12:15.763 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:12:45.858 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:12:45.866 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:13:15.958 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:13:15.968 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:13:46.026 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:13:46.031 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:14:16.104 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:14:16.110 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:14:46.177 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:14:46.184 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:15:16.252 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:15:16.256 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:15:46.322 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:15:46.327 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:16:16.428 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:16:16.434 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:16:46.507 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:16:46.513 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:17:16.607 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:17:16.620 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:17:46.700 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:17:46.706 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:18:16.781 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:18:16.793 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:18:46.864 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:18:46.871 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:19:16.947 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:19:16.952 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:19:47.035 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:19:47.042 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:20:17.115 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:20:17.125 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:20:47.195 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:20:47.205 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:21:17.267 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:21:17.272 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:21:47.338 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:21:47.343 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:22:17.417 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:22:17.424 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:22:47.525 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:22:47.532 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:23:17.618 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:23:17.626 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:23:47.693 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:23:47.706 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:24:17.794 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:24:17.798 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:24:47.871 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:24:47.878 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:25:17.946 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:25:17.952 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:25:48.041 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:25:48.046 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:26:18.128 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:26:18.138 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:26:48.201 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:26:48.207 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:27:18.280 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:27:18.286 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:27:48.359 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:27:48.364 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:28:18.445 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:28:18.452 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:28:48.535 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:28:48.543 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:29:18.607 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:29:18.612 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:29:48.677 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:29:48.684 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:30:18.758 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:30:18.772 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:30:48.837 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:30:48.843 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:31:18.901 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:31:18.906 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:31:48.991 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:31:49.000 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:32:19.077 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:32:19.083 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:32:49.156 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:32:49.162 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:33:19.244 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:33:19.249 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:33:49.304 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:33:49.314 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:34:19.388 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:34:19.394 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:34:49.487 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:34:49.493 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:35:19.555 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:35:19.561 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:35:49.630 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:35:49.638 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:36:19.715 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:36:19.723 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:36:49.785 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:36:49.790 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:37:19.866 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:37:19.872 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:37:49.948 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:37:49.961 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:38:20.037 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:38:20.042 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:38:50.111 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:38:50.118 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:39:20.185 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:39:20.200 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:39:50.264 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:39:50.269 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:40:20.344 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:40:20.349 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:40:50.430 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:40:50.438 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:41:20.521 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:41:20.527 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:41:50.591 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:41:50.596 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:42:20.674 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:42:20.686 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:42:50.759 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:42:50.764 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:43:20.839 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:43:20.854 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:43:50.925 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:43:50.933 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:44:21.014 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:44:21.021 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:44:51.087 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:44:51.093 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:45:21.161 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:45:21.174 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:45:51.238 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:45:51.249 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:46:21.319 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:46:21.326 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:46:51.391 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:46:51.415 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:47:21.511 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:47:21.527 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:47:51.585 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:47:51.590 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:48:21.673 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:48:21.678 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:48:51.754 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:48:51.766 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:49:21.836 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:49:21.843 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:49:51.911 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:49:51.916 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:50:22.225 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:50:22.231 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:50:52.289 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:50:52.296 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:51:22.368 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:51:22.377 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:51:52.446 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:51:52.454 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:52:22.535 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:52:22.544 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:52:52.618 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:52:52.624 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:53:22.720 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:53:22.725 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:53:52.785 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:53:52.795 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:54:22.875 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:54:22.886 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:54:52.971 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:54:52.980 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:55:23.056 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:55:23.062 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:55:53.136 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:55:53.142 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:56:23.204 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:56:23.212 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:56:53.301 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:56:53.307 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:57:23.383 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:57:23.389 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:57:53.465 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:57:53.481 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:58:23.567 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:58:23.573 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:58:53.633 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:58:53.639 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:59:23.702 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:59:23.709 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:59:53.805 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T14:59:53.813 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:00:00.000 INFO:tasks.ceph.mon.b.smithi018.stderr:2020-07-18T14:59:59.999+0000 7f96d21a8700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 filesystem is offline 2020-07-18T15:00:23.886 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:00:23.890 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:00:53.949 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:00:53.955 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:01:24.029 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:01:24.036 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:01:54.092 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:01:54.099 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:02:24.164 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:02:24.172 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:02:54.254 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:02:54.260 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:03:24.339 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:03:24.344 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:03:54.409 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:03:54.418 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:04:24.500 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:04:24.508 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:04:54.571 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:04:54.578 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:05:24.642 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:05:24.649 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:05:54.730 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:05:54.736 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:06:24.802 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:06:24.807 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:06:54.865 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:06:54.890 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:07:24.955 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:07:24.962 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:07:55.035 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:07:55.048 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:08:25.128 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:08:25.142 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:08:55.220 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:08:55.227 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:09:25.305 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:09:25.311 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:09:55.388 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:09:55.395 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:10:25.473 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:10:25.482 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:10:55.550 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:10:55.557 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:11:25.632 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:11:25.637 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:11:55.728 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:11:55.733 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:12:25.808 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:12:25.815 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:12:55.881 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:12:55.892 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:13:25.960 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:13:25.968 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:13:56.039 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:13:56.051 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:14:26.127 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:14:26.134 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:14:56.204 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:14:56.213 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:15:26.291 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:15:26.298 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:15:56.365 INFO:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:15:56.376 INFO:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-07-18T15:16:02.375 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-07-18T15:16:02.378 DEBUG:teuthology.task.console_log:Killing console logger for smithi047 2020-07-18T15:16:02.380 DEBUG:teuthology.task.console_log:Killing console logger for smithi018 2020-07-18T15:16:02.380 DEBUG:teuthology.task.console_log:Killing console logger for smithi047 2020-07-18T15:16:02.380 DEBUG:teuthology.task.console_log:Killing console logger for smithi018 2020-07-18T15:16:02.381 DEBUG:teuthology.exit:Finished running handlers